2023-10-06T20:48:45.153 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T20:48:45.153 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose -- /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/orig.config.yaml 2023-10-06T20:48:45.210 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T20:48:45.297 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652 branch: rishabh-cephfs-mon description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7414652' kernel: flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-cephfs-mon ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_client_delegate_inos_pct: 50 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 14d8d032507701c76ca224e769ee65a96b660a8c subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 14d8d032507701c76ca224e769ee65a96b660a8c kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-cephfs-mon sha1: 14d8d032507701c76ca224e769ee65a96b660a8c owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7175 sha1: 14d8d032507701c76ca224e769ee65a96b660a8c sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: rishabh-cephfs-mon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 14d8d032507701c76ca224e769ee65a96b660a8c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-06_11:56:52 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T20:48:45.299 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa; will attempt to use it 2023-10-06T20:48:45.299 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks 2023-10-06T20:48:45.300 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T20:48:45.301 INFO:teuthology.task.internal:Checking packages... 2023-10-06T20:48:45.324 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '14d8d032507701c76ca224e769ee65a96b660a8c' 2023-10-06T20:48:45.324 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T20:48:45.324 INFO:teuthology.packaging:ref: None 2023-10-06T20:48:45.324 INFO:teuthology.packaging:tag: None 2023-10-06T20:48:45.324 INFO:teuthology.packaging:branch: rishabh-cephfs-mon 2023-10-06T20:48:45.325 INFO:teuthology.packaging:sha1: 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:48:45.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-cephfs-mon 2023-10-06T20:48:45.542 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6616.g14d8d032 2023-10-06T20:48:45.542 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T20:48:45.550 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T20:48:45.550 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T20:48:45.558 INFO:teuthology.task.internal:Saving configuration 2023-10-06T20:48:45.579 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T20:48:45.585 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T20:48:45.612 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.070021', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:48:45.635 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.072660', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:48:45.654 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.071680', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:48:45.654 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T20:48:45.661 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-06T20:48:45.661 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-06T20:48:45.661 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-06T20:48:45.662 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T20:48:45.760 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5ee5ee8af0>, signals=[15]) 2023-10-06T20:48:45.761 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T20:48:45.767 INFO:teuthology.task.internal:Opening connections... 2023-10-06T20:48:45.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T20:48:45.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:48:45.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2023-10-06T20:48:45.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:48:45.905 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2023-10-06T20:48:45.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:48:45.973 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T20:48:45.978 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-10-06T20:48:46.003 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-10-06T20:48:46.003 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-10-06T20:48:46.063 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:48:46.063 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:48:46.063 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-10-06T20:48:46.063 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.6" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:48:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:48:46.065 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:48:46.065 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:48:46.065 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-10-06T20:48:46.092 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2023-10-06T20:48:46.112 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2023-10-06T20:48:46.113 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2023-10-06T20:48:46.171 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.6" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:48:46.172 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:48:46.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T20:48:46.173 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:48:46.173 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:48:46.173 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:48:46.173 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:48:46.173 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-10-06T20:48:46.191 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-10-06T20:48:46.212 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-10-06T20:48:46.212 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.6" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:48:46.271 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:48:46.272 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:48:46.273 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-10-06T20:48:46.291 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T20:48:46.302 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T20:48:46.307 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T20:48:46.307 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:48:46.310 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:48:46.312 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:48:46.329 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T20:48:46.335 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T20:48:46.336 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:48:46.369 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:48:46.371 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:48:46.398 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T20:48:46.473 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T20:48:46.487 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-06T20:48:46.487 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, timeout 300 2023-10-06T20:48:46.487 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:48:46.488 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:48:46.488 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:48:46.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:48:46.506 DEBUG:teuthology.task.kernel:sha1 for mon.c is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.506 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T20:48:46.507 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-10-06T20:48:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:48:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:48:46.508 DEBUG:teuthology.task.kernel:sha1 for mon.a is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.508 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T20:48:46.509 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T20:48:46.509 DEBUG:teuthology.task.kernel:sha1 for mon.b is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.509 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T20:48:46.509 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2023-10-06T20:48:46.564 INFO:teuthology.orchestra.run.smithi146.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-06T20:48:46.564 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.565 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T20:48:46.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.569 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-06T20:48:46.569 INFO:teuthology.orchestra.run.smithi142.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-06T20:48:46.570 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.570 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T20:48:46.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.571 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.571 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T20:48:46.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:46.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum upgrade -y linux-firmware 2023-10-06T20:48:46.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum upgrade -y linux-firmware 2023-10-06T20:48:46.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum upgrade -y linux-firmware 2023-10-06T20:48:47.808 INFO:teuthology.orchestra.run.smithi146.stdout:Updating Subscription Management repositories. 2023-10-06T20:48:47.808 INFO:teuthology.orchestra.run.smithi146.stdout:Unable to read consumer identity 2023-10-06T20:48:47.941 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-10-06T20:48:47.941 INFO:teuthology.orchestra.run.smithi142.stdout:Unable to read consumer identity 2023-10-06T20:48:48.058 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T20:48:48.059 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-10-06T20:48:49.171 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-06T20:48:49.405 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-06T20:48:49.495 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-06T20:48:50.547 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-06T20:48:51.621 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2023-10-06T20:48:51.830 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 16 MB 00:02 2023-10-06T20:48:54.048 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 365 kB/s | 24 kB 00:00 2023-10-06T20:48:54.819 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-10-06T20:48:54.820 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2023-10-06T20:48:54.820 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-10-06T20:48:54.979 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.c... 2023-10-06T20:48:54.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:55.126 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T20:48:55.126 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T20:48:55.128 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi146.front.sepia.ceph.com to complete... 2023-10-06T20:48:55.347 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 541 kB/s | 24 kB 00:00 2023-10-06T20:48:55.362 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 556 kB/s | 24 kB 00:00 2023-10-06T20:48:56.065 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-10-06T20:48:56.066 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2023-10-06T20:48:56.067 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-10-06T20:48:56.123 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T20:48:56.124 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2023-10-06T20:48:56.124 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T20:48:56.202 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-06T20:48:56.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:56.242 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-06T20:48:56.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:48:56.384 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T20:48:56.384 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T20:48:56.387 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2023-10-06T20:48:56.426 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T20:48:56.426 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T20:48:56.429 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi142.front.sepia.ceph.com to complete... 2023-10-06T20:49:25.120 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06 20:49:25 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T20:49:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:FINISHED --2023-10-06 20:49:25-- 2023-10-06T20:49:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:Total wall clock time: 30s 2023-10-06T20:49:25.121 INFO:teuthology.orchestra.run.smithi146.stderr:Downloaded: 1 files, 363M in 30s (12.2 MB/s) 2023-10-06T20:49:25.122 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.c... 2023-10-06T20:49:25.122 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T20:49:26.699 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06 20:49:26 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T20:49:26.699 INFO:teuthology.orchestra.run.smithi142.stderr:FINISHED --2023-10-06 20:49:26-- 2023-10-06T20:49:26.699 INFO:teuthology.orchestra.run.smithi142.stderr:Total wall clock time: 30s 2023-10-06T20:49:26.699 INFO:teuthology.orchestra.run.smithi142.stderr:Downloaded: 1 files, 363M in 30s (12.1 MB/s) 2023-10-06T20:49:26.701 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-06T20:49:26.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T20:49:26.718 INFO:teuthology.orchestra.run.smithi146.stdout:Verifying... ######################################## 2023-10-06T20:49:26.785 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing... ######################################## 2023-10-06T20:49:26.870 INFO:teuthology.orchestra.run.smithi146.stdout:Updating / installing... 2023-10-06T20:49:28.333 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying... ######################################## 2023-10-06T20:49:28.771 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing... ######################################## 2023-10-06T20:49:29.080 INFO:teuthology.orchestra.run.smithi142.stdout:Updating / installing... 2023-10-06T20:49:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06 20:49:33 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T20:49:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2023-10-06 20:49:33-- 2023-10-06T20:49:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 37s 2023-10-06T20:49:33.440 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 363M in 37s (9.87 MB/s) 2023-10-06T20:49:33.442 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-06T20:49:33.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T20:49:35.117 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying... ######################################## 2023-10-06T20:49:35.187 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing... ######################################## 2023-10-06T20:49:35.279 INFO:teuthology.orchestra.run.smithi031.stdout:Updating / installing... 2023-10-06T20:50:05.079 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T20:50:07.076 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T20:50:15.932 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T20:50:31.115 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T20:50:31.116 DEBUG:teuthology.orchestra.run.smithi142:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:31.182 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T20:50:31.183 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T20:50:31.184 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T20:50:31.185 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T20:50:31.186 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T20:50:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T20:50:31.188 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T20:50:31.189 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T20:50:31.190 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T20:50:31.191 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T20:50:31.192 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T20:50:31.193 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T20:50:31.194 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T20:50:31.195 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T20:50:31.196 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T20:50:31.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T20:50:31.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T20:50:31.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T20:50:31.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T20:50:31.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T20:50:31.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T20:50:31.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T20:50:31.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T20:50:31.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T20:50:31.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T20:50:31.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T20:50:31.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T20:50:31.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T20:50:31.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T20:50:31.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T20:50:31.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T20:50:31.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T20:50:31.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T20:50:31.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T20:50:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T20:50:31.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T20:50:31.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T20:50:31.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T20:50:31.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T20:50:31.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T20:50:31.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T20:50:31.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T20:50:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T20:50:31.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T20:50:31.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T20:50:31.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T20:50:31.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T20:50:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T20:50:31.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T20:50:31.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T20:50:31.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T20:50:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T20:50:31.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T20:50:31.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T20:50:31.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T20:50:31.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T20:50:31.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T20:50:31.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T20:50:31.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T20:50:31.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T20:50:31.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T20:50:31.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T20:50:31.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T20:50:31.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T20:50:31.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T20:50:31.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T20:50:31.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T20:50:31.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T20:50:31.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T20:50:31.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T20:50:31.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T20:50:31.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T20:50:31.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T20:50:31.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T20:50:31.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T20:50:31.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T20:50:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T20:50:31.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T20:50:31.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T20:50:31.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T20:50:31.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T20:50:31.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T20:50:31.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T20:50:31.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T20:50:31.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T20:50:31.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T20:50:31.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T20:50:31.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T20:50:31.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T20:50:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T20:50:31.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T20:50:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T20:50:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T20:50:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T20:50:31.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T20:50:31.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T20:50:31.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T20:50:31.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T20:50:31.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T20:50:31.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T20:50:31.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T20:50:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T20:50:31.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T20:50:31.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T20:50:31.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T20:50:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T20:50:31.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T20:50:31.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T20:50:31.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T20:50:31.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T20:50:31.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T20:50:31.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T20:50:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T20:50:31.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T20:50:31.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T20:50:31.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T20:50:31.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T20:50:31.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T20:50:31.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T20:50:31.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T20:50:31.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T20:50:31.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T20:50:31.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T20:50:31.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T20:50:31.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T20:50:31.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T20:50:31.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T20:50:31.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T20:50:31.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T20:50:31.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T20:50:31.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T20:50:31.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T20:50:31.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T20:50:31.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T20:50:31.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T20:50:31.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T20:50:31.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T20:50:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T20:50:31.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T20:50:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T20:50:31.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T20:50:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T20:50:31.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T20:50:31.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T20:50:31.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T20:50:31.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T20:50:31.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T20:50:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T20:50:31.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T20:50:31.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T20:50:31.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T20:50:31.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T20:50:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T20:50:31.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T20:50:31.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T20:50:31.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T20:50:31.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T20:50:31.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T20:50:31.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T20:50:31.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T20:50:31.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T20:50:31.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T20:50:31.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T20:50:31.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T20:50:31.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T20:50:31.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T20:50:31.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T20:50:31.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T20:50:31.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T20:50:31.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T20:50:31.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T20:50:31.345 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T20:50:31.346 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T20:50:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T20:50:31.348 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T20:50:31.349 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T20:50:31.349 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T20:50:31.349 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T20:50:31.349 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T20:50:31.349 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.350 DEBUG:teuthology.orchestra.run.smithi142:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:31.375 INFO:teuthology.orchestra.run.smithi142.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.375 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T20:50:31.375 INFO:teuthology.orchestra.run.smithi142.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-10-06T20:50:31.376 INFO:teuthology.orchestra.run.smithi142.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.377 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T20:50:31.377 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.377 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-10-06T20:50:31.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2023-10-06T20:50:31.452 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.02 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 123.el8 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Group : System Environment/Base 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 9265450 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2023-10-06T20:50:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Relocations : (not relocatable) 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : Red Hat, Inc. 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : Red Hat, Inc. 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2023-10-06T20:50:31.709 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T20:50:31.710 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T20:50:31.712 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T20:50:31.712 DEBUG:teuthology.orchestra.run.smithi146:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:31.713 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T20:50:31.713 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T20:50:31.780 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T20:50:31.781 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T20:50:31.782 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T20:50:31.783 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T20:50:31.784 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T20:50:31.785 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T20:50:31.786 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T20:50:31.787 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T20:50:31.788 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T20:50:31.789 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T20:50:31.790 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T20:50:31.791 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T20:50:31.792 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T20:50:31.792 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T20:50:31.792 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T20:50:31.792 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T20:50:31.792 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T20:50:31.793 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T20:50:31.793 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T20:50:31.793 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T20:50:31.793 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T20:50:31.793 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T20:50:31.794 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T20:50:31.794 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T20:50:31.794 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T20:50:31.794 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T20:50:31.794 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T20:50:31.795 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T20:50:31.795 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T20:50:31.795 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T20:50:31.795 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T20:50:31.795 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T20:50:31.796 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T20:50:31.797 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T20:50:31.798 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T20:50:31.799 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T20:50:31.800 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T20:50:31.801 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T20:50:31.802 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T20:50:31.803 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T20:50:31.804 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T20:50:31.805 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T20:50:31.806 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T20:50:31.807 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T20:50:31.808 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T20:50:31.809 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T20:50:31.810 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T20:50:31.811 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T20:50:31.812 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T20:50:31.813 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T20:50:31.814 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T20:50:31.815 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T20:50:31.816 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T20:50:31.817 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T20:50:31.818 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T20:50:31.819 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T20:50:31.820 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T20:50:31.821 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T20:50:31.822 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T20:50:31.823 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T20:50:31.824 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T20:50:31.825 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T20:50:31.826 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T20:50:31.827 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T20:50:31.828 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T20:50:31.829 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T20:50:31.830 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T20:50:31.831 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T20:50:31.832 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T20:50:31.833 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T20:50:31.834 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T20:50:31.835 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T20:50:31.836 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T20:50:31.837 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T20:50:31.838 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T20:50:31.839 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T20:50:31.840 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T20:50:31.841 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T20:50:31.842 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T20:50:31.843 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T20:50:31.844 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T20:50:31.845 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T20:50:31.846 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T20:50:31.847 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T20:50:31.848 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T20:50:31.849 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T20:50:31.850 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T20:50:31.851 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T20:50:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T20:50:31.853 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T20:50:31.854 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T20:50:31.855 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T20:50:31.856 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T20:50:31.857 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T20:50:31.858 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T20:50:31.859 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T20:50:31.860 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T20:50:31.861 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T20:50:31.862 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T20:50:31.863 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T20:50:31.864 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T20:50:31.865 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T20:50:31.866 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T20:50:31.867 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T20:50:31.868 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T20:50:31.869 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T20:50:31.870 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T20:50:31.871 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T20:50:31.872 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T20:50:31.873 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T20:50:31.874 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T20:50:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T20:50:31.876 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T20:50:31.877 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T20:50:31.878 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T20:50:31.879 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T20:50:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T20:50:31.881 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T20:50:31.882 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T20:50:31.883 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T20:50:31.884 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T20:50:31.885 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T20:50:31.886 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T20:50:31.887 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T20:50:31.888 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T20:50:31.889 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T20:50:31.890 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T20:50:31.891 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T20:50:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T20:50:31.893 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T20:50:31.894 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T20:50:31.895 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T20:50:31.896 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T20:50:31.897 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T20:50:31.898 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T20:50:31.899 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T20:50:31.900 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T20:50:31.901 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T20:50:31.902 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T20:50:31.903 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T20:50:31.904 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T20:50:31.905 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T20:50:31.906 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T20:50:31.907 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T20:50:31.908 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T20:50:31.909 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T20:50:31.910 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T20:50:31.911 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T20:50:31.912 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T20:50:31.913 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T20:50:31.914 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T20:50:31.915 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T20:50:31.916 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T20:50:31.917 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T20:50:31.918 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T20:50:31.919 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T20:50:31.920 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T20:50:31.921 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T20:50:31.922 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T20:50:31.923 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T20:50:31.924 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T20:50:31.925 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T20:50:31.926 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T20:50:31.927 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T20:50:31.928 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T20:50:31.929 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T20:50:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T20:50:31.931 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T20:50:31.932 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T20:50:31.933 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T20:50:31.934 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T20:50:31.935 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T20:50:31.936 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T20:50:31.937 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T20:50:31.938 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T20:50:31.939 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T20:50:31.940 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T20:50:31.941 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T20:50:31.942 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T20:50:31.943 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T20:50:31.944 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T20:50:31.945 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T20:50:31.946 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T20:50:31.947 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T20:50:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T20:50:31.949 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T20:50:31.950 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T20:50:31.951 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T20:50:31.952 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T20:50:31.953 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T20:50:31.954 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T20:50:31.957 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T20:50:31.957 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T20:50:31.958 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T20:50:31.958 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.958 DEBUG:teuthology.orchestra.run.smithi146:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:31.986 INFO:teuthology.orchestra.run.smithi146.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.986 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T20:50:31.986 INFO:teuthology.orchestra.run.smithi146.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.986 INFO:teuthology.orchestra.run.smithi146.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:31.987 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T20:50:31.988 INFO:teuthology.orchestra.run.smithi146.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:31.988 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-10-06T20:50:31.989 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2023-10-06T20:50:32.066 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2023-10-06T20:50:32.066 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2023-10-06T20:50:32.066 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.02 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 123.el8 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Group : System Environment/Base 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 9265450 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Relocations : (not relocatable) 2023-10-06T20:50:32.067 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : Red Hat, Inc. 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : Red Hat, Inc. 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T20:50:32.068 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T20:50:32.069 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T20:50:32.069 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:32.070 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T20:50:34.291 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2023-10-06T20:50:36.051 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2023-10-06T20:50:36.339 INFO:teuthology.orchestra.run.smithi146.stderr:done 2023-10-06T20:50:36.341 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2023-10-06T20:50:36.361 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.9BfSIluUug 2023-10-06T20:50:36.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9BfSIluUug 2023-10-06T20:50:36.435 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.9BfSIluUug 2023-10-06T20:50:36.592 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.9BfSIluUug is 6KB 2023-10-06T20:50:36.653 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.9BfSIluUug 2023-10-06T20:50:36.680 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T20:50:36.753 INFO:teuthology.orchestra.run.smithi146.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-06T20:50:36.753 INFO:teuthology.orchestra.run.smithi146.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-06T20:50:36.753 INFO:teuthology.orchestra.run.smithi146.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T20:50:36.755 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:36.924 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2023-10-06T20:50:37.803 INFO:teuthology.orchestra.run.smithi142.stderr:done 2023-10-06T20:50:37.815 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2023-10-06T20:50:37.832 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.86y8abuSZn 2023-10-06T20:50:37.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.86y8abuSZn 2023-10-06T20:50:37.944 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.86y8abuSZn 2023-10-06T20:50:38.044 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.86y8abuSZn is 6KB 2023-10-06T20:50:38.092 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.86y8abuSZn 2023-10-06T20:50:38.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T20:50:38.178 INFO:teuthology.orchestra.run.smithi142.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-06T20:50:38.179 INFO:teuthology.orchestra.run.smithi142.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-06T20:50:38.179 INFO:teuthology.orchestra.run.smithi142.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T20:50:38.180 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:38.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2023-10-06T20:50:44.431 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T20:50:44.431 DEBUG:teuthology.orchestra.run.smithi031:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:44.490 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.490 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T20:50:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T20:50:44.492 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T20:50:44.493 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T20:50:44.494 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T20:50:44.495 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T20:50:44.496 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T20:50:44.497 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T20:50:44.498 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T20:50:44.499 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T20:50:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T20:50:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T20:50:44.502 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T20:50:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T20:50:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T20:50:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T20:50:44.507 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T20:50:44.508 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T20:50:44.509 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T20:50:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T20:50:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T20:50:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T20:50:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T20:50:44.514 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T20:50:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T20:50:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T20:50:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T20:50:44.518 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T20:50:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T20:50:44.520 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T20:50:44.521 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T20:50:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T20:50:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T20:50:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T20:50:44.525 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T20:50:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T20:50:44.527 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T20:50:44.528 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T20:50:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T20:50:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T20:50:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T20:50:44.532 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T20:50:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T20:50:44.534 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T20:50:44.535 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T20:50:44.536 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T20:50:44.537 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T20:50:44.538 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T20:50:44.539 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T20:50:44.540 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T20:50:44.541 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T20:50:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T20:50:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T20:50:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T20:50:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T20:50:44.546 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T20:50:44.547 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T20:50:44.548 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T20:50:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T20:50:44.550 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T20:50:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T20:50:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T20:50:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T20:50:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T20:50:44.555 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T20:50:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T20:50:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T20:50:44.558 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T20:50:44.559 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T20:50:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T20:50:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T20:50:44.562 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T20:50:44.563 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T20:50:44.564 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T20:50:44.565 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T20:50:44.566 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T20:50:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T20:50:44.568 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T20:50:44.569 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T20:50:44.570 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T20:50:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T20:50:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T20:50:44.573 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T20:50:44.574 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T20:50:44.575 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T20:50:44.576 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T20:50:44.577 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T20:50:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T20:50:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T20:50:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T20:50:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T20:50:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T20:50:44.583 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T20:50:44.584 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T20:50:44.585 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T20:50:44.586 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T20:50:44.587 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T20:50:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T20:50:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T20:50:44.590 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T20:50:44.591 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T20:50:44.592 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T20:50:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T20:50:44.594 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T20:50:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T20:50:44.596 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T20:50:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T20:50:44.598 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T20:50:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T20:50:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T20:50:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T20:50:44.602 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T20:50:44.603 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T20:50:44.604 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T20:50:44.605 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T20:50:44.606 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T20:50:44.607 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T20:50:44.608 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T20:50:44.609 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T20:50:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T20:50:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T20:50:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T20:50:44.613 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T20:50:44.614 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T20:50:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T20:50:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T20:50:44.617 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T20:50:44.618 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T20:50:44.619 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T20:50:44.620 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T20:50:44.621 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T20:50:44.622 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T20:50:44.623 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T20:50:44.624 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T20:50:44.625 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T20:50:44.626 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T20:50:44.627 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T20:50:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T20:50:44.629 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T20:50:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T20:50:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T20:50:44.632 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T20:50:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T20:50:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T20:50:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T20:50:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T20:50:44.637 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T20:50:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T20:50:44.639 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T20:50:44.640 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T20:50:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T20:50:44.642 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T20:50:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T20:50:44.644 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T20:50:44.645 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T20:50:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T20:50:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T20:50:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T20:50:44.650 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T20:50:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T20:50:44.652 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T20:50:44.653 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T20:50:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T20:50:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T20:50:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T20:50:44.657 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T20:50:44.658 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T20:50:44.659 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T20:50:44.660 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T20:50:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T20:50:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T20:50:44.663 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T20:50:44.664 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T20:50:44.664 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.664 DEBUG:teuthology.orchestra.run.smithi031:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.693 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T20:50:44.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.02 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 123.el8 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2023-10-06T20:50:44.777 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Group : System Environment/Base 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 9265450 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Relocations : (not relocatable) 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : Red Hat, Inc. 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : Red Hat, Inc. 2023-10-06T20:50:44.778 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T20:50:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T20:50:44.780 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T20:50:44.780 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:44.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T20:50:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2023-10-06T20:50:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:done 2023-10-06T20:50:48.176 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-10-06T20:50:48.198 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.JpdMEs5gXB 2023-10-06T20:50:48.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JpdMEs5gXB 2023-10-06T20:50:48.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.JpdMEs5gXB 2023-10-06T20:50:48.419 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.JpdMEs5gXB is 6KB 2023-10-06T20:50:48.466 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.JpdMEs5gXB 2023-10-06T20:50:48.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T20:50:48.561 INFO:teuthology.orchestra.run.smithi031.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-06T20:50:48.561 INFO:teuthology.orchestra.run.smithi031.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-06T20:50:48.561 INFO:teuthology.orchestra.run.smithi031.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T20:50:48.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:50:48.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2023-10-06T20:51:06.950 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2023-10-06T20:51:06.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T20:51:06.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:51:08.338 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T20:51:08.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-10-06T20:51:08.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:51:18.718 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T20:51:18.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T20:51:18.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:51:41.562 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-10-06T20:51:50.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T20:51:50.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:52:08.358 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T20:52:17.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-10-06T20:52:17.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:52:18.723 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T20:52:25.146 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-10-06T20:52:25.571 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-10-06T20:52:25.571 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T20:52:25.572 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2023-10-06T20:52:25.614 INFO:teuthology.orchestra.run.smithi142.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T20:52:25.614 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:52:25.614 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T20:52:25.615 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T20:52:26.615 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T20:52:26.943 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T20:52:27.071 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2023-10-06T20:52:27.111 DEBUG:teuthology.parallel:result is None 2023-10-06T20:52:27.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T20:52:27.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:52:28.074 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T20:52:28.452 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T20:52:28.452 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T20:52:28.452 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T20:52:28.494 INFO:teuthology.orchestra.run.smithi031.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T20:52:28.494 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:52:28.494 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T20:52:28.494 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T20:52:29.495 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T20:52:29.495 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T20:52:29.620 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2023-10-06T20:52:29.645 DEBUG:teuthology.parallel:result is None 2023-10-06T20:52:50.573 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T20:53:02.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T20:53:02.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:53:02.935 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-10-06T20:53:03.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T20:53:03.361 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T20:53:03.362 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-10-06T20:53:03.408 INFO:teuthology.orchestra.run.smithi146.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T20:53:03.408 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:53:03.408 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T20:53:03.408 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T20:53:04.409 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-10-06T20:53:04.409 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T20:53:04.560 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2023-10-06T20:53:04.588 DEBUG:teuthology.parallel:result is None 2023-10-06T20:53:04.588 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T20:53:04.596 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T20:53:04.596 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:53:04.601 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:53:04.605 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:53:04.649 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T20:53:04.655 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T20:53:04.665 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T20:53:04.666 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:53:04.670 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:53:04.675 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:53:04.752 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T20:53:04.759 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T20:53:04.759 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:53:04.764 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:53:04.768 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:53:04.841 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.851 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.856 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.868 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.880 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.884 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:53:04.887 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T20:53:04.894 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T20:53:04.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:53:04.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:53:04.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:53:04.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T20:53:04.982 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T20:53:04.982 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:53:04.986 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:53:04.990 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:53:05.032 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.205 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:05.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:05.360 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T20:53:05.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:53:05.429 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.585 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:05.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:05.724 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T20:53:05.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:53:05.792 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:53:05.938 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:06.011 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:53:06.078 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T20:53:06.078 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:53:06.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-10-06T20:53:06.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2023-10-06T20:53:06.155 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2023-10-06T20:53:06.262 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:53:06.263 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:53:06.301 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:53:06.683 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T20:53:06.692 INFO:teuthology.task.internal:Starting timer... 2023-10-06T20:53:06.692 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T20:53:06.749 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T20:53:06.758 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T20:53:06.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2023-10-06T20:53:06.877 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2023-10-06T20:53:06.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2023-10-06T20:53:07.012 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2023-10-06T20:53:07.017 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2023-10-06T20:53:07.133 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2023-10-06T20:53:07.136 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T20:53:07.137 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2023-10-06T20:53:07.191 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2023-10-06T20:53:07.191 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2023-10-06T20:53:07.244 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2023-10-06T20:53:07.244 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2023-10-06T20:53:07.294 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2023-10-06T20:53:07.294 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2023-10-06T20:53:07.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:53:07.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:53:07.366 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:53:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:53:07.442 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:53:07.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:53:07.513 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T20:53:07.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:53:07.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:53:07.658 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:53:07.731 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T20:53:07.741 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T20:53:07.748 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T20:53:07.883 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T20:53:07.900 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T20:53:07.901 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T21:05:55.283 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2023-10-06T21:05:55.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T21:05:55.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:05:55.378 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T21:05:55.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T21:05:55.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-10-06T21:05:55.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:05:55.579 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-10-06T21:05:55.690 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-10-06T21:05:55.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T21:05:55.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:05:55.780 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-10-06T21:05:55.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-10-06T21:05:55.899 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T21:05:55.909 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T21:05:55.909 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T21:05:55.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:05:55.915 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T21:05:55.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:05:55.919 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T21:05:55.919 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:05:55.990 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T21:05:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T21:05:56.016 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T21:05:56.024 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T21:05:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T21:05:56.050 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T21:05:56.122 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2023-10-06T21:05:56.123 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2023-10-06T21:05:56.139 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2023-10-06T21:05:56.152 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2023-10-06T21:05:56.153 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2023-10-06T21:05:56.168 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2023-10-06T21:05:56.186 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T21:05:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T21:05:56.201 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T21:05:56.220 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T21:05:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T21:05:56.235 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T21:05:56.316 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2023-10-06T21:05:56.317 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T21:05:56.323 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:05:56.323 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-10-06T21:05:56.324 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.324 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.324 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.324 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.325 INFO:teuthology.orchestra.run.smithi146.stderr:bash: ntpq: command not found 2023-10-06T21:05:56.326 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:05:56.326 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T21:05:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.331 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:05:56.332 INFO:teuthology.run_tasks:Running task pexec... 2023-10-06T21:05:56.343 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-06T21:05:56.344 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:05:56.344 DEBUG:teuthology.orchestra.run.smithi031:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T21:05:56.345 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2023-10-06T21:05:56.345 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T21:05:56.345 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi146.front.sepia.ceph.com 2023-10-06T21:05:56.345 DEBUG:teuthology.orchestra.run.smithi146:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T21:05:56.833 INFO:teuthology.orchestra.run.smithi146.stdout:Updating Subscription Management repositories. 2023-10-06T21:05:56.837 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-10-06T21:05:56.866 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T21:05:58.594 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:11 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:05:58.666 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:59 ago on Fri 06 Oct 2023 09:03:59 PM UTC. 2023-10-06T21:05:58.739 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:11 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:00.387 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-10-06T21:06:00.388 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-10-06T21:06:00.388 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout:Resetting modules: 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-10-06T21:06:00.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:00.547 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-10-06T21:06:00.549 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-10-06T21:06:00.549 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2023-10-06T21:06:00.550 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-10-06T21:06:00.550 INFO:teuthology.orchestra.run.smithi146.stdout:Resetting modules: 2023-10-06T21:06:00.550 INFO:teuthology.orchestra.run.smithi146.stdout: container-tools 2023-10-06T21:06:00.550 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:00.551 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-10-06T21:06:00.551 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-10-06T21:06:00.551 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:00.753 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:Resetting modules: 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools 2023-10-06T21:06:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:00.756 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T21:06:00.756 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T21:06:00.756 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:01.622 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:14 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:01.802 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:14 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:02 ago on Fri 06 Oct 2023 09:03:59 PM UTC. 2023-10-06T21:06:07.137 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T21:06:07.254 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-10-06T21:06:07.757 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:07.836 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:08.962 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-10-06T21:06:09.433 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:09 ago on Fri 06 Oct 2023 09:03:59 PM UTC. 2023-10-06T21:06:09.540 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:22 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:09.548 INFO:teuthology.orchestra.run.smithi146.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:11.297 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:23 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:11.320 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-10-06T21:06:11.422 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-10-06T21:06:11.422 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-10-06T21:06:11.422 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-10-06T21:06:11.423 INFO:teuthology.orchestra.run.smithi142.stdout:Installing group/module packages: 2023-10-06T21:06:11.423 INFO:teuthology.orchestra.run.smithi142.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T21:06:11.423 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T21:06:11.424 INFO:teuthology.orchestra.run.smithi142.stdout:Installing module profiles: 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools/common 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools rhel8 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout:Install 10 Packages 2023-10-06T21:06:11.425 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:11.426 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 18 M 2023-10-06T21:06:11.426 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 65 M 2023-10-06T21:06:11.426 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-10-06T21:06:11.493 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T21:06:11.598 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================================================ 2023-10-06T21:06:11.598 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================================================ 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout:Installing group/module packages: 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T21:06:11.599 INFO:teuthology.orchestra.run.smithi031.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout:Installing module profiles: 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools/common 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools rhel8 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================================================ 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout:Install 10 Packages 2023-10-06T21:06:11.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 18 M 2023-10-06T21:06:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 65 M 2023-10-06T21:06:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-10-06T21:06:11.711 INFO:teuthology.orchestra.run.smithi031.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 860 kB/s | 94 kB 00:00 2023-10-06T21:06:11.736 INFO:teuthology.orchestra.run.smithi031.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 185 kB/s | 25 kB 00:00 2023-10-06T21:06:11.762 INFO:teuthology.orchestra.run.smithi031.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 256 kB/s | 41 kB 00:00 2023-10-06T21:06:11.787 INFO:teuthology.orchestra.run.smithi031.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-06T21:06:11.829 INFO:teuthology.orchestra.run.smithi031.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-10-06T21:06:11.854 INFO:teuthology.orchestra.run.smithi031.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-10-06T21:06:11.887 INFO:teuthology.orchestra.run.smithi031.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-10-06T21:06:11.896 INFO:teuthology.orchestra.run.smithi142.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 200 kB/s | 94 kB 00:00 2023-10-06T21:06:11.918 INFO:teuthology.orchestra.run.smithi142.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 50 kB/s | 25 kB 00:00 2023-10-06T21:06:11.943 INFO:teuthology.orchestra.run.smithi142.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 79 kB/s | 41 kB 00:00 2023-10-06T21:06:11.969 INFO:teuthology.orchestra.run.smithi142.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 683 kB/s | 49 kB 00:00 2023-10-06T21:06:12.002 INFO:teuthology.orchestra.run.smithi142.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-06T21:06:12.013 INFO:teuthology.orchestra.run.smithi031.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-10-06T21:06:12.028 INFO:teuthology.orchestra.run.smithi142.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-06T21:06:12.062 INFO:teuthology.orchestra.run.smithi142.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-10-06T21:06:12.171 INFO:teuthology.orchestra.run.smithi031.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-10-06T21:06:12.195 INFO:teuthology.orchestra.run.smithi142.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-10-06T21:06:12.288 INFO:teuthology.orchestra.run.smithi031.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 16 MB/s | 7.0 MB 00:00 2023-10-06T21:06:12.289 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:06:12.289 INFO:teuthology.orchestra.run.smithi031.stdout:Total 26 MB/s | 18 MB 00:00 2023-10-06T21:06:12.337 INFO:teuthology.orchestra.run.smithi142.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-10-06T21:06:12.420 INFO:teuthology.orchestra.run.smithi142.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-10-06T21:06:12.421 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:06:12.421 INFO:teuthology.orchestra.run.smithi142.stdout:Total 18 MB/s | 18 MB 00:00 2023-10-06T21:06:12.632 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-10-06T21:06:12.656 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-10-06T21:06:12.656 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-10-06T21:06:12.754 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-10-06T21:06:12.776 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-10-06T21:06:12.777 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-10-06T21:06:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-10-06T21:06:13.164 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-10-06T21:06:13.180 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================================ 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================================ 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout:Installing group/module packages: 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T21:06:13.278 INFO:teuthology.orchestra.run.smithi146.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout:Installing module profiles: 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: container-tools/common 2023-10-06T21:06:13.279 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout:Enabling module streams: 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout: container-tools rhel8 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================================ 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout:Install 10 Packages 2023-10-06T21:06:13.280 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:13.281 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 18 M 2023-10-06T21:06:13.281 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 65 M 2023-10-06T21:06:13.281 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-10-06T21:06:13.289 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-10-06T21:06:13.294 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-10-06T21:06:13.453 INFO:teuthology.orchestra.run.smithi146.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 237 kB/s | 41 kB 00:00 2023-10-06T21:06:13.479 INFO:teuthology.orchestra.run.smithi146.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 126 kB/s | 25 kB 00:00 2023-10-06T21:06:13.504 INFO:teuthology.orchestra.run.smithi146.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 421 kB/s | 94 kB 00:00 2023-10-06T21:06:13.604 INFO:teuthology.orchestra.run.smithi146.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.1 MB/s | 524 kB 00:00 2023-10-06T21:06:13.630 INFO:teuthology.orchestra.run.smithi146.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 281 kB/s | 49 kB 00:00 2023-10-06T21:06:13.663 INFO:teuthology.orchestra.run.smithi146.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-06T21:06:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-10-06T21:06:13.788 INFO:teuthology.orchestra.run.smithi146.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.3 MB/s | 213 kB 00:00 2023-10-06T21:06:13.862 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-10-06T21:06:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T21:06:13.947 INFO:teuthology.orchestra.run.smithi146.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-10-06T21:06:14.039 INFO:teuthology.orchestra.run.smithi146.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.8 MB/s | 2.2 MB 00:00 2023-10-06T21:06:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T21:06:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 15 MB/s | 7.0 MB 00:00 2023-10-06T21:06:14.132 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:06:14.132 INFO:teuthology.orchestra.run.smithi146.stdout:Total 21 MB/s | 18 MB 00:00 2023-10-06T21:06:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T21:06:14.464 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-10-06T21:06:14.487 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-10-06T21:06:14.487 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-10-06T21:06:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T21:06:14.508 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T21:06:14.753 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T21:06:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T21:06:15.034 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-10-06T21:06:15.039 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-10-06T21:06:15.169 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T21:06:15.620 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-10-06T21:06:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T21:06:15.858 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T21:06:16.004 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T21:06:16.031 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:16.248 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T21:06:16.497 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T21:06:16.760 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T21:06:16.874 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T21:06:16.985 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T21:06:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T21:06:17.240 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T21:06:17.268 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T21:06:17.430 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:17.478 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T21:06:17.703 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:17.710 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T21:06:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T21:06:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T21:06:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T21:06:18.089 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T21:06:18.090 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T21:06:18.091 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T21:06:18.091 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T21:06:18.091 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T21:06:18.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T21:06:18.958 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T21:06:19.153 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T21:06:19.302 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T21:06:19.302 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:19 ago on Fri 06 Oct 2023 09:03:59 PM UTC. 2023-10-06T21:06:19.361 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T21:06:19.486 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:31 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:19.750 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T21:06:19.750 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T21:06:19.750 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T21:06:19.750 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T21:06:19.750 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T21:06:19.751 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T21:06:19.751 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T21:06:19.751 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T21:06:19.751 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T21:06:19.751 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T21:06:21.174 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T21:06:21.174 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:33 ago on Fri 06 Oct 2023 09:03:47 PM UTC. 2023-10-06T21:06:23.114 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2023-10-06T21:06:23.177 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-10-06T21:06:23.328 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:23.328 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T21:06:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:23.330 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-10-06T21:06:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:23.396 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-10-06T21:06:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T21:06:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:06:23.398 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T21:06:23.544 DEBUG:teuthology.parallel:result is None 2023-10-06T21:06:23.610 DEBUG:teuthology.parallel:result is None 2023-10-06T21:06:24.658 INFO:teuthology.orchestra.run.smithi146.stdout:Installed products updated. 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:24.865 INFO:teuthology.orchestra.run.smithi146.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:24.866 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-10-06T21:06:25.315 DEBUG:teuthology.parallel:result is None 2023-10-06T21:06:25.315 INFO:teuthology.run_tasks:Running task install... 2023-10-06T21:06:25.324 DEBUG:teuthology.task.install:project ceph 2023-10-06T21:06:25.324 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'}} 2023-10-06T21:06:25.324 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'} 2023-10-06T21:06:25.324 INFO:teuthology.task.install:Using flavor: default 2023-10-06T21:06:25.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T21:06:25.330 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T21:06:25.331 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T21:06:25.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:06:25.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T21:06:25.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:06:25.334 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T21:06:25.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:06:25.492 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T21:06:25.492 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T21:06:25.503 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T21:06:25.503 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T21:06:25.569 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T21:06:25.569 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T21:06:25.793 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T21:06:25.793 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:06:25.793 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T21:06:25.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T21:06:25.826 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:06:25.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T21:06:25.851 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T21:06:25.851 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:06:25.851 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T21:06:25.867 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T21:06:25.868 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T21:06:25.902 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T21:06:25.902 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T21:06:25.921 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T21:06:25.921 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T21:06:25.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T21:06:26.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T21:06:26.032 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T21:06:26.074 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T21:06:26.113 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T21:06:26.127 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T21:06:26.142 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2023-10-06T21:06:26.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2023-10-06T21:06:26.181 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2023-10-06T21:06:26.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2023-10-06T21:06:26.194 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2023-10-06T21:06:26.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2023-10-06T21:06:26.594 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:26.644 INFO:teuthology.orchestra.run.smithi146.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:26.665 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:27.658 INFO:teuthology.orchestra.run.smithi146.stdout:44 files removed 2023-10-06T21:06:27.659 INFO:teuthology.orchestra.run.smithi142.stdout:44 files removed 2023-10-06T21:06:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:44 files removed 2023-10-06T21:06:27.713 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T21:06:27.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T21:06:27.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T21:06:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:28.166 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:28.239 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-10-06T21:06:29.431 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 206 kB/s | 81 kB 00:00 2023-10-06T21:06:29.658 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 150 kB/s | 81 kB 00:00 2023-10-06T21:06:29.930 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 103 kB/s | 81 kB 00:00 2023-10-06T21:06:30.201 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-10-06T21:06:30.308 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-10-06T21:06:30.548 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-10-06T21:06:30.926 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-10-06T21:06:31.059 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-10-06T21:06:31.315 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 2.0 kB/s | 1.5 kB 00:00 2023-10-06T21:06:31.359 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 8.4 kB/s | 3.5 kB 00:00 2023-10-06T21:06:31.590 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-10-06T21:06:31.676 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 5.8 kB/s | 3.5 kB 00:00 2023-10-06T21:06:32.780 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-06T21:06:33.454 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-10-06T21:06:34.085 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 16 MB 00:02 2023-10-06T21:06:36.853 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 475 kB/s | 24 kB 00:00 2023-10-06T21:06:37.643 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 417 kB/s | 24 kB 00:00 2023-10-06T21:06:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-10-06T21:06:38.424 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 400 kB/s | 24 kB 00:00 2023-10-06T21:06:38.742 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-10-06T21:06:39.736 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-10-06T21:06:46.672 INFO:teuthology.orchestra.run.smithi146.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-10-06T21:06:47.721 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 53 MB 00:01 2023-10-06T21:06:48.877 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-10-06T21:06:58.165 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:06:58.166 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:06:58.181 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T21:06:58.182 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T21:06:58.183 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T21:06:58.185 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T21:06:58.187 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T21:06:58.190 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T21:06:58.192 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T21:06:58.332 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-10-06T21:06:58.345 INFO:teuthology.orchestra.run.smithi146.stdout:=========================================================================================================================== 2023-10-06T21:06:58.345 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2023-10-06T21:06:58.345 INFO:teuthology.orchestra.run.smithi146.stdout:=========================================================================================================================== 2023-10-06T21:06:58.345 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2023-10-06T21:06:58.345 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T21:06:58.346 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T21:06:58.347 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T21:06:58.348 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:06:58.349 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T21:06:58.350 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T21:06:58.351 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T21:06:58.352 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T21:06:58.353 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T21:06:58.354 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T21:06:58.355 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T21:06:58.356 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T21:06:58.357 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T21:06:58.358 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout:Enabling module streams: 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql 10 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout:=========================================================================================================================== 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout:Install 128 Packages 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 3 Packages 2023-10-06T21:06:58.359 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:06:58.360 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 208 M 2023-10-06T21:06:58.360 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-10-06T21:06:58.556 INFO:teuthology.orchestra.run.smithi146.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-10-06T21:06:58.798 INFO:teuthology.orchestra.run.smithi146.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 3.7 MB/s | 918 kB 00:00 2023-10-06T21:06:58.898 INFO:teuthology.orchestra.run.smithi146.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 1.6 MB/s | 167 kB 00:00 2023-10-06T21:06:59.199 INFO:teuthology.orchestra.run.smithi146.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 7.7 MB/s | 2.3 MB 00:00 2023-10-06T21:06:59.489 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:06:59.490 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:06:59.506 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T21:06:59.507 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T21:06:59.508 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T21:06:59.509 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T21:06:59.511 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T21:06:59.515 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T21:06:59.516 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T21:06:59.654 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-10-06T21:06:59.666 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-10-06T21:06:59.666 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-10-06T21:06:59.666 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T21:06:59.667 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T21:06:59.668 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T21:06:59.669 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2023-10-06T21:06:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T21:06:59.671 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T21:06:59.672 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T21:06:59.673 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T21:06:59.674 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T21:06:59.675 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T21:06:59.676 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T21:06:59.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T21:06:59.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T21:06:59.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:06:59.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T21:06:59.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T21:06:59.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T21:06:59.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T21:06:59.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T21:06:59.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T21:06:59.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T21:06:59.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T21:06:59.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T21:06:59.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T21:06:59.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T21:06:59.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T21:06:59.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T21:06:59.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T21:06:59.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T21:06:59.684 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql 10 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout:Install 128 Packages 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 3 Packages 2023-10-06T21:06:59.685 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:06:59.686 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 208 M 2023-10-06T21:06:59.686 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-10-06T21:07:00.147 INFO:teuthology.orchestra.run.smithi142.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2023-10-06T21:07:00.524 INFO:teuthology.orchestra.run.smithi146.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 1.2 MB/s | 1.6 MB 00:01 2023-10-06T21:07:00.971 INFO:teuthology.orchestra.run.smithi146.stdout:(6/131): ceph-common-18.0.0-6616.g14d8d032.el8. 8.9 MB/s | 23 MB 00:02 2023-10-06T21:07:01.138 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:07:01.139 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T21:07:01.154 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T21:07:01.156 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T21:07:01.157 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T21:07:01.158 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T21:07:01.160 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T21:07:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T21:07:01.165 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T21:07:01.289 INFO:teuthology.orchestra.run.smithi142.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 805 kB/s | 918 kB 00:01 2023-10-06T21:07:01.316 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T21:07:01.329 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T21:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T21:07:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T21:07:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T21:07:01.333 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T21:07:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T21:07:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T21:07:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T21:07:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T21:07:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T21:07:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T21:07:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql 10 2023-10-06T21:07:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:Install 128 Packages 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 3 Packages 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 208 M 2023-10-06T21:07:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-10-06T21:07:01.380 INFO:teuthology.orchestra.run.smithi142.stdout:(3/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 3.3 MB/s | 5.4 MB 00:01 2023-10-06T21:07:01.702 INFO:teuthology.orchestra.run.smithi031.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-10-06T21:07:01.747 INFO:teuthology.orchestra.run.smithi142.stdout:(4/131): ceph-immutable-object-cache-18.0.0-661 364 kB/s | 167 kB 00:00 2023-10-06T21:07:01.775 INFO:teuthology.orchestra.run.smithi146.stdout:(7/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-10-06T21:07:01.958 INFO:teuthology.orchestra.run.smithi146.stdout:(8/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 3.2 MB/s | 4.6 MB 00:01 2023-10-06T21:07:02.050 INFO:teuthology.orchestra.run.smithi146.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 266 kB/s | 24 kB 00:00 2023-10-06T21:07:02.386 INFO:teuthology.orchestra.run.smithi031.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 1.3 MB/s | 918 kB 00:00 2023-10-06T21:07:02.456 INFO:teuthology.orchestra.run.smithi142.stdout:(5/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-10-06T21:07:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 1.6 MB/s | 167 kB 00:00 2023-10-06T21:07:03.286 INFO:teuthology.orchestra.run.smithi031.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-10-06T21:07:03.820 INFO:teuthology.orchestra.run.smithi031.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 3.0 MB/s | 1.6 MB 00:00 2023-10-06T21:07:04.748 INFO:teuthology.orchestra.run.smithi142.stdout:(6/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 2.0 MB/s | 4.6 MB 00:02 2023-10-06T21:07:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:(6/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 3.9 MB/s | 4.6 MB 00:01 2023-10-06T21:07:05.114 INFO:teuthology.orchestra.run.smithi142.stdout:(7/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 494 kB/s | 1.6 MB 00:03 2023-10-06T21:07:06.728 INFO:teuthology.orchestra.run.smithi031.stdout:(7/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-10-06T21:07:06.950 INFO:teuthology.orchestra.run.smithi146.stdout:(10/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 3.0 MB/s | 18 MB 00:05 2023-10-06T21:07:07.133 INFO:teuthology.orchestra.run.smithi146.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 1.4 MB/s | 260 kB 00:00 2023-10-06T21:07:07.234 INFO:teuthology.orchestra.run.smithi146.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 310 kB/s | 31 kB 00:00 2023-10-06T21:07:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:(13/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 1.8 MB/s | 840 kB 00:00 2023-10-06T21:07:07.859 INFO:teuthology.orchestra.run.smithi146.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 1.1 MB/s | 183 kB 00:00 2023-10-06T21:07:07.951 INFO:teuthology.orchestra.run.smithi146.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 1.5 MB/s | 139 kB 00:00 2023-10-06T21:07:08.243 INFO:teuthology.orchestra.run.smithi146.stdout:(16/131): libradosstriper1-18.0.0-6616.g14d8d03 1.7 MB/s | 518 kB 00:00 2023-10-06T21:07:10.452 INFO:teuthology.orchestra.run.smithi146.stdout:(17/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 1.6 MB/s | 14 MB 00:08 2023-10-06T21:07:10.486 INFO:teuthology.orchestra.run.smithi031.stdout:(8/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x86 3.3 MB/s | 18 MB 00:05 2023-10-06T21:07:10.578 INFO:teuthology.orchestra.run.smithi146.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 357 kB/s | 45 kB 00:00 2023-10-06T21:07:10.578 INFO:teuthology.orchestra.run.smithi031.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 267 kB/s | 24 kB 00:00 2023-10-06T21:07:10.727 INFO:teuthology.orchestra.run.smithi146.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 845 kB/s | 126 kB 00:00 2023-10-06T21:07:11.002 INFO:teuthology.orchestra.run.smithi146.stdout:(20/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 2.5 MB/s | 6.9 MB 00:02 2023-10-06T21:07:11.056 INFO:teuthology.orchestra.run.smithi146.stdout:(21/131): python3-cephfs-18.0.0-6616.g14d8d032. 599 kB/s | 196 kB 00:00 2023-10-06T21:07:11.145 INFO:teuthology.orchestra.run.smithi146.stdout:(22/131): python3-rados-18.0.0-6616.g14d8d032.e 2.6 MB/s | 371 kB 00:00 2023-10-06T21:07:11.238 INFO:teuthology.orchestra.run.smithi146.stdout:(23/131): python3-rgw-18.0.0-6616.g14d8d032.el8 1.1 MB/s | 107 kB 00:00 2023-10-06T21:07:11.331 INFO:teuthology.orchestra.run.smithi146.stdout:(24/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 94 kB 00:00 2023-10-06T21:07:11.356 INFO:teuthology.orchestra.run.smithi146.stdout:(25/131): python3-rbd-18.0.0-6616.g14d8d032.el8 1.2 MB/s | 364 kB 00:00 2023-10-06T21:07:11.573 INFO:teuthology.orchestra.run.smithi146.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 824 kB/s | 179 kB 00:00 2023-10-06T21:07:11.690 INFO:teuthology.orchestra.run.smithi146.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 198 kB/s | 23 kB 00:00 2023-10-06T21:07:11.890 INFO:teuthology.orchestra.run.smithi146.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 662 kB/s | 132 kB 00:00 2023-10-06T21:07:12.582 INFO:teuthology.orchestra.run.smithi146.stdout:(29/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 2.9 MB/s | 3.6 MB 00:01 2023-10-06T21:07:12.583 INFO:teuthology.orchestra.run.smithi142.stdout:(8/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x86 2.3 MB/s | 18 MB 00:07 2023-10-06T21:07:12.674 INFO:teuthology.orchestra.run.smithi142.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 267 kB/s | 24 kB 00:00 2023-10-06T21:07:13.422 INFO:teuthology.orchestra.run.smithi031.stdout:(10/131): ceph-common-18.0.0-6616.g14d8d032.el8 1.9 MB/s | 23 MB 00:12 2023-10-06T21:07:13.491 INFO:teuthology.orchestra.run.smithi146.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 1.1 MB/s | 1.8 MB 00:01 2023-10-06T21:07:13.847 INFO:teuthology.orchestra.run.smithi031.stdout:(11/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 2.0 MB/s | 14 MB 00:07 2023-10-06T21:07:13.872 INFO:teuthology.orchestra.run.smithi031.stdout:(12/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 579 kB/s | 260 kB 00:00 2023-10-06T21:07:13.930 INFO:teuthology.orchestra.run.smithi031.stdout:(13/131): libcephfs-devel-18.0.0-6616.g14d8d032 373 kB/s | 31 kB 00:00 2023-10-06T21:07:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 1.8 MB/s | 183 kB 00:00 2023-10-06T21:07:14.116 INFO:teuthology.orchestra.run.smithi146.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 388 kB/s | 242 kB 00:00 2023-10-06T21:07:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 1.4 MB/s | 139 kB 00:00 2023-10-06T21:07:14.241 INFO:teuthology.orchestra.run.smithi146.stdout:(32/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 380 kB/s | 47 kB 00:00 2023-10-06T21:07:14.375 INFO:teuthology.orchestra.run.smithi146.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6616.g1 109 kB/s | 15 kB 00:00 2023-10-06T21:07:14.390 INFO:teuthology.orchestra.run.smithi031.stdout:(16/131): libradosstriper1-18.0.0-6616.g14d8d03 2.0 MB/s | 518 kB 00:00 2023-10-06T21:07:14.540 INFO:teuthology.orchestra.run.smithi031.stdout:(17/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 1.2 MB/s | 840 kB 00:00 2023-10-06T21:07:14.640 INFO:teuthology.orchestra.run.smithi031.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 448 kB/s | 45 kB 00:00 2023-10-06T21:07:14.700 INFO:teuthology.orchestra.run.smithi146.stdout:(34/131): ceph-volume-18.0.0-6616.g14d8d032.el8 804 kB/s | 261 kB 00:00 2023-10-06T21:07:14.774 INFO:teuthology.orchestra.run.smithi031.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 944 kB/s | 126 kB 00:00 2023-10-06T21:07:14.976 INFO:teuthology.orchestra.run.smithi146.stdout:(35/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 698 kB/s | 192 kB 00:00 2023-10-06T21:07:15.007 INFO:teuthology.orchestra.run.smithi031.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 842 kB/s | 196 kB 00:00 2023-10-06T21:07:15.067 INFO:teuthology.orchestra.run.smithi146.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-10-06T21:07:15.092 INFO:teuthology.orchestra.run.smithi146.stdout:(37/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 155 kB/s | 18 kB 00:00 2023-10-06T21:07:15.260 INFO:teuthology.orchestra.run.smithi146.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 353 kB/s | 59 kB 00:00 2023-10-06T21:07:15.350 INFO:teuthology.orchestra.run.smithi031.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 1.1 MB/s | 371 kB 00:00 2023-10-06T21:07:15.379 INFO:teuthology.orchestra.run.smithi146.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 986 kB/s | 306 kB 00:00 2023-10-06T21:07:15.410 INFO:teuthology.orchestra.run.smithi146.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 499 kB/s | 75 kB 00:00 2023-10-06T21:07:15.608 INFO:teuthology.orchestra.run.smithi031.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 1.4 MB/s | 364 kB 00:00 2023-10-06T21:07:15.734 INFO:teuthology.orchestra.run.smithi031.stdout:(23/131): python3-rgw-18.0.0-6616.g14d8d032.el8 854 kB/s | 107 kB 00:00 2023-10-06T21:07:15.867 INFO:teuthology.orchestra.run.smithi031.stdout:(24/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 4.7 MB/s | 6.9 MB 00:01 2023-10-06T21:07:15.892 INFO:teuthology.orchestra.run.smithi031.stdout:(25/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 592 kB/s | 94 kB 00:00 2023-10-06T21:07:15.961 INFO:teuthology.orchestra.run.smithi146.stdout:(41/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 3.5 MB/s | 49 MB 00:13 2023-10-06T21:07:15.986 INFO:teuthology.orchestra.run.smithi146.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 76 kB/s | 44 kB 00:00 2023-10-06T21:07:16.017 INFO:teuthology.orchestra.run.smithi031.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 1.4 MB/s | 179 kB 00:00 2023-10-06T21:07:16.093 INFO:teuthology.orchestra.run.smithi031.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 308 kB/s | 23 kB 00:00 2023-10-06T21:07:16.210 INFO:teuthology.orchestra.run.smithi031.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 1.1 MB/s | 132 kB 00:00 2023-10-06T21:07:16.211 INFO:teuthology.orchestra.run.smithi146.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 571 kB/s | 476 kB 00:00 2023-10-06T21:07:16.228 INFO:teuthology.orchestra.run.smithi146.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 136 kB/s | 33 kB 00:00 2023-10-06T21:07:16.262 INFO:teuthology.orchestra.run.smithi146.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-10-06T21:07:16.354 INFO:teuthology.orchestra.run.smithi146.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-10-06T21:07:16.409 INFO:teuthology.orchestra.run.smithi146.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 957 kB/s | 173 kB 00:00 2023-10-06T21:07:16.410 INFO:teuthology.orchestra.run.smithi031.stdout:(29/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 6.6 MB/s | 3.6 MB 00:00 2023-10-06T21:07:16.438 INFO:teuthology.orchestra.run.smithi146.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 976 kB/s | 81 kB 00:00 2023-10-06T21:07:16.464 INFO:teuthology.orchestra.run.smithi146.stdout:(49/131): python3-beautifulsoup4-4.6.3-2.el8.1. 369 kB/s | 185 kB 00:00 2023-10-06T21:07:16.489 INFO:teuthology.orchestra.run.smithi146.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 140 kB/s | 11 kB 00:00 2023-10-06T21:07:16.514 INFO:teuthology.orchestra.run.smithi146.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2023-10-06T21:07:16.581 INFO:teuthology.orchestra.run.smithi146.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 531 kB/s | 49 kB 00:00 2023-10-06T21:07:16.606 INFO:teuthology.orchestra.run.smithi146.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 640 kB/s | 59 kB 00:00 2023-10-06T21:07:16.665 INFO:teuthology.orchestra.run.smithi146.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 723 kB/s | 60 kB 00:00 2023-10-06T21:07:16.707 INFO:teuthology.orchestra.run.smithi146.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-10-06T21:07:16.732 INFO:teuthology.orchestra.run.smithi146.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2023-10-06T21:07:16.765 INFO:teuthology.orchestra.run.smithi146.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 3.9 MB/s | 1.2 MB 00:00 2023-10-06T21:07:16.791 INFO:teuthology.orchestra.run.smithi146.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 403 kB/s | 34 kB 00:00 2023-10-06T21:07:16.833 INFO:teuthology.orchestra.run.smithi146.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 961 kB/s | 64 kB 00:00 2023-10-06T21:07:16.888 INFO:teuthology.orchestra.run.smithi146.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-10-06T21:07:16.917 INFO:teuthology.orchestra.run.smithi146.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2023-10-06T21:07:16.942 INFO:teuthology.orchestra.run.smithi146.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-10-06T21:07:16.967 INFO:teuthology.orchestra.run.smithi146.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 368 kB/s | 29 kB 00:00 2023-10-06T21:07:16.993 INFO:teuthology.orchestra.run.smithi146.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-10-06T21:07:17.026 INFO:teuthology.orchestra.run.smithi146.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-06T21:07:17.051 INFO:teuthology.orchestra.run.smithi146.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-06T21:07:17.077 INFO:teuthology.orchestra.run.smithi146.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-10-06T21:07:17.102 INFO:teuthology.orchestra.run.smithi146.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-06T21:07:17.127 INFO:teuthology.orchestra.run.smithi146.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-10-06T21:07:17.152 INFO:teuthology.orchestra.run.smithi031.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 1.9 MB/s | 1.8 MB 00:00 2023-10-06T21:07:17.153 INFO:teuthology.orchestra.run.smithi146.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-10-06T21:07:17.228 INFO:teuthology.orchestra.run.smithi146.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-10-06T21:07:17.253 INFO:teuthology.orchestra.run.smithi146.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-10-06T21:07:17.319 INFO:teuthology.orchestra.run.smithi031.stdout:(31/131): ceph-mgr-diskprediction-local-18.0.0- 8.1 MB/s | 7.4 MB 00:00 2023-10-06T21:07:17.345 INFO:teuthology.orchestra.run.smithi031.stdout:(32/131): ceph-mgr-modules-core-18.0.0-6616.g14 1.2 MB/s | 242 kB 00:00 2023-10-06T21:07:17.370 INFO:teuthology.orchestra.run.smithi031.stdout:(33/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 947 kB/s | 47 kB 00:00 2023-10-06T21:07:17.387 INFO:teuthology.orchestra.run.smithi146.stdout:(73/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 181 kB/s | 47 kB 00:00 2023-10-06T21:07:17.403 INFO:teuthology.orchestra.run.smithi031.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6616.g1 249 kB/s | 15 kB 00:00 2023-10-06T21:07:17.437 INFO:teuthology.orchestra.run.smithi031.stdout:(35/131): ceph-volume-18.0.0-6616.g14d8d032.el8 3.8 MB/s | 261 kB 00:00 2023-10-06T21:07:17.462 INFO:teuthology.orchestra.run.smithi146.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-10-06T21:07:17.487 INFO:teuthology.orchestra.run.smithi146.stdout:(75/131): python3-jwt-1.6.1-2.el8.noarch.rpm 165 kB/s | 43 kB 00:00 2023-10-06T21:07:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 311 kB/s | 18 kB 00:00 2023-10-06T21:07:17.521 INFO:teuthology.orchestra.run.smithi031.stdout:(37/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 1.6 MB/s | 192 kB 00:00 2023-10-06T21:07:17.579 INFO:teuthology.orchestra.run.smithi031.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-10-06T21:07:17.621 INFO:teuthology.orchestra.run.smithi031.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-10-06T21:07:17.629 INFO:teuthology.orchestra.run.smithi146.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 198 kB/s | 48 kB 00:00 2023-10-06T21:07:17.647 INFO:teuthology.orchestra.run.smithi031.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.0 MB/s | 306 kB 00:00 2023-10-06T21:07:17.680 INFO:teuthology.orchestra.run.smithi031.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 8.0 MB/s | 476 kB 00:00 2023-10-06T21:07:17.706 INFO:teuthology.orchestra.run.smithi031.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-10-06T21:07:17.731 INFO:teuthology.orchestra.run.smithi031.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-10-06T21:07:17.755 INFO:teuthology.orchestra.run.smithi146.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 298 kB/s | 87 kB 00:00 2023-10-06T21:07:17.756 INFO:teuthology.orchestra.run.smithi031.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-10-06T21:07:17.782 INFO:teuthology.orchestra.run.smithi031.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-10-06T21:07:17.807 INFO:teuthology.orchestra.run.smithi031.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-10-06T21:07:17.840 INFO:teuthology.orchestra.run.smithi031.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-10-06T21:07:17.863 INFO:teuthology.orchestra.run.smithi146.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-10-06T21:07:17.865 INFO:teuthology.orchestra.run.smithi031.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-06T21:07:17.891 INFO:teuthology.orchestra.run.smithi031.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-06T21:07:17.905 INFO:teuthology.orchestra.run.smithi146.stdout:(79/131): python3-requests-oauthlib-1.0.0-1.el8 102 kB/s | 43 kB 00:00 2023-10-06T21:07:17.916 INFO:teuthology.orchestra.run.smithi031.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2023-10-06T21:07:17.950 INFO:teuthology.orchestra.run.smithi031.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-10-06T21:07:17.992 INFO:teuthology.orchestra.run.smithi031.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-10-06T21:07:18.006 INFO:teuthology.orchestra.run.smithi146.stdout:(80/131): python3-pycparser-2.14-14.el8.noarch. 290 kB/s | 109 kB 00:00 2023-10-06T21:07:18.017 INFO:teuthology.orchestra.run.smithi031.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 880 kB/s | 59 kB 00:00 2023-10-06T21:07:18.043 INFO:teuthology.orchestra.run.smithi031.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-06T21:07:18.068 INFO:teuthology.orchestra.run.smithi031.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-10-06T21:07:18.073 INFO:teuthology.orchestra.run.smithi146.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 112 kB/s | 19 kB 00:00 2023-10-06T21:07:18.083 INFO:teuthology.orchestra.run.smithi142.stdout:(10/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 1.1 MB/s | 14 MB 00:12 2023-10-06T21:07:18.085 INFO:teuthology.orchestra.run.smithi031.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 370 kB/s | 16 kB 00:00 2023-10-06T21:07:18.102 INFO:teuthology.orchestra.run.smithi031.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-10-06T21:07:18.136 INFO:teuthology.orchestra.run.smithi031.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-06T21:07:18.161 INFO:teuthology.orchestra.run.smithi031.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-10-06T21:07:18.164 INFO:teuthology.orchestra.run.smithi146.stdout:(82/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 1.8 MB/s | 560 kB 00:00 2023-10-06T21:07:18.186 INFO:teuthology.orchestra.run.smithi031.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-10-06T21:07:18.206 INFO:teuthology.orchestra.run.smithi146.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 773 kB/s | 155 kB 00:00 2023-10-06T21:07:18.211 INFO:teuthology.orchestra.run.smithi031.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 479 kB/s | 24 kB 00:00 2023-10-06T21:07:18.237 INFO:teuthology.orchestra.run.smithi031.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 585 kB/s | 29 kB 00:00 2023-10-06T21:07:18.241 INFO:teuthology.orchestra.run.smithi142.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 1.6 MB/s | 260 kB 00:00 2023-10-06T21:07:18.262 INFO:teuthology.orchestra.run.smithi031.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 551 kB/s | 27 kB 00:00 2023-10-06T21:07:18.287 INFO:teuthology.orchestra.run.smithi031.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-10-06T21:07:18.313 INFO:teuthology.orchestra.run.smithi031.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-10-06T21:07:18.316 INFO:teuthology.orchestra.run.smithi142.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 416 kB/s | 31 kB 00:00 2023-10-06T21:07:18.338 INFO:teuthology.orchestra.run.smithi031.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-06T21:07:18.340 INFO:teuthology.orchestra.run.smithi146.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 2.0 MB/s | 559 kB 00:00 2023-10-06T21:07:18.357 INFO:teuthology.orchestra.run.smithi146.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 93 kB/s | 18 kB 00:00 2023-10-06T21:07:18.363 INFO:teuthology.orchestra.run.smithi031.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-10-06T21:07:18.388 INFO:teuthology.orchestra.run.smithi031.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 460 kB/s | 23 kB 00:00 2023-10-06T21:07:18.391 INFO:teuthology.orchestra.run.smithi146.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-10-06T21:07:18.414 INFO:teuthology.orchestra.run.smithi031.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-06T21:07:18.464 INFO:teuthology.orchestra.run.smithi031.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-10-06T21:07:18.581 INFO:teuthology.orchestra.run.smithi031.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-10-06T21:07:18.591 INFO:teuthology.orchestra.run.smithi146.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 361 kB/s | 72 kB 00:00 2023-10-06T21:07:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 244 kB/s | 47 kB 00:00 2023-10-06T21:07:18.625 INFO:teuthology.orchestra.run.smithi146.stdout:(88/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 837 kB/s | 238 kB 00:00 2023-10-06T21:07:18.673 INFO:teuthology.orchestra.run.smithi031.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 464 kB/s | 43 kB 00:00 2023-10-06T21:07:18.691 INFO:teuthology.orchestra.run.smithi142.stdout:(13/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 2.2 MB/s | 840 kB 00:00 2023-10-06T21:07:18.700 INFO:teuthology.orchestra.run.smithi146.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.8 MB/s | 2.3 MB 00:00 2023-10-06T21:07:18.707 INFO:teuthology.orchestra.run.smithi031.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2023-10-06T21:07:18.750 INFO:teuthology.orchestra.run.smithi146.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 521 kB/s | 83 kB 00:00 2023-10-06T21:07:18.790 INFO:teuthology.orchestra.run.smithi031.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 413 kB/s | 48 kB 00:00 2023-10-06T21:07:18.816 INFO:teuthology.orchestra.run.smithi031.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 802 kB/s | 87 kB 00:00 2023-10-06T21:07:18.825 INFO:teuthology.orchestra.run.smithi142.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 1.3 MB/s | 183 kB 00:00 2023-10-06T21:07:18.909 INFO:teuthology.orchestra.run.smithi146.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 211 kB/s | 60 kB 00:00 2023-10-06T21:07:18.916 INFO:teuthology.orchestra.run.smithi031.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 341 kB/s | 43 kB 00:00 2023-10-06T21:07:18.949 INFO:teuthology.orchestra.run.smithi031.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 818 kB/s | 109 kB 00:00 2023-10-06T21:07:18.950 INFO:teuthology.orchestra.run.smithi142.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 1.1 MB/s | 139 kB 00:00 2023-10-06T21:07:19.042 INFO:teuthology.orchestra.run.smithi146.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 287 kB/s | 98 kB 00:00 2023-10-06T21:07:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-10-06T21:07:19.093 INFO:teuthology.orchestra.run.smithi146.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 169 kB/s | 58 kB 00:00 2023-10-06T21:07:19.100 INFO:teuthology.orchestra.run.smithi031.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-10-06T21:07:19.218 INFO:teuthology.orchestra.run.smithi146.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 125 kB/s | 39 kB 00:00 2023-10-06T21:07:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 112 kB/s | 19 kB 00:00 2023-10-06T21:07:19.259 INFO:teuthology.orchestra.run.smithi031.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 978 kB/s | 155 kB 00:00 2023-10-06T21:07:19.284 INFO:teuthology.orchestra.run.smithi142.stdout:(16/131): ceph-common-18.0.0-6616.g14d8d032.el8 1.2 MB/s | 23 MB 00:19 2023-10-06T21:07:19.287 INFO:teuthology.orchestra.run.smithi146.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 882 kB/s | 171 kB 00:00 2023-10-06T21:07:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 6.6 MB/s | 559 kB 00:00 2023-10-06T21:07:19.317 INFO:teuthology.orchestra.run.smithi142.stdout:(17/131): libradosstriper1-18.0.0-6616.g14d8d03 1.4 MB/s | 518 kB 00:00 2023-10-06T21:07:19.335 INFO:teuthology.orchestra.run.smithi146.stdout:(96/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 644 kB 00:00 2023-10-06T21:07:19.343 INFO:teuthology.orchestra.run.smithi031.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2023-10-06T21:07:19.360 INFO:teuthology.orchestra.run.smithi146.stdout:(97/131): NetworkManager-initscripts-updown-1.3 968 kB/s | 138 kB 00:00 2023-10-06T21:07:19.385 INFO:teuthology.orchestra.run.smithi031.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-10-06T21:07:19.409 INFO:teuthology.orchestra.run.smithi142.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 488 kB/s | 45 kB 00:00 2023-10-06T21:07:19.485 INFO:teuthology.orchestra.run.smithi142.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 1.6 MB/s | 126 kB 00:00 2023-10-06T21:07:19.485 INFO:teuthology.orchestra.run.smithi031.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2023-10-06T21:07:19.502 INFO:teuthology.orchestra.run.smithi146.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 659 kB/s | 110 kB 00:00 2023-10-06T21:07:19.544 INFO:teuthology.orchestra.run.smithi146.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 295 kB/s | 54 kB 00:00 2023-10-06T21:07:19.577 INFO:teuthology.orchestra.run.smithi031.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-10-06T21:07:19.603 INFO:teuthology.orchestra.run.smithi031.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 618 kB/s | 72 kB 00:00 2023-10-06T21:07:19.618 INFO:teuthology.orchestra.run.smithi142.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 1.4 MB/s | 196 kB 00:00 2023-10-06T21:07:19.637 INFO:teuthology.orchestra.run.smithi146.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2023-10-06T21:07:19.670 INFO:teuthology.orchestra.run.smithi146.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2023-10-06T21:07:19.728 INFO:teuthology.orchestra.run.smithi031.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 480 kB/s | 60 kB 00:00 2023-10-06T21:07:19.753 INFO:teuthology.orchestra.run.smithi031.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 471 kB/s | 83 kB 00:00 2023-10-06T21:07:19.777 INFO:teuthology.orchestra.run.smithi142.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 2.3 MB/s | 371 kB 00:00 2023-10-06T21:07:19.795 INFO:teuthology.orchestra.run.smithi146.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 244 kB/s | 39 kB 00:00 2023-10-06T21:07:19.846 INFO:teuthology.orchestra.run.smithi146.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-10-06T21:07:19.944 INFO:teuthology.orchestra.run.smithi142.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 2.1 MB/s | 364 kB 00:00 2023-10-06T21:07:19.970 INFO:teuthology.orchestra.run.smithi031.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 266 kB/s | 58 kB 00:00 2023-10-06T21:07:19.995 INFO:teuthology.orchestra.run.smithi031.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 367 kB/s | 98 kB 00:00 2023-10-06T21:07:20.011 INFO:teuthology.orchestra.run.smithi142.stdout:(23/131): python3-rgw-18.0.0-6616.g14d8d032.el8 1.6 MB/s | 107 kB 00:00 2023-10-06T21:07:20.021 INFO:teuthology.orchestra.run.smithi146.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 19 MB/s | 14 MB 00:00 2023-10-06T21:07:20.038 INFO:teuthology.orchestra.run.smithi146.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 818 kB/s | 157 kB 00:00 2023-10-06T21:07:20.071 INFO:teuthology.orchestra.run.smithi146.stdout:(106/131): python3-jinja2-2.10.1-3.el8.noarch.r 1.9 MB/s | 538 kB 00:00 2023-10-06T21:07:20.111 INFO:teuthology.orchestra.run.smithi142.stdout:(24/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 940 kB/s | 94 kB 00:00 2023-10-06T21:07:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 272 kB/s | 39 kB 00:00 2023-10-06T21:07:20.163 INFO:teuthology.orchestra.run.smithi031.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 3.8 MB/s | 644 kB 00:00 2023-10-06T21:07:20.188 INFO:teuthology.orchestra.run.smithi146.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-10-06T21:07:20.196 INFO:teuthology.orchestra.run.smithi031.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.0 MB/s | 171 kB 00:00 2023-10-06T21:07:20.222 INFO:teuthology.orchestra.run.smithi031.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-10-06T21:07:20.251 INFO:teuthology.orchestra.run.smithi146.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 3.3 MB/s | 713 kB 00:00 2023-10-06T21:07:20.264 INFO:teuthology.orchestra.run.smithi031.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-06T21:07:20.280 INFO:teuthology.orchestra.run.smithi146.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 289 kB/s | 27 kB 00:00 2023-10-06T21:07:20.314 INFO:teuthology.orchestra.run.smithi031.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-06T21:07:20.314 INFO:teuthology.orchestra.run.smithi146.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 5.0 MB/s | 320 kB 00:00 2023-10-06T21:07:20.339 INFO:teuthology.orchestra.run.smithi146.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 751 kB/s | 44 kB 00:00 2023-10-06T21:07:20.364 INFO:teuthology.orchestra.run.smithi031.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-10-06T21:07:20.432 INFO:teuthology.orchestra.run.smithi146.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 4.9 MB/s | 457 kB 00:00 2023-10-06T21:07:20.440 INFO:teuthology.orchestra.run.smithi031.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-10-06T21:07:20.498 INFO:teuthology.orchestra.run.smithi031.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 668 kB/s | 39 kB 00:00 2023-10-06T21:07:20.507 INFO:teuthology.orchestra.run.smithi146.stdout:(113/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-10-06T21:07:20.540 INFO:teuthology.orchestra.run.smithi146.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-10-06T21:07:20.565 INFO:teuthology.orchestra.run.smithi031.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-06T21:07:20.599 INFO:teuthology.orchestra.run.smithi146.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-10-06T21:07:20.624 INFO:teuthology.orchestra.run.smithi146.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2023-10-06T21:07:20.650 INFO:teuthology.orchestra.run.smithi146.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2023-10-06T21:07:20.675 INFO:teuthology.orchestra.run.smithi146.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 266 kB/s | 20 kB 00:00 2023-10-06T21:07:20.729 INFO:teuthology.orchestra.run.smithi146.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 987 kB/s | 103 kB 00:00 2023-10-06T21:07:20.740 INFO:teuthology.orchestra.run.smithi031.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-10-06T21:07:20.775 INFO:teuthology.orchestra.run.smithi146.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-10-06T21:07:20.790 INFO:teuthology.orchestra.run.smithi031.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.3 MB/s | 538 kB 00:00 2023-10-06T21:07:20.809 INFO:teuthology.orchestra.run.smithi146.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.4 MB/s | 198 kB 00:00 2023-10-06T21:07:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-10-06T21:07:20.891 INFO:teuthology.orchestra.run.smithi031.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-10-06T21:07:20.941 INFO:teuthology.orchestra.run.smithi031.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 5.6 MB/s | 713 kB 00:00 2023-10-06T21:07:20.943 INFO:teuthology.orchestra.run.smithi146.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-10-06T21:07:20.968 INFO:teuthology.orchestra.run.smithi146.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-10-06T21:07:21.042 INFO:teuthology.orchestra.run.smithi031.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 266 kB/s | 27 kB 00:00 2023-10-06T21:07:21.043 INFO:teuthology.orchestra.run.smithi146.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-10-06T21:07:21.085 INFO:teuthology.orchestra.run.smithi146.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-10-06T21:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-10-06T21:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-10-06T21:07:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 588 kB/s | 44 kB 00:00 2023-10-06T21:07:21.237 INFO:teuthology.orchestra.run.smithi146.stdout:(126/131): postgresql-contrib-10.21-2.module+el 4.1 MB/s | 808 kB 00:00 2023-10-06T21:07:21.295 INFO:teuthology.orchestra.run.smithi146.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 7.2 MB/s | 1.5 MB 00:00 2023-10-06T21:07:21.334 INFO:teuthology.orchestra.run.smithi031.stdout:(112/131): python3-numpy-1.14.3-10.el8.x86_64.r 24 MB/s | 3.7 MB 00:00 2023-10-06T21:07:21.368 INFO:teuthology.orchestra.run.smithi031.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.0 MB/s | 457 kB 00:00 2023-10-06T21:07:21.387 INFO:teuthology.orchestra.run.smithi146.stdout:(128/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-10-06T21:07:21.393 INFO:teuthology.orchestra.run.smithi031.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 3.8 MB/s | 225 kB 00:00 2023-10-06T21:07:21.462 INFO:teuthology.orchestra.run.smithi146.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 10 MB/s | 798 kB 00:00 2023-10-06T21:07:21.469 INFO:teuthology.orchestra.run.smithi031.stdout:(115/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.6 MB/s | 126 kB 00:00 2023-10-06T21:07:21.494 INFO:teuthology.orchestra.run.smithi031.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 509 kB/s | 64 kB 00:00 2023-10-06T21:07:21.519 INFO:teuthology.orchestra.run.smithi031.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-10-06T21:07:21.553 INFO:teuthology.orchestra.run.smithi031.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.7 MB/s | 103 kB 00:00 2023-10-06T21:07:21.578 INFO:teuthology.orchestra.run.smithi142.stdout:(25/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 2.4 MB/s | 3.6 MB 00:01 2023-10-06T21:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-10-06T21:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-10-06T21:07:21.812 INFO:teuthology.orchestra.run.smithi142.stdout:(26/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 2.7 MB/s | 6.9 MB 00:02 2023-10-06T21:07:21.812 INFO:teuthology.orchestra.run.smithi031.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2023-10-06T21:07:21.837 INFO:teuthology.orchestra.run.smithi142.stdout:(27/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 692 kB/s | 179 kB 00:00 2023-10-06T21:07:21.887 INFO:teuthology.orchestra.run.smithi146.stdout:(130/131): librados2-18.0.0-6616.g14d8d032.el8. 5.8 MB/s | 3.7 MB 00:00 2023-10-06T21:07:21.904 INFO:teuthology.orchestra.run.smithi142.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6616.g 251 kB/s | 23 kB 00:00 2023-10-06T21:07:21.938 INFO:teuthology.orchestra.run.smithi142.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 1.3 MB/s | 132 kB 00:00 2023-10-06T21:07:21.946 INFO:teuthology.orchestra.run.smithi031.stdout:(122/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-10-06T21:07:21.979 INFO:teuthology.orchestra.run.smithi031.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-10-06T21:07:22.080 INFO:teuthology.orchestra.run.smithi031.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-10-06T21:07:22.263 INFO:teuthology.orchestra.run.smithi031.stdout:(125/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-10-06T21:07:22.305 INFO:teuthology.orchestra.run.smithi031.stdout:(126/131): postgresql-contrib-10.21-2.module+el 3.5 MB/s | 808 kB 00:00 2023-10-06T21:07:22.472 INFO:teuthology.orchestra.run.smithi031.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 7.2 MB/s | 1.5 MB 00:00 2023-10-06T21:07:22.754 INFO:teuthology.orchestra.run.smithi142.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 2.1 MB/s | 1.8 MB 00:00 2023-10-06T21:07:22.981 INFO:teuthology.orchestra.run.smithi031.stdout:(128/131): librados2-18.0.0-6616.g14d8d032.el8. 5.5 MB/s | 3.7 MB 00:00 2023-10-06T21:07:23.173 INFO:teuthology.orchestra.run.smithi031.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.1 MB/s | 798 kB 00:00 2023-10-06T21:07:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:(31/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 4.6 MB/s | 49 MB 00:10 2023-10-06T21:07:23.414 INFO:teuthology.orchestra.run.smithi142.stdout:(32/131): ceph-mgr-modules-core-18.0.0-6616.g14 368 kB/s | 242 kB 00:00 2023-10-06T21:07:23.439 INFO:teuthology.orchestra.run.smithi142.stdout:(33/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 948 kB/s | 47 kB 00:00 2023-10-06T21:07:23.506 INFO:teuthology.orchestra.run.smithi142.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6616.g1 158 kB/s | 15 kB 00:00 2023-10-06T21:07:23.531 INFO:teuthology.orchestra.run.smithi142.stdout:(35/131): ceph-volume-18.0.0-6616.g14d8d032.el8 2.8 MB/s | 261 kB 00:00 2023-10-06T21:07:23.582 INFO:teuthology.orchestra.run.smithi142.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 363 kB/s | 18 kB 00:00 2023-10-06T21:07:23.690 INFO:teuthology.orchestra.run.smithi031.stdout:(130/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 3.0 MB/s | 3.7 MB 00:01 2023-10-06T21:07:23.732 INFO:teuthology.orchestra.run.smithi142.stdout:(37/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 851 kB/s | 192 kB 00:00 2023-10-06T21:07:23.918 INFO:teuthology.orchestra.run.smithi142.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 911 kB/s | 306 kB 00:00 2023-10-06T21:07:23.949 INFO:teuthology.orchestra.run.smithi142.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 271 kB/s | 59 kB 00:00 2023-10-06T21:07:24.007 INFO:teuthology.orchestra.run.smithi142.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 847 kB/s | 75 kB 00:00 2023-10-06T21:07:24.108 INFO:teuthology.orchestra.run.smithi142.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 435 kB/s | 44 kB 00:00 2023-10-06T21:07:24.208 INFO:teuthology.orchestra.run.smithi142.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-10-06T21:07:24.234 INFO:teuthology.orchestra.run.smithi142.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-10-06T21:07:24.292 INFO:teuthology.orchestra.run.smithi142.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 395 kB/s | 33 kB 00:00 2023-10-06T21:07:24.317 INFO:teuthology.orchestra.run.smithi142.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-10-06T21:07:24.393 INFO:teuthology.orchestra.run.smithi142.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-06T21:07:24.435 INFO:teuthology.orchestra.run.smithi142.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-10-06T21:07:24.485 INFO:teuthology.orchestra.run.smithi142.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 884 kB/s | 81 kB 00:00 2023-10-06T21:07:24.519 INFO:teuthology.orchestra.run.smithi142.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-10-06T21:07:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 211 kB/s | 18 kB 00:00 2023-10-06T21:07:24.661 INFO:teuthology.orchestra.run.smithi142.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 531 kB/s | 49 kB 00:00 2023-10-06T21:07:24.753 INFO:teuthology.orchestra.run.smithi142.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 642 kB/s | 59 kB 00:00 2023-10-06T21:07:24.787 INFO:teuthology.orchestra.run.smithi142.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2023-10-06T21:07:24.829 INFO:teuthology.orchestra.run.smithi142.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-10-06T21:07:24.862 INFO:teuthology.orchestra.run.smithi142.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-10-06T21:07:24.887 INFO:teuthology.orchestra.run.smithi142.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-10-06T21:07:24.929 INFO:teuthology.orchestra.run.smithi142.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 502 kB/s | 34 kB 00:00 2023-10-06T21:07:24.980 INFO:teuthology.orchestra.run.smithi142.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-10-06T21:07:25.005 INFO:teuthology.orchestra.run.smithi142.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 851 kB/s | 64 kB 00:00 2023-10-06T21:07:25.028 INFO:teuthology.orchestra.run.smithi146.stdout:(131/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 3.7 MB 00:03 2023-10-06T21:07:25.032 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:07:25.032 INFO:teuthology.orchestra.run.smithi146.stdout:Total 7.8 MB/s | 208 MB 00:26 2023-10-06T21:07:25.055 INFO:teuthology.orchestra.run.smithi142.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-10-06T21:07:25.073 INFO:teuthology.orchestra.run.smithi142.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-10-06T21:07:25.131 INFO:teuthology.orchestra.run.smithi142.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-10-06T21:07:25.156 INFO:teuthology.orchestra.run.smithi142.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 330 kB/s | 27 kB 00:00 2023-10-06T21:07:25.240 INFO:teuthology.orchestra.run.smithi142.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-10-06T21:07:25.265 INFO:teuthology.orchestra.run.smithi142.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-10-06T21:07:25.316 INFO:teuthology.orchestra.run.smithi142.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 815 kB/s | 61 kB 00:00 2023-10-06T21:07:25.349 INFO:teuthology.orchestra.run.smithi142.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T21:07:25.383 INFO:teuthology.orchestra.run.smithi142.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 344 kB/s | 23 kB 00:00 2023-10-06T21:07:25.408 INFO:teuthology.orchestra.run.smithi142.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-06T21:07:25.550 INFO:teuthology.orchestra.run.smithi142.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 8.7 MB/s | 1.7 MB 00:00 2023-10-06T21:07:25.581 INFO:teuthology.orchestra.run.smithi031.stdout:(131/131): ceph-test-18.0.0-6616.g14d8d032.el8. 3.3 MB/s | 49 MB 00:15 2023-10-06T21:07:25.585 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:07:25.585 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.6 MB/s | 208 MB 00:24 2023-10-06T21:07:25.626 INFO:teuthology.orchestra.run.smithi142.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-10-06T21:07:25.659 INFO:teuthology.orchestra.run.smithi142.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-06T21:07:25.709 INFO:teuthology.orchestra.run.smithi142.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-10-06T21:07:25.760 INFO:teuthology.orchestra.run.smithi142.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-10-06T21:07:25.802 INFO:teuthology.orchestra.run.smithi142.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 930 kB/s | 39 kB 00:00 2023-10-06T21:07:25.868 INFO:teuthology.orchestra.run.smithi142.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-06T21:07:26.011 INFO:teuthology.orchestra.run.smithi142.stdout:(77/131): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-10-06T21:07:26.044 INFO:teuthology.orchestra.run.smithi142.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2023-10-06T21:07:26.069 INFO:teuthology.orchestra.run.smithi142.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-06T21:07:26.095 INFO:teuthology.orchestra.run.smithi142.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-06T21:07:26.137 INFO:teuthology.orchestra.run.smithi142.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-06T21:07:26.179 INFO:teuthology.orchestra.run.smithi142.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 640 kB/s | 27 kB 00:00 2023-10-06T21:07:26.221 INFO:teuthology.orchestra.run.smithi142.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-06T21:07:26.263 INFO:teuthology.orchestra.run.smithi142.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-10-06T21:07:26.582 INFO:teuthology.orchestra.run.smithi142.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 9.8 MB/s | 4.8 MB 00:00 2023-10-06T21:07:26.683 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-10-06T21:07:26.757 INFO:teuthology.orchestra.run.smithi142.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.6 MB/s | 457 kB 00:00 2023-10-06T21:07:26.807 INFO:teuthology.orchestra.run.smithi142.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-10-06T21:07:26.817 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-10-06T21:07:26.817 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-10-06T21:07:26.849 INFO:teuthology.orchestra.run.smithi142.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-10-06T21:07:26.909 INFO:teuthology.orchestra.run.smithi142.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 5.8 MB/s | 3.7 MB 00:00 2023-10-06T21:07:26.933 INFO:teuthology.orchestra.run.smithi142.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.5 MB/s | 126 kB 00:00 2023-10-06T21:07:26.958 INFO:teuthology.orchestra.run.smithi142.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 400 kB/s | 20 kB 00:00 2023-10-06T21:07:26.984 INFO:teuthology.orchestra.run.smithi142.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-06T21:07:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-06T21:07:27.068 INFO:teuthology.orchestra.run.smithi142.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-06T21:07:27.201 INFO:teuthology.orchestra.run.smithi142.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-10-06T21:07:27.243 INFO:teuthology.orchestra.run.smithi142.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-06T21:07:27.281 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-10-06T21:07:27.318 INFO:teuthology.orchestra.run.smithi142.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-10-06T21:07:27.352 INFO:teuthology.orchestra.run.smithi142.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-06T21:07:27.410 INFO:teuthology.orchestra.run.smithi142.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-10-06T21:07:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-10-06T21:07:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-10-06T21:07:27.494 INFO:teuthology.orchestra.run.smithi142.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-10-06T21:07:27.570 INFO:teuthology.orchestra.run.smithi142.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-10-06T21:07:27.595 INFO:teuthology.orchestra.run.smithi142.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 468 kB/s | 47 kB 00:00 2023-10-06T21:07:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-10-06T21:07:27.645 INFO:teuthology.orchestra.run.smithi142.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-10-06T21:07:27.671 INFO:teuthology.orchestra.run.smithi142.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-06T21:07:27.696 INFO:teuthology.orchestra.run.smithi142.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-10-06T21:07:27.721 INFO:teuthology.orchestra.run.smithi142.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-06T21:07:27.747 INFO:teuthology.orchestra.run.smithi142.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-10-06T21:07:27.772 INFO:teuthology.orchestra.run.smithi142.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-06T21:07:27.805 INFO:teuthology.orchestra.run.smithi142.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-10-06T21:07:27.839 INFO:teuthology.orchestra.run.smithi142.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-10-06T21:07:27.884 INFO:teuthology.orchestra.run.smithi142.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 238 kB/s | 19 kB 00:00 2023-10-06T21:07:27.923 INFO:teuthology.orchestra.run.smithi142.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-10-06T21:07:27.957 INFO:teuthology.orchestra.run.smithi142.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 7.6 MB/s | 559 kB 00:00 2023-10-06T21:07:27.982 INFO:teuthology.orchestra.run.smithi142.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-10-06T21:07:28.024 INFO:teuthology.orchestra.run.smithi142.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-10-06T21:07:28.049 INFO:teuthology.orchestra.run.smithi142.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-10-06T21:07:28.091 INFO:teuthology.orchestra.run.smithi142.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-10-06T21:07:28.133 INFO:teuthology.orchestra.run.smithi142.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-10-06T21:07:28.175 INFO:teuthology.orchestra.run.smithi142.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 15 MB/s | 2.3 MB 00:00 2023-10-06T21:07:28.200 INFO:teuthology.orchestra.run.smithi142.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 897 kB/s | 60 kB 00:00 2023-10-06T21:07:28.225 INFO:teuthology.orchestra.run.smithi142.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-06T21:07:28.251 INFO:teuthology.orchestra.run.smithi142.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-06T21:07:28.276 INFO:teuthology.orchestra.run.smithi142.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-10-06T21:07:28.318 INFO:teuthology.orchestra.run.smithi142.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-10-06T21:07:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-10-06T21:07:28.369 INFO:teuthology.orchestra.run.smithi142.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-06T21:07:28.387 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-10-06T21:07:28.392 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-10-06T21:07:28.777 INFO:teuthology.orchestra.run.smithi142.stdout:(128/131): ceph-mgr-diskprediction-local-18.0.0 1.1 MB/s | 7.4 MB 00:06 2023-10-06T21:07:28.836 INFO:teuthology.orchestra.run.smithi142.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-06T21:07:29.049 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-10-06T21:07:29.054 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-10-06T21:07:29.769 INFO:teuthology.orchestra.run.smithi142.stdout:(130/131): librados2-18.0.0-6616.g14d8d032.el8. 2.6 MB/s | 3.7 MB 00:01 2023-10-06T21:07:32.086 INFO:teuthology.orchestra.run.smithi142.stdout:(131/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 3.7 MB 00:03 2023-10-06T21:07:32.089 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:07:32.089 INFO:teuthology.orchestra.run.smithi142.stdout:Total 6.4 MB/s | 208 MB 00:32 2023-10-06T21:07:33.573 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-10-06T21:07:33.728 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-10-06T21:07:33.783 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T21:07:33.861 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-10-06T21:07:33.862 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-10-06T21:07:33.977 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T21:07:34.142 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T21:07:34.310 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T21:07:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-10-06T21:07:34.518 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T21:07:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T21:07:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T21:07:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T21:07:34.991 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T21:07:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T21:07:35.234 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T21:07:35.406 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-10-06T21:07:35.411 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-10-06T21:07:35.503 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T21:07:35.705 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T21:07:35.892 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T21:07:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:36.225 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T21:07:36.330 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:36.515 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:07:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T21:07:36.739 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-06T21:07:36.911 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:36.992 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T21:07:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:37.247 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T21:07:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:07:37.686 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T21:07:37.733 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-06T21:07:37.925 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T21:07:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T21:07:38.116 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T21:07:38.249 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:38.384 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:38.552 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T21:07:38.798 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-06T21:07:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T21:07:38.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-06T21:07:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:39.212 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:39.217 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-06T21:07:39.358 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T21:07:39.377 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:39.535 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T21:07:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:39.716 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T21:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-06T21:07:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-06T21:07:40.193 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-06T21:07:40.254 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T21:07:40.413 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T21:07:40.457 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T21:07:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T21:07:40.625 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T21:07:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T21:07:40.637 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T21:07:40.851 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-06T21:07:41.049 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T21:07:41.182 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T21:07:41.306 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T21:07:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T21:07:41.441 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T21:07:41.508 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T21:07:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T21:07:41.950 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T21:07:42.199 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T21:07:42.358 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T21:07:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T21:07:42.649 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-06T21:07:42.668 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:07:42.867 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-06T21:07:42.874 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:43.032 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:07:43.144 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T21:07:43.207 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:07:43.329 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:07:43.331 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T21:07:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T21:07:43.504 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T21:07:43.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T21:07:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-06T21:07:43.754 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T21:07:43.798 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T21:07:43.898 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-06T21:07:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T21:07:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T21:07:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T21:07:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T21:07:44.481 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T21:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T21:07:44.724 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T21:07:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T21:07:44.882 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T21:07:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T21:07:45.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:45.238 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T21:07:45.398 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T21:07:45.569 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T21:07:45.732 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T21:07:46.062 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T21:07:46.434 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T21:07:46.648 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T21:07:47.159 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T21:07:47.216 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-06T21:07:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T21:07:47.523 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T21:07:47.614 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-06T21:07:47.923 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-06T21:07:48.181 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-06T21:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-06T21:07:48.359 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T21:07:48.428 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T21:07:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-06T21:07:48.630 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T21:07:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-06T21:07:48.848 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T21:07:48.890 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-06T21:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-06T21:07:49.065 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T21:07:49.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-06T21:07:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-06T21:07:49.235 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-06T21:07:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-06T21:07:49.395 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-06T21:07:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T21:07:49.565 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T21:07:49.599 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-06T21:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T21:07:49.652 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T21:07:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T21:07:49.868 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T21:07:49.932 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-06T21:07:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T21:07:50.055 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T21:07:50.146 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-06T21:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-06T21:07:50.194 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T21:07:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T21:07:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-06T21:07:50.429 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T21:07:50.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T21:07:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T21:07:50.668 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-06T21:07:50.853 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-06T21:07:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-06T21:07:51.024 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-06T21:07:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-06T21:07:51.210 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-06T21:07:51.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T21:07:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T21:07:51.388 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T21:07:51.401 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-06T21:07:51.554 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-06T21:07:51.585 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-06T21:07:51.630 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-06T21:07:51.806 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-06T21:07:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-06T21:07:51.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-06T21:07:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-06T21:07:52.038 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-06T21:07:52.140 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-06T21:07:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-06T21:07:52.347 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T21:07:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-06T21:07:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-06T21:07:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-06T21:07:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-06T21:07:52.863 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-06T21:07:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-06T21:07:53.104 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-06T21:07:53.376 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-06T21:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-06T21:07:53.599 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-06T21:07:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-06T21:07:53.967 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-06T21:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-06T21:07:54.216 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-06T21:07:54.217 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-06T21:07:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T21:07:54.417 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-06T21:07:54.469 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-06T21:07:54.692 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-06T21:07:54.747 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-06T21:07:54.826 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-06T21:07:54.880 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-06T21:07:54.959 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T21:07:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-06T21:07:55.078 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-06T21:07:55.165 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T21:07:55.270 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-06T21:07:55.413 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T21:07:55.440 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-06T21:07:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-06T21:07:55.677 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T21:07:55.734 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-06T21:07:55.768 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-06T21:07:55.848 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-06T21:07:55.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-06T21:07:56.026 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-06T21:07:56.029 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-06T21:07:56.258 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-06T21:07:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-06T21:07:56.300 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-06T21:07:56.451 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-06T21:07:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T21:07:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-06T21:07:56.697 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T21:07:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-06T21:07:56.834 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-06T21:07:56.970 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-06T21:07:56.980 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T21:07:57.069 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-06T21:07:57.239 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T21:07:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T21:07:57.394 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T21:07:57.484 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T21:07:57.627 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T21:07:57.668 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-06T21:07:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T21:07:57.845 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T21:07:57.971 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-06T21:07:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T21:07:58.080 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T21:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T21:07:58.186 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-06T21:07:58.279 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T21:07:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T21:07:58.448 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-06T21:07:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T21:07:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T21:07:58.655 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T21:07:58.707 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-06T21:07:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T21:07:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T21:07:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-06T21:07:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T21:07:59.137 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T21:07:59.281 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-06T21:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T21:07:59.322 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T21:07:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-06T21:07:59.530 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T21:07:59.612 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T21:07:59.739 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-06T21:07:59.765 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T21:07:59.815 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T21:07:59.911 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-06T21:07:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T21:08:00.025 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T21:08:00.093 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-06T21:08:00.265 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T21:08:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T21:08:00.435 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-06T21:08:00.500 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T21:08:00.694 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T21:08:00.724 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-06T21:08:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T21:08:00.970 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-06T21:08:01.196 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-06T21:08:01.388 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-06T21:08:01.608 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-06T21:08:01.610 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T21:08:01.796 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T21:08:01.885 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T21:08:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T21:08:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-06T21:08:02.176 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T21:08:02.352 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-06T21:08:02.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T21:08:02.512 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T21:08:02.560 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T21:08:02.596 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T21:08:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T21:08:02.864 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T21:08:02.890 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-06T21:08:02.997 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T21:08:03.073 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-06T21:08:03.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T21:08:03.255 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T21:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T21:08:03.453 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T21:08:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T21:08:03.736 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T21:08:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T21:08:03.954 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T21:08:03.966 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T21:08:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T21:08:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:04.317 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T21:08:04.463 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T21:08:04.543 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T21:08:04.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T21:08:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T21:08:04.943 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T21:08:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:05.141 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T21:08:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T21:08:05.353 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T21:08:05.604 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T21:08:05.827 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T21:08:06.044 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T21:08:06.336 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T21:08:06.571 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T21:08:06.756 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T21:08:06.996 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T21:08:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T21:08:08.607 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T21:08:08.875 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T21:08:09.268 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T21:08:09.494 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:09.549 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T21:08:09.712 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T21:08:10.406 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:10.643 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T21:08:10.687 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:10.811 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:10.828 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:10.922 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T21:08:11.020 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:11.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T21:08:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:16.336 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:17.028 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T21:08:17.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:17.447 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T21:08:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:31.677 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /run 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:08:32.961 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:33.484 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2023-10-06T21:08:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2023-10-06T21:08:34.050 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2023-10-06T21:08:34.199 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2023-10-06T21:08:34.199 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T21:08:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:08:34.200 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:34.200 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:34.278 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:36.298 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:36.501 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:37.852 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /run 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2023-10-06T21:08:37.902 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2023-10-06T21:08:37.903 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2023-10-06T21:08:37.903 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2023-10-06T21:08:37.903 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2023-10-06T21:08:37.903 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:08:38.042 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:38.676 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:38.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T21:08:38.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T21:08:38.816 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:39.085 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:39.273 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:40.011 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T21:08:40.182 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:40.215 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:40.382 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:40.486 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T21:08:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T21:08:40.582 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:40.673 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:40.711 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T21:08:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:41.154 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T21:08:41.202 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T21:08:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:41.426 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:41.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T21:08:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:41.503 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:41.706 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:41.735 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:42.985 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:43.219 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T21:08:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T21:08:43.452 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:43.523 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:43.682 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:43.763 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:43.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:43.887 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:43.948 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:44.262 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:44.382 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:44.443 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T21:08:44.548 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T21:08:44.787 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T21:08:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:45.265 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T21:08:45.438 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T21:08:45.447 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:45.875 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:46.116 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:46.553 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T21:08:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:46.764 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:46.891 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T21:08:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T21:08:47.528 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:47.593 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T21:08:47.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:47.816 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T21:08:49.282 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T21:08:49.485 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:49.546 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T21:08:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:49.713 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:49.802 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T21:08:49.972 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:50.394 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T21:08:50.580 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:50.899 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T21:08:51.290 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-06T21:08:51.511 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 117/134 2023-10-06T21:08:51.768 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-06T21:08:51.963 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T21:08:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:52.217 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T21:08:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-06T21:08:52.668 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T21:08:52.823 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T21:08:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T21:08:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-06T21:08:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 117/134 2023-10-06T21:08:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-06T21:08:54.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T21:08:54.937 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-06T21:08:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T21:08:55.274 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T21:08:55.679 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T21:08:56.013 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T21:08:56.178 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T21:08:56.220 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:56.445 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T21:08:56.555 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:56.593 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:56.831 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T21:08:57.039 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T21:08:57.395 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T21:08:57.653 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-06T21:08:57.762 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T21:08:57.954 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T21:08:57.981 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-06T21:08:58.065 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T21:08:58.192 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 119/134 2023-10-06T21:08:58.296 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T21:08:58.324 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T21:08:58.417 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T21:08:58.521 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T21:08:58.543 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T21:08:58.625 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T21:08:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:58.749 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-06T21:08:58.750 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:08:58.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T21:08:58.977 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:08:58.977 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:08:58.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:59.035 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:08:59.287 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:08:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T21:08:59.487 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:08:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T21:08:59.736 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:08:59.737 INFO:teuthology.orchestra.run.smithi146.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T21:08:59.737 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:08:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T21:09:00.189 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T21:09:00.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T21:09:00.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T21:09:00.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T21:09:00.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T21:09:00.353 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T21:09:00.354 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T21:09:00.355 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-06T21:09:00.356 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-06T21:09:00.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-06T21:09:00.358 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-06T21:09:00.359 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-06T21:09:00.360 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:00.361 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T21:09:00.380 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T21:09:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T21:09:00.697 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T21:09:00.725 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T21:09:00.905 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T21:09:00.928 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T21:09:01.084 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-06T21:09:01.084 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:09:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:09:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:02.020 INFO:teuthology.orchestra.run.smithi031.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T21:09:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:09:02.048 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:09:02.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:09:02.348 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T21:09:02.612 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T21:09:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T21:09:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T21:09:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T21:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T21:09:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T21:09:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T21:09:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T21:09:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T21:09:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T21:09:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T21:09:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T21:09:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-06T21:09:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-06T21:09:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-06T21:09:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-06T21:09:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-06T21:09:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T21:09:02.829 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T21:09:03.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:03.026 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:17 ago on Fri 06 Oct 2023 09:06:46 PM UTC. 2023-10-06T21:09:03.155 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T21:09:03.480 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T21:09:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T21:09:03.674 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T21:09:03.906 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-06T21:09:04.069 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T21:09:04.177 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T21:09:04.369 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T21:09:04.397 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T21:09:04.397 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:04.642 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:04.642 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:09:04.934 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T21:09:05.143 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:05.272 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:17 ago on Fri 06 Oct 2023 09:06:48 PM UTC. 2023-10-06T21:09:05.399 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:05.399 INFO:teuthology.orchestra.run.smithi142.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T21:09:05.399 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:09:06.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T21:09:06.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T21:09:06.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T21:09:06.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T21:09:06.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T21:09:06.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T21:09:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T21:09:06.013 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T21:09:06.013 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T21:09:06.013 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T21:09:06.014 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T21:09:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T21:09:06.016 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-06T21:09:06.017 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-06T21:09:06.018 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-06T21:09:06.019 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-06T21:09:06.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-06T21:09:06.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T21:09:06.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T21:09:06.023 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T21:09:06.023 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T21:09:06.023 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T21:09:06.023 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T21:09:08.693 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T21:09:08.694 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:21 ago on Fri 06 Oct 2023 09:06:47 PM UTC. 2023-10-06T21:09:10.426 INFO:teuthology.orchestra.run.smithi146.stdout:Installed products updated. 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.649 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.650 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: jq-1.6-3.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T21:09:10.652 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T21:09:10.653 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T21:09:10.654 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T21:09:10.655 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T21:09:10.656 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T21:09:10.657 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T21:09:10.658 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T21:09:10.659 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T21:09:10.660 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T21:09:10.661 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T21:09:10.662 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T21:09:10.662 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:09:10.662 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-10-06T21:09:11.509 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:12.012 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-10-06T21:09:12.212 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T21:09:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-3.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T21:09:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T21:09:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T21:09:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T21:09:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T21:09:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T21:09:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T21:09:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T21:09:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:09:12.263 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2023-10-06T21:09:12.447 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.448 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.449 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-3.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.450 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.451 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T21:09:12.452 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.453 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T21:09:12.454 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T21:09:12.455 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T21:09:12.456 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T21:09:12.457 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T21:09:12.458 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T21:09:12.459 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-10-06T21:09:12.459 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-10-06T21:09:12.864 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:13.251 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:13.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:13.400 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T21:09:13.456 INFO:teuthology.orchestra.run.smithi031.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:13.457 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:13.457 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T21:09:13.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:13.748 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T21:09:13.802 INFO:teuthology.orchestra.run.smithi142.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:13.802 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:13.802 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T21:09:13.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:13.944 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T21:09:14.004 INFO:teuthology.orchestra.run.smithi146.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:14.005 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T21:09:14.005 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T21:09:14.005 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T21:09:14.006 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:09:14.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T21:09:14.103 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:09:14.103 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T21:09:14.193 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:09:14.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T21:09:14.287 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T21:09:14.287 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:09:14.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T21:09:14.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T21:09:14.467 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:09:14.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T21:09:14.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T21:09:14.643 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:09:14.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T21:09:14.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T21:09:14.826 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T21:09:14.826 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:09:14.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T21:09:14.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T21:09:15.004 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:09:15.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T21:09:15.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T21:09:15.181 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:09:15.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T21:09:15.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T21:09:15.360 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T21:09:15.360 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:09:15.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T21:09:15.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T21:09:15.539 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:09:15.539 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T21:09:15.607 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T21:09:15.715 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:09:15.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T21:09:15.784 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T21:09:15.892 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-06T21:09:15.985 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_client_delegate_inos_pct': 50, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T21:09:15.985 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:15.985 INFO:tasks.cephadm:Cluster fsid is 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:09:15.986 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-06T21:09:15.986 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.146'} 2023-10-06T21:09:15.986 INFO:tasks.cephadm:First mon is mon.a on smithi031 2023-10-06T21:09:15.986 INFO:tasks.cephadm:First mgr is x 2023-10-06T21:09:15.986 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-06T21:09:15.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2023-10-06T21:09:16.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2023-10-06T21:09:16.129 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2023-10-06T21:09:16.199 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-06T21:09:16.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:16.339 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73047/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6616-g14d8d032', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6616.g14d8d032'}, 'url': 'https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-06 10:46:00.441906', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-cephfs-mon', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-06T21:09:16.481 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref rishabh-cephfs-mon, sha1 14d8d032507701c76ca224e769ee65a96b660a8c from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:09:16.482 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm 2023-10-06T21:09:16.482 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 21:09 /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:17.538 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:18.608 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 21:09 /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:18.608 DEBUG:teuthology.orchestra.run.smithi146:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:19.587 INFO:teuthology.orchestra.run.smithi146.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 21:09 /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:19.587 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:19.639 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:19.690 DEBUG:teuthology.orchestra.run.smithi146:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T21:09:19.755 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c on all hosts... 2023-10-06T21:09:19.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T21:09:19.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T21:09:19.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T21:09:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:09:22.622 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:09:24.283 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:09:52.350 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-10-06T21:09:52.350 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:09:52.351 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:09:52.351 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2023-10-06T21:09:52.351 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:09:52.351 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2023-10-06T21:09:52.351 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-10-06T21:10:12.675 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-10-06T21:10:12.675 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:10:12.675 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:10:12.675 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2023-10-06T21:10:12.675 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:10:12.676 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2023-10-06T21:10:12.676 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-10-06T21:10:18.407 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:10:18.408 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:10:18.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2023-10-06T21:10:18.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2023-10-06T21:10:18.604 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2023-10-06T21:10:18.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2023-10-06T21:10:18.753 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2023-10-06T21:10:18.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2023-10-06T21:10:18.895 INFO:tasks.cephadm:Writing seed config... 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-06T21:10:18.896 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] mds_client_delegate_inos_pct = 50 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-06T21:10:18.897 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-06T21:10:18.898 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-06T21:10:18.899 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-06T21:10:18.900 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-06T21:10:18.900 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-06T21:10:18.900 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-06T21:10:18.900 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-06T21:10:18.900 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:10:18.901 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-06T21:10:18.943 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_client_delegate_inos_pct = 50 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-06T21:10:18.946 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-06T21:10:18.988 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-06T21:10:19.033 INFO:tasks.cephadm:Bootstrapping... 2023-10-06T21:10:19.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c -v bootstrap --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:19.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:10:19.103 INFO:journalctl@ceph.mgr.x.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:10:19.272 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:10:19.273 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c', '-v', 'bootstrap', '--fsid', '9b5d7b0a-648c-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2023-10-06T21:10:19.445 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:10:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-06T21:10:19.446 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2023-10-06T21:10:19.618 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:10:19.618 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2023-10-06T21:10:19.618 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2023-10-06T21:10:19.633 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T21:10:19.633 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T21:10:19.646 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T21:10:19.646 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:10:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2023-10-06T21:10:19.673 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2023-10-06T21:10:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2023-10-06T21:10:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2023-10-06T21:10:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:10:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-06T21:10:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2023-10-06T21:10:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2023-10-06T21:10:19.858 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T21:10:19.858 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T21:10:19.870 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T21:10:19.870 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:10:19.884 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2023-10-06T21:10:19.897 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2023-10-06T21:10:19.897 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2023-10-06T21:10:19.897 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2023-10-06T21:10:19.897 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:19.897 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 139823811242472 on /run/cephadm/9b5d7b0a-648c-11ee-8db5-212e2dc638e7.lock 2023-10-06T21:10:19.898 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 139823811242472 acquired on /run/cephadm/9b5d7b0a-648c-11ee-8db5-212e2dc638e7.lock 2023-10-06T21:10:19.898 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2023-10-06T21:10:19.898 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2023-10-06T21:10:19.899 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2023-10-06T21:10:19.906 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-06T21:10:19.906 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2023-10-06T21:10:19.913 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-06T21:10:19.913 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-06T21:10:19.913 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2023-10-06T21:10:19.920 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-06T21:10:19.920 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-06T21:10:19.920 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T21:10:19.921 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-06T21:10:19.921 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T21:10:19.921 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T21:10:19.922 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2023-10-06T21:10:19.922 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2023-10-06T21:10:19.922 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-06T21:10:19.922 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-06T21:10:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:a951a90ba2bd822dba777faf493a1625bb5a2f5c5042f0fdc315d633d2c600c8 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T21:10:20.398 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-06T21:10:20.399 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Storing signatures 2023-10-06T21:10:21.375 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:10:21.375 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:10:21.375 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2023-10-06T21:10:22.151 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2023-10-06T21:10:22.152 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2023-10-06T21:10:23.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC+dyBlzrT1IBAAMF1lCm7Ko1vmIdOLnJUoVw== 2023-10-06T21:10:23.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC/dyBlUi1+GhAAJLvfsOMup8bqptEfXVE8aw== 2023-10-06T21:10:24.584 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQDAdyBl212MCxAA/IBafYLGA39t4MCtR8/agg== 2023-10-06T21:10:24.585 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2023-10-06T21:10:25.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T21:10:25.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-06T21:10:25.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T21:10:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:10:25.463 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2023-10-06T21:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2023-10-06T21:10:26.602 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-06T21:10:26.810 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-06T21:10:27.058 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target. 2023-10-06T21:10:27.058 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target. 2023-10-06T21:10:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T21:10:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-06T21:10:27.651 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target.wants/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@.service. 2023-10-06T21:10:27.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:27 smithi031 systemd[1]: Starting Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:10:28.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:28 smithi031 podman[97994]: 2023-10-06T21:10:28.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:28 smithi031 bash[97994]: 2773d9dd0714a88ffba6d551dc1cdbc5d896522382a1f71e93ecbc069ad39ca3 2023-10-06T21:10:28.465 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:10:28.465 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-10-06T21:10:28.481 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:10:28.481 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:10:28.481 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-10-06T21:10:28.481 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T21:10:28.481 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2023-10-06T21:10:28.482 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2023-10-06T21:10:28.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:28 smithi031 ceph-mon[98034]: mkfs 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:28.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:28 smithi031 ceph-mon[98034]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:10:28.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:28 smithi031 systemd[1]: Started Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:10:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2023-10-06T21:10:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-06T21:10:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.682038s) 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.018+0000 7f701f8c5700 1 Processor -- start 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.018+0000 7f701f8c5700 1 -- start start 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.019+0000 7f701f8c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.019+0000 7f701f8c5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701807a8a0 con 0x7f701807be70 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.019+0000 7f701d661700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.019+0000 7f701d661700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44838/0 (socket says 172.21.15.31:44838) 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.019+0000 7f701d661700 1 -- 172.21.15.31:0/3974473574 learned_addr learned my addr 172.21.15.31:0/3974473574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.020+0000 7f701d661700 1 -- 172.21.15.31:0/3974473574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701807a9e0 con 0x7f701807be70 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.020+0000 7f701d661700 1 --2- 172.21.15.31:0/3974473574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70140098a0 tx=0x7f7014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638db1f5c907bfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.020+0000 7f700ffff700 1 -- 172.21.15.31:0/3974473574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7014004160 con 0x7f701807be70 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.020+0000 7f700ffff700 1 -- 172.21.15.31:0/3974473574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70140042c0 con 0x7f701807be70 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.020+0000 7f700ffff700 1 -- 172.21.15.31:0/3974473574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70140045a0 con 0x7f701807be70 2023-10-06T21:10:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 -- 172.21.15.31:0/3974473574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 msgr2=0x7f701807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 --2- 172.21.15.31:0/3974473574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70140098a0 tx=0x7f7014009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 -- 172.21.15.31:0/3974473574 shutdown_connections 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 --2- 172.21.15.31:0/3974473574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701807a2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 -- 172.21.15.31:0/3974473574 >> 172.21.15.31:0/3974473574 conn(0x7f7018101ce0 msgr2=0x7f7018104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 -- 172.21.15.31:0/3974473574 shutdown_connections 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.021+0000 7f701f8c5700 1 -- 172.21.15.31:0/3974473574 wait complete. 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.022+0000 7f701f8c5700 1 Processor -- start 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.022+0000 7f701f8c5700 1 -- start start 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701f8c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701f8c5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701819c2a0 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701d661700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701d661700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44848/0 (socket says 172.21.15.31:44848) 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701d661700 1 -- 172.21.15.31:0/844117111 learned_addr learned my addr 172.21.15.31:0/844117111 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.023+0000 7f701d661700 1 -- 172.21.15.31:0/844117111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7014009550 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f701d661700 1 --2- 172.21.15.31:0/844117111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f701807b340 tx=0x7f7014004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7014011630 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014011790 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7014011a30 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701819c4a0 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701819c8c0 con 0x7f701807be70 2023-10-06T21:10:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7014022070 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.024+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f701401c790 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.025+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7018195ce0 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.028+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f701401bcf0 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.105+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f701807ad50 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.105+0000 7f700e7fc700 1 -- 172.21.15.31:0/844117111 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f701401b450 con 0x7f701807be70 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 msgr2=0x7f701819bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 --2- 172.21.15.31:0/844117111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f701807b340 tx=0x7f7014004750 comp rx=0 tx=0).stop 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 shutdown_connections 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 --2- 172.21.15.31:0/844117111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f701807be70 0x7f701819bd60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 >> 172.21.15.31:0/844117111 conn(0x7f7018101ce0 msgr2=0x7f70181035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 shutdown_connections 2023-10-06T21:10:29.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:29.106+0000 7f701f8c5700 1 -- 172.21.15.31:0/844117111 wait complete. 2023-10-06T21:10:29.575 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2023-10-06T21:10:29.575 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2023-10-06T21:10:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:10:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: monmap e1: 1 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} 2023-10-06T21:10:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: fsmap 2023-10-06T21:10:29.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T21:10:29.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: mgrmap e1: no daemons active 2023-10-06T21:10:29.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:29 smithi031 ceph-mon[98034]: from='client.? 172.21.15.31:0/844117111' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-06T21:10:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:30 smithi031 ceph-mon[98034]: from='client.? 172.21.15.31:0/186370021' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T21:10:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:30 smithi031 ceph-mon[98034]: from='client.? 172.21.15.31:0/186370021' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-06T21:10:30.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:30.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:30.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.135+0000 7f5f30fa2700 1 Processor -- start 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.135+0000 7f5f30fa2700 1 -- start start 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.136+0000 7f5f30fa2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.136+0000 7f5f30fa2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c07a8c0 con 0x7f5f2c07be90 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.136+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:30.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44864/0 (socket says 172.21.15.31:44864) 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2a59c700 1 -- 172.21.15.31:0/2283953442 learned_addr learned my addr 172.21.15.31:0/2283953442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2a59c700 1 -- 172.21.15.31:0/2283953442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c07aa00 con 0x7f5f2c07be90 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2a59c700 1 --2- 172.21.15.31:0/2283953442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0098a0 tx=0x7f5f1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d83f1100c8086cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2959a700 1 -- 172.21.15.31:0/2283953442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c004160 con 0x7f5f2c07be90 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2959a700 1 -- 172.21.15.31:0/2283953442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c0042c0 con 0x7f5f2c07be90 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.138+0000 7f5f2959a700 1 -- 172.21.15.31:0/2283953442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c0045a0 con 0x7f5f2c07be90 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 -- 172.21.15.31:0/2283953442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 msgr2=0x7f5f2c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 --2- 172.21.15.31:0/2283953442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0098a0 tx=0x7f5f1c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 -- 172.21.15.31:0/2283953442 shutdown_connections 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 --2- 172.21.15.31:0/2283953442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c07a2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:30.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 -- 172.21.15.31:0/2283953442 >> 172.21.15.31:0/2283953442 conn(0x7f5f2c101ce0 msgr2=0x7f5f2c104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 -- 172.21.15.31:0/2283953442 shutdown_connections 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.139+0000 7f5f30fa2700 1 -- 172.21.15.31:0/2283953442 wait complete. 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.140+0000 7f5f30fa2700 1 Processor -- start 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.140+0000 7f5f30fa2700 1 -- start start 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f30fa2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f30fa2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c19c2d0 con 0x7f5f2c07be90 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44876/0 (socket says 172.21.15.31:44876) 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f2a59c700 1 -- 172.21.15.31:0/186370021 learned_addr learned my addr 172.21.15.31:0/186370021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f2a59c700 1 -- 172.21.15.31:0/186370021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c009550 con 0x7f5f2c07be90 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.141+0000 7f5f2a59c700 1 --2- 172.21.15.31:0/186370021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c004000 tx=0x7f5f1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:30.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c011630 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c011790 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c011a30 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c19c4d0 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c19c8f0 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5f1c022070 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.142+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5f1c01c790 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.143+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c195c30 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.145+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5f1c01bcf0 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.219+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5f2c07af80 con 0x7f5f2c07be90 2023-10-06T21:10:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.223+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5f1c01b450 con 0x7f5f2c07be90 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.224+0000 7f5f1b7fe700 1 -- 172.21.15.31:0/186370021 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5f1c01be90 con 0x7f5f2c07be90 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 msgr2=0x7f5f2c19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 --2- 172.21.15.31:0/186370021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c004000 tx=0x7f5f1c004750 comp rx=0 tx=0).stop 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 shutdown_connections 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 --2- 172.21.15.31:0/186370021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c07be90 0x7f5f2c19bd90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 >> 172.21.15.31:0/186370021 conn(0x7f5f2c101ce0 msgr2=0x7f5f2c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 shutdown_connections 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:30.225+0000 7f5f30fa2700 1 -- 172.21.15.31:0/186370021 wait complete. 2023-10-06T21:10:30.663 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.201+0000 7f8b47e80700 1 Processor -- start 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.201+0000 7f8b47e80700 1 -- start start 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.201+0000 7f8b47e80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.202+0000 7f8b47e80700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401070f0 con 0x7f8b40106700 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.202+0000 7f8b45c1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:31.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.202+0000 7f8b45c1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44878/0 (socket says 172.21.15.31:44878) 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.202+0000 7f8b45c1c700 1 -- 172.21.15.31:0/347750248 learned_addr learned my addr 172.21.15.31:0/347750248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.202+0000 7f8b45c1c700 1 -- 172.21.15.31:0/347750248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40107900 con 0x7f8b40106700 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b45c1c700 1 --2- 172.21.15.31:0/347750248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0098a0 tx=0x7f8b3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db2d496987b18c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b44c1a700 1 -- 172.21.15.31:0/347750248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b3c004160 con 0x7f8b40106700 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b44c1a700 1 -- 172.21.15.31:0/347750248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b3c0042c0 con 0x7f8b40106700 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b44c1a700 1 -- 172.21.15.31:0/347750248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b3c011940 con 0x7f8b40106700 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b47e80700 1 -- 172.21.15.31:0/347750248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 msgr2=0x7f8b40106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b47e80700 1 --2- 172.21.15.31:0/347750248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0098a0 tx=0x7f8b3c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b47e80700 1 -- 172.21.15.31:0/347750248 shutdown_connections 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b47e80700 1 --2- 172.21.15.31:0/347750248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b40106b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.203+0000 7f8b47e80700 1 -- 172.21.15.31:0/347750248 >> 172.21.15.31:0/347750248 conn(0x7f8b40101ce0 msgr2=0x7f8b40104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:31.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.204+0000 7f8b47e80700 1 -- 172.21.15.31:0/347750248 shutdown_connections 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.204+0000 7f8b47e80700 1 -- 172.21.15.31:0/347750248 wait complete. 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b47e80700 1 Processor -- start 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b47e80700 1 -- start start 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b47e80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b47e80700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4019a820 con 0x7f8b40106700 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b45c1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b45c1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44888/0 (socket says 172.21.15.31:44888) 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.205+0000 7f8b45c1c700 1 -- 172.21.15.31:0/918773802 learned_addr learned my addr 172.21.15.31:0/918773802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b45c1c700 1 -- 172.21.15.31:0/918773802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c009550 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b45c1c700 1 --2- 172.21.15.31:0/918773802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000c00 tx=0x7f8b3c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b3c0036a0 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b3c003c70 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4019aa20 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b3c0193f0 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4019ae40 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.206+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8b3c019920 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.207+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8b3c01a3d0 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.207+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b400624e0 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.210+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8b3c029060 con 0x7f8b40106700 2023-10-06T21:10:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.284+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8b4010ab30 con 0x7f8b40106700 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.285+0000 7f8b36ffd700 1 -- 172.21.15.31:0/918773802 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8b3c033090 con 0x7f8b40106700 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.285+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 msgr2=0x7f8b4019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.285+0000 7f8b47e80700 1 --2- 172.21.15.31:0/918773802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000c00 tx=0x7f8b3c011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.286+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 shutdown_connections 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.286+0000 7f8b47e80700 1 --2- 172.21.15.31:0/918773802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b40106700 0x7f8b4019a2e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.286+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 >> 172.21.15.31:0/918773802 conn(0x7f8b40101ce0 msgr2=0x7f8b40102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.286+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 shutdown_connections 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:31.286+0000 7f8b47e80700 1 -- 172.21.15.31:0/918773802 wait complete. 2023-10-06T21:10:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2023-10-06T21:10:31.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:31 smithi031 ceph-mon[98034]: from='client.? 172.21.15.31:0/918773802' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:32.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:31 smithi031 systemd[1]: Stopping Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:10:32.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98010]: 2023-10-06T21:10:32.201+0000 7fcd985e9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:10:32.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98010]: 2023-10-06T21:10:32.201+0000 7fcd985e9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-06T21:10:32.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 bash[98371]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T21:10:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 systemd[1]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-06T21:10:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 systemd[1]: Stopped Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:10:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:32 smithi031 systemd[1]: Starting Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:10:33.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 podman[98498]: 2023-10-06T21:10:33.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 bash[98498]: 41c3e23ba6011a92ac58ac4f04f3472ec693aab337c30023f9476f5522c43f69 2023-10-06T21:10:33.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 8 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: pidfile_write: ignore empty --pid-file 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: load: jerasure load: lrc 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: RocksDB version: 7.9.2 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Git sha 0 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: DB SUMMARY 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: DB Session ID: TFNXEWXTO8500CF41IY2 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: CURRENT file: CURRENT 2023-10-06T21:10:33.643 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99323 ; 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.error_if_exists: 0 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.create_if_missing: 0 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.paranoid_checks: 1 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.env: 0x55c007454da0 2023-10-06T21:10:33.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.info_log: 0x55c0092815c0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.statistics: (nil) 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.use_fsync: 0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_log_file_size: 0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_fallocate: 1 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T21:10:33.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.use_direct_reads: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.db_log_dir: 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.wal_dir: 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T21:10:33.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.write_buffer_manager: 0x55c009239360 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T21:10:33.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.unordered_write: 0 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.row_cache: None 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.wal_filter: None 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.two_write_queues: 0 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T21:10:33.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.wal_compression: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.atomic_flush: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.log_readahead_size: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T21:10:33.649 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T21:10:33.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T21:10:33.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_background_jobs: 2 2023-10-06T21:10:33.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_background_compactions: -1 2023-10-06T21:10:33.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_subcompactions: 1 2023-10-06T21:10:33.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_open_files: -1 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T21:10:33.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_background_flushes: -1 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Compression algorithms supported: 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kZSTD supported: 0 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kXpressCompression supported: 0 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kZlibCompression supported: 1 2023-10-06T21:10:33.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kSnappyCompression supported: 1 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kLZ4Compression supported: 1 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: kBZip2Compression supported: 0 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.merge_operator: 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_filter: None 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T21:10:33.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c0092816c0) 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2023-10-06T21:10:33.654 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x55c0092d71f0 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2023-10-06T21:10:33.655 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2023-10-06T21:10:33.656 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2023-10-06T21:10:33.657 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2023-10-06T21:10:33.658 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T21:10:33.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T21:10:33.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T21:10:33.702 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression: NoCompression 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.num_levels: 7 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T21:10:33.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T21:10:33.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T21:10:33.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T21:10:33.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T21:10:33.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.table_properties_collectors: 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.inplace_update_support: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.bloom_locality: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.max_successive_merges: 0 2023-10-06T21:10:33.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.ttl: 2592000 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enable_blob_files: false 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.min_blob_size: 0 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T21:10:33.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed2e136c-d5ef-4acc-a688-3aa4e5dd361e 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626633634076, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626633636175, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94535, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92540, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13773, "raw_average_key_size": 50, "raw_value_size": 85011, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696626633, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed2e136c-d5ef-4acc-a688-3aa4e5dd361e", "db_session_id": "TFNXEWXTO8500CF41IY2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-06T21:10:33.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626633636313, "job": 1, "event": "recovery_finished"} 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c009374000 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: DB pointer 0x55c009360000 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:10:33.950 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 94.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 94.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.951 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T21:10:33.952 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 13.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 13.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x55c0092d71f0#8 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???) e1 preinit fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).mds e1 new map 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).mds e1 print_map 2023-10-06T21:10:33.953 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expand map: {default=false} 2023-10-06T21:10:33.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta from 'false' to 'false' 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expanded map: {default=false} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expand map: {default=info} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta from 'info' to 'info' 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expanded map: {default=info} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expand map: {default=daemon} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expand map: {default=debug} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: expand_channel_meta expanded map: {default=debug} 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 systemd[1]: Started Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:10:33.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:10:33.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: monmap e1: 1 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} 2023-10-06T21:10:33.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: fsmap 2023-10-06T21:10:33.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T21:10:33.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:33 smithi031 ceph-mon[98539]: mgrmap e1: no daemons active 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.206+0000 7fb689d04700 1 Processor -- start 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.206+0000 7fb689d04700 1 -- start start 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.207+0000 7fb689d04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.207+0000 7fb689d04700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68407a8c0 con 0x7fb68407be90 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.207+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.207+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37326/0 (socket says 172.21.15.31:37326) 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.207+0000 7fb6837fe700 1 -- 172.21.15.31:0/2407040917 learned_addr learned my addr 172.21.15.31:0/2407040917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:34.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.208+0000 7fb6837fe700 1 -- 172.21.15.31:0/2407040917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68407aa00 con 0x7fb68407be90 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.208+0000 7fb6837fe700 1 --2- 172.21.15.31:0/2407040917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fec6e5456daa8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.208+0000 7fb6827fc700 1 -- 172.21.15.31:0/2407040917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb674004160 con 0x7fb68407be90 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.208+0000 7fb6827fc700 1 -- 172.21.15.31:0/2407040917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb6740042c0 con 0x7fb68407be90 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.208+0000 7fb6827fc700 1 -- 172.21.15.31:0/2407040917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb674011940 con 0x7fb68407be90 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 -- 172.21.15.31:0/2407040917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 msgr2=0x7fb68407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 --2- 172.21.15.31:0/2407040917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 -- 172.21.15.31:0/2407040917 shutdown_connections 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 --2- 172.21.15.31:0/2407040917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68407a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 -- 172.21.15.31:0/2407040917 >> 172.21.15.31:0/2407040917 conn(0x7fb684101cd0 msgr2=0x7fb6841040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 -- 172.21.15.31:0/2407040917 shutdown_connections 2023-10-06T21:10:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.209+0000 7fb689d04700 1 -- 172.21.15.31:0/2407040917 wait complete. 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.210+0000 7fb689d04700 1 Processor -- start 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.211+0000 7fb689d04700 1 -- start start 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.211+0000 7fb689d04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.211+0000 7fb689d04700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68419c9a0 con 0x7fb68407be90 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.211+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37328/0 (socket says 172.21.15.31:37328) 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb6837fe700 1 -- 172.21.15.31:0/102947634 learned_addr learned my addr 172.21.15.31:0/102947634 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb6837fe700 1 -- 172.21.15.31:0/102947634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009550 con 0x7fb68407be90 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb6837fe700 1 --2- 172.21.15.31:0/102947634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6740036a0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb674003c70 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68419cba0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.212+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6740193f0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.213+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68419d040 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.213+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb674019970 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.213+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb67401a3d0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.214+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6840624e0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.217+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb674029060 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.290+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fb68410cc40 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb680ff9700 1 -- 172.21.15.31:0/102947634 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb6740330c0 con 0x7fb68407be90 2023-10-06T21:10:34.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 msgr2=0x7fb68419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb689d04700 1 --2- 172.21.15.31:0/102947634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 shutdown_connections 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb689d04700 1 --2- 172.21.15.31:0/102947634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb68407be90 0x7fb68419c460 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.293+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 >> 172.21.15.31:0/102947634 conn(0x7fb684101cd0 msgr2=0x7fb684102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.294+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 shutdown_connections 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:34.294+0000 7fb689d04700 1 -- 172.21.15.31:0/102947634 wait complete. 2023-10-06T21:10:34.692 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-06T21:10:34.703 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:34.703 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2023-10-06T21:10:34.704 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-06T21:10:34.705 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-06T21:10:35.077 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T21:10:35.077 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-06T21:10:35.289 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7.target.wants/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@.service. 2023-10-06T21:10:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:35 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/102947634' entity='client.admin' 2023-10-06T21:10:35.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:35 smithi031 systemd[1]: Starting Ceph mgr.x for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:10:36.048 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:35 smithi031 podman[98770]: 2023-10-06T21:10:36.103 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:10:36.103 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-10-06T21:10:36.120 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:10:36.120 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:10:36.120 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-10-06T21:10:36.120 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T21:10:36.138 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:10:36.139 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-10-06T21:10:36.155 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:10:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:10:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-10-06T21:10:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-06T21:10:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2023-10-06T21:10:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2023-10-06T21:10:36.351 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 bash[98770]: eab7bc355e228fd934baa72752c35cc145d2feccdcdaccf46146c59016356c63 2023-10-06T21:10:36.351 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 systemd[1]: Started Ceph mgr.x for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:10:36.351 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:36.239+0000 7f4377ae9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:10:36.759 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:36.349+0000 7f4377ae9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:36.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:36.470+0000 7f4377ae9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:10:36.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:36.597+0000 7f4377ae9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:10:37.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:36.838+0000 7f4377ae9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:10:37.643 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:37.339+0000 7f4377ae9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:10:37.643 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:37.512+0000 7f4377ae9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:10:38.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:37.641+0000 7f4377ae9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:10:38.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:37.772+0000 7f4377ae9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:10:38.740 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:38.346+0000 7f4377ae9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:10:39.759 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:39.267+0000 7f4377ae9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:10:39.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:39.377+0000 7f4377ae9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:10:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2408292262' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T21:10:40.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:40.378+0000 7f4377ae9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:40.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:40.507+0000 7f4377ae9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:10:41.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:40.740+0000 7f4377ae9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:41.516 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:41.120+0000 7f4377ae9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:10:41.516 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:41.401+0000 7f4377ae9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:10:41.989 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:41.618+0000 7f4377ae9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:10:42.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:41.987+0000 7f4377ae9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:10:43.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:42 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:42.960+0000 7f4377ae9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:10:43.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:43 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:43.144+0000 7f4377ae9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:10:43.683 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:43 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:43.399+0000 7f4377ae9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:10:43.683 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:43 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:43.550+0000 7f4377ae9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:44.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:43 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:43.838+0000 7f4377ae9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:10:44.460 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:44.460 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T21:10:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T21:10:44.462 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T21:10:44.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T21:10:44.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T21:10:44.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T21:10:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T21:10:44.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T21:10:44.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T21:10:28.426229+0000", 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:10:44.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.381+0000 7f67d1849700 1 Processor -- start 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.381+0000 7f67d1849700 1 -- start start 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.382+0000 7f67d1849700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.382+0000 7f67d1849700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1070f0 con 0x7f67cc106700 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.382+0000 7f67caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.382+0000 7f67caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37348/0 (socket says 172.21.15.31:37348) 2023-10-06T21:10:44.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.382+0000 7f67caffd700 1 -- 172.21.15.31:0/3653578911 learned_addr learned my addr 172.21.15.31:0/3653578911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.383+0000 7f67caffd700 1 -- 172.21.15.31:0/3653578911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc107900 con 0x7f67cc106700 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.383+0000 7f67caffd700 1 --2- 172.21.15.31:0/3653578911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0098a0 tx=0x7f67bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3ca0518f9b1f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.383+0000 7f67c9ffb700 1 -- 172.21.15.31:0/3653578911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67bc004160 con 0x7f67cc106700 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.383+0000 7f67c9ffb700 1 -- 172.21.15.31:0/3653578911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f67bc0042c0 con 0x7f67cc106700 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.383+0000 7f67c9ffb700 1 -- 172.21.15.31:0/3653578911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67bc011940 con 0x7f67cc106700 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 -- 172.21.15.31:0/3653578911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 msgr2=0x7f67cc106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 --2- 172.21.15.31:0/3653578911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0098a0 tx=0x7f67bc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 -- 172.21.15.31:0/3653578911 shutdown_connections 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 --2- 172.21.15.31:0/3653578911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc106b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 -- 172.21.15.31:0/3653578911 >> 172.21.15.31:0/3653578911 conn(0x7f67cc101ce0 msgr2=0x7f67cc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:44.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 -- 172.21.15.31:0/3653578911 shutdown_connections 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.384+0000 7f67d1849700 1 -- 172.21.15.31:0/3653578911 wait complete. 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.385+0000 7f67d1849700 1 Processor -- start 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.385+0000 7f67d1849700 1 -- start start 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.385+0000 7f67d1849700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.385+0000 7f67d1849700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc19ca30 con 0x7f67cc106700 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37362/0 (socket says 172.21.15.31:37362) 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67caffd700 1 -- 172.21.15.31:0/2408292262 learned_addr learned my addr 172.21.15.31:0/2408292262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67caffd700 1 -- 172.21.15.31:0/2408292262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc009550 con 0x7f67cc106700 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67caffd700 1 --2- 172.21.15.31:0/2408292262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67bc000c00 tx=0x7f67bc011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67bc0036a0 con 0x7f67cc106700 2023-10-06T21:10:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f67bc003c70 con 0x7f67cc106700 2023-10-06T21:10:44.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.386+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67bc0193f0 con 0x7f67cc106700 2023-10-06T21:10:44.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.387+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc19cc30 con 0x7f67cc106700 2023-10-06T21:10:44.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.387+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc19d050 con 0x7f67cc106700 2023-10-06T21:10:44.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.387+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f67bc019920 con 0x7f67cc106700 2023-10-06T21:10:44.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.387+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f67bc01a3d0 con 0x7f67cc106700 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.388+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc0624e0 con 0x7f67cc106700 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.390+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f67bc029060 con 0x7f67cc106700 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.472+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f67cc10ac20 con 0x7f67cc106700 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.472+0000 7f67d0847700 1 -- 172.21.15.31:0/2408292262 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f67bc033090 con 0x7f67cc106700 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 msgr2=0x7f67cc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:44.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 --2- 172.21.15.31:0/2408292262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67bc000c00 tx=0x7f67bc011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 shutdown_connections 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 --2- 172.21.15.31:0/2408292262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67cc106700 0x7f67cc19c4f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 >> 172.21.15.31:0/2408292262 conn(0x7f67cc101ce0 msgr2=0x7f67cc102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 shutdown_connections 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:40.473+0000 7f67d1849700 1 -- 172.21.15.31:0/2408292262 wait complete. 2023-10-06T21:10:44.480 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2023-10-06T21:10:45.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:44 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:44.962+0000 7f4377ae9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:10:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: Activating manager daemon x 2023-10-06T21:10:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: mgrmap e2: x(active, starting, since 0.00413125s) 2023-10-06T21:10:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:10:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: Manager daemon x is now available 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' 2023-10-06T21:10:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:45 smithi031 ceph-mon[98539]: from='mgr.14100 172.21.15.31:0/3876975678' entity='mgr.x' 2023-10-06T21:10:47.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:46 smithi031 ceph-mon[98539]: mgrmap e3: x(active, since 1.00989s) 2023-10-06T21:10:47.752 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:47.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:10:47.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:10:47.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T21:10:47.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T21:10:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T21:10:47.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T21:10:47.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T21:10:47.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T21:10:47.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T21:10:47.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T21:10:47.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T21:10:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T21:10:28.426229+0000", 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T21:10:47.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.007+0000 7f16c1328700 1 Processor -- start 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.007+0000 7f16c1328700 1 -- start start 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16c1328700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16c1328700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc07a8e0 con 0x7f16bc07beb0 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45962/0 (socket says 172.21.15.31:45962) 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16baffd700 1 -- 172.21.15.31:0/3634653898 learned_addr learned my addr 172.21.15.31:0/3634653898 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.008+0000 7f16baffd700 1 -- 172.21.15.31:0/3634653898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc07aa20 con 0x7f16bc07beb0 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.009+0000 7f16baffd700 1 --2- 172.21.15.31:0/3634653898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0098a0 tx=0x7f16ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3952414da9330c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.009+0000 7f16b9ffb700 1 -- 172.21.15.31:0/3634653898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac004160 con 0x7f16bc07beb0 2023-10-06T21:10:47.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.009+0000 7f16b9ffb700 1 -- 172.21.15.31:0/3634653898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f16ac0042c0 con 0x7f16bc07beb0 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.009+0000 7f16b9ffb700 1 -- 172.21.15.31:0/3634653898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac011940 con 0x7f16bc07beb0 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 -- 172.21.15.31:0/3634653898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 msgr2=0x7f16bc07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 --2- 172.21.15.31:0/3634653898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0098a0 tx=0x7f16ac009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 -- 172.21.15.31:0/3634653898 shutdown_connections 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 --2- 172.21.15.31:0/3634653898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc07a310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 -- 172.21.15.31:0/3634653898 >> 172.21.15.31:0/3634653898 conn(0x7f16bc101ce0 msgr2=0x7f16bc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 -- 172.21.15.31:0/3634653898 shutdown_connections 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.010+0000 7f16c1328700 1 -- 172.21.15.31:0/3634653898 wait complete. 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16c1328700 1 Processor -- start 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16c1328700 1 -- start start 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16c1328700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:47.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16c1328700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc109f90 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45976/0 (socket says 172.21.15.31:45976) 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.011+0000 7f16baffd700 1 -- 172.21.15.31:0/856647277 learned_addr learned my addr 172.21.15.31:0/856647277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16baffd700 1 -- 172.21.15.31:0/856647277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac009550 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16baffd700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac0036a0 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f16ac003c70 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc10a190 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac0193f0 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.012+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc106680 con 0x7f16bc07beb0 2023-10-06T21:10:47.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.013+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f16ac01b440 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.013+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc0624e0 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.014+0000 7f16a3fff700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 0x7f16a403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.014+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16ac04c640 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.014+0000 7f16ba7fc700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 0x7f16a403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.015+0000 7f16ba7fc700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 0x7f16a403ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.020+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f16ac022950 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.309+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f16bc10ca50 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.310+0000 7f16a3fff700 1 -- 172.21.15.31:0/856647277 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f16ac01f070 con 0x7f16bc07beb0 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 msgr2=0x7f16a403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 0x7f16a403ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0006e40 comp rx=0 tx=0).stop 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 msgr2=0x7f16bc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:47.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 shutdown_connections 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f16a40387d0 0x7f16a403ac90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 --2- 172.21.15.31:0/856647277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc07beb0 0x7f16bc109a50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 >> 172.21.15.31:0/856647277 conn(0x7f16bc101ce0 msgr2=0x7f16bc102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 shutdown_connections 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:47.312+0000 7f16c1328700 1 -- 172.21.15.31:0/856647277 wait complete. 2023-10-06T21:10:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2023-10-06T21:10:48.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:47 smithi031 ceph-mon[98539]: mgrmap e4: x(active, since 2s) 2023-10-06T21:10:48.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:47 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/856647277' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T21:10:48.937 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:48.943 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T21:10:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.288+0000 7fa1e38eb700 1 Processor -- start 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.288+0000 7fa1e38eb700 1 -- start start 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.289+0000 7fa1e38eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.289+0000 7fa1e38eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc07a8e0 con 0x7fa1dc07beb0 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.289+0000 7fa1e1687700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.289+0000 7fa1e1687700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45986/0 (socket says 172.21.15.31:45986) 2023-10-06T21:10:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.289+0000 7fa1e1687700 1 -- 172.21.15.31:0/1384482958 learned_addr learned my addr 172.21.15.31:0/1384482958 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.290+0000 7fa1e1687700 1 -- 172.21.15.31:0/1384482958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc07aa20 con 0x7fa1dc07beb0 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.290+0000 7fa1e1687700 1 --2- 172.21.15.31:0/1384482958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098a0 tx=0x7fa1d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e96200f29ce9a9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.290+0000 7fa1d3fff700 1 -- 172.21.15.31:0/1384482958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d8004160 con 0x7fa1dc07beb0 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.290+0000 7fa1d3fff700 1 -- 172.21.15.31:0/1384482958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa1d80042c0 con 0x7fa1dc07beb0 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.290+0000 7fa1d3fff700 1 -- 172.21.15.31:0/1384482958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d8011940 con 0x7fa1dc07beb0 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1384482958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 msgr2=0x7fa1dc07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1384482958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098a0 tx=0x7fa1d8009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1384482958 shutdown_connections 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1384482958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc07a310 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1384482958 >> 172.21.15.31:0/1384482958 conn(0x7fa1dc101ce0 msgr2=0x7fa1dc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.946 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1384482958 shutdown_connections 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.291+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1384482958 wait complete. 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.292+0000 7fa1e38eb700 1 Processor -- start 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.292+0000 7fa1e38eb700 1 -- start start 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.292+0000 7fa1e38eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.292+0000 7fa1e38eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc109f90 con 0x7fa1dc07beb0 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1e1687700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1e1687700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46002/0 (socket says 172.21.15.31:46002) 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1e1687700 1 -- 172.21.15.31:0/1568207385 learned_addr learned my addr 172.21.15.31:0/1568207385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1e1687700 1 -- 172.21.15.31:0/1568207385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009550 con 0x7fa1dc07beb0 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1e1687700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8000c00 tx=0x7fa1d8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.293+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d80036a0 con 0x7fa1dc07beb0 2023-10-06T21:10:48.947 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.294+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa1d8003c70 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.294+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc10a190 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.294+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d80193f0 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.294+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc106680 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.295+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7fa1d801b440 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.295+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0624e0 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.295+0000 7fa1d27fc700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 0x7fa1c803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.295+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa1d804c640 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.295+0000 7fa1e0e86700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 0x7fa1c803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.296+0000 7fa1e0e86700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 0x7fa1c803ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.300+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa1d8022950 con 0x7fa1dc07beb0 2023-10-06T21:10:48.948 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.508+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa1dc10a320 con 0x7fa1dc07beb0 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.509+0000 7fa1d27fc700 1 -- 172.21.15.31:0/1568207385 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fa1d801f070 con 0x7fa1dc07beb0 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 msgr2=0x7fa1c803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 0x7fa1c803ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc006e40 comp rx=0 tx=0).stop 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 msgr2=0x7fa1dc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8000c00 tx=0x7fa1d8011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 shutdown_connections 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fa1c80387d0 0x7fa1c803ac90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 --2- 172.21.15.31:0/1568207385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1dc07beb0 0x7fa1dc109a50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 >> 172.21.15.31:0/1568207385 conn(0x7fa1dc101ce0 msgr2=0x7fa1dc102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 shutdown_connections 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:48.511+0000 7fa1e38eb700 1 -- 172.21.15.31:0/1568207385 wait complete. 2023-10-06T21:10:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2023-10-06T21:10:49.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:48 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1568207385' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T21:10:50.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:49 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2965565957' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-06T21:10:50.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setuser ceph since I am not root 2023-10-06T21:10:50.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setgroup ceph since I am not root 2023-10-06T21:10:50.495 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:50.267+0000 7f41c49ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:10:50.495 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:50.374+0000 7f41c49ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.498+0000 7fb86d8ab700 1 Processor -- start 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.498+0000 7fb86d8ab700 1 -- start start 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb86d8ab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb86d8ab700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86807a8c0 con 0x7fb86807be90 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46012/0 (socket says 172.21.15.31:46012) 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb866ffd700 1 -- 172.21.15.31:0/2996542016 learned_addr learned my addr 172.21.15.31:0/2996542016 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.499+0000 7fb866ffd700 1 -- 172.21.15.31:0/2996542016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86807aa00 con 0x7fb86807be90 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb866ffd700 1 --2- 172.21.15.31:0/2996542016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687362ae3cdb43ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb865ffb700 1 -- 172.21.15.31:0/2996542016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb858004160 con 0x7fb86807be90 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb865ffb700 1 -- 172.21.15.31:0/2996542016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb8580042c0 con 0x7fb86807be90 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb865ffb700 1 -- 172.21.15.31:0/2996542016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb858011940 con 0x7fb86807be90 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2996542016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 msgr2=0x7fb86807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2996542016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2996542016 shutdown_connections 2023-10-06T21:10:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2996542016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86807a2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.500+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2996542016 >> 172.21.15.31:0/2996542016 conn(0x7fb868101cd0 msgr2=0x7fb8681040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.501+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2996542016 shutdown_connections 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.501+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2996542016 wait complete. 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb86d8ab700 1 Processor -- start 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb86d8ab700 1 -- start start 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb86d8ab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb86d8ab700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86819c9a0 con 0x7fb86807be90 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46028/0 (socket says 172.21.15.31:46028) 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.502+0000 7fb866ffd700 1 -- 172.21.15.31:0/2965565957 learned_addr learned my addr 172.21.15.31:0/2965565957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:50.530 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb866ffd700 1 -- 172.21.15.31:0/2965565957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009550 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb866ffd700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb858000c00 tx=0x7fb858011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8580036a0 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb858003c70 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86819cba0 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8580193f0 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.503+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86819d040 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.504+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7fb85801b440 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.504+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86810ca40 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.505+0000 7fb86c8a9700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 0x7fb85003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.505+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb85804c5f0 con 0x7fb86807be90 2023-10-06T21:10:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.505+0000 7fb8667fc700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 0x7fb85003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.506+0000 7fb8667fc700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 0x7fb85003ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.510+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb858022930 con 0x7fb86807be90 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.779+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb86819fa00 con 0x7fb86807be90 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.982+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb85801f070 con 0x7fb86807be90 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.984+0000 7fb86c8a9700 1 -- 172.21.15.31:0/2965565957 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fb858003990 con 0x7fb86807be90 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.985+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 msgr2=0x7fb85003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:50.532 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.985+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 0x7fb85003ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c006e40 comp rx=0 tx=0).stop 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.985+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 msgr2=0x7fb86819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.985+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb858000c00 tx=0x7fb858011e70 comp rx=0 tx=0).stop 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 shutdown_connections 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7fb850038870 0x7fb85003ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 --2- 172.21.15.31:0/2965565957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb86807be90 0x7fb86819c460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 >> 172.21.15.31:0/2965565957 conn(0x7fb868101cd0 msgr2=0x7fb868102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 shutdown_connections 2023-10-06T21:10:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:49.986+0000 7fb86d8ab700 1 -- 172.21.15.31:0/2965565957 wait complete. 2023-10-06T21:10:50.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:50.493+0000 7f41c49ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:10:50.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:50.628+0000 7f41c49ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:10:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2965565957' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-06T21:10:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-mon[98539]: mgrmap e5: x(active, since 5s) 2023-10-06T21:10:51.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:50.864+0000 7f41c49ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:10:51.643 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:51.344+0000 7f41c49ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:10:51.644 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:51.519+0000 7f41c49ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:10:51.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:10:51.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-06T21:10:51.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.085+0000 7f8a5bebe700 1 Processor -- start 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a5bebe700 1 -- start start 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a5bebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a5bebe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a541070f0 con 0x7f8a54106700 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a59c5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a59c5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46068/0 (socket says 172.21.15.31:46068) 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.086+0000 7f8a59c5a700 1 -- 172.21.15.31:0/1171248974 learned_addr learned my addr 172.21.15.31:0/1171248974 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.087+0000 7f8a59c5a700 1 -- 172.21.15.31:0/1171248974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54107900 con 0x7f8a54106700 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.087+0000 7f8a59c5a700 1 --2- 172.21.15.31:0/1171248974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a500098a0 tx=0x7f8a50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcdd2715c304e13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.087+0000 7f8a58c58700 1 -- 172.21.15.31:0/1171248974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a50004160 con 0x7f8a54106700 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.087+0000 7f8a58c58700 1 -- 172.21.15.31:0/1171248974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8a500042c0 con 0x7f8a54106700 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.087+0000 7f8a58c58700 1 -- 172.21.15.31:0/1171248974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a50011940 con 0x7f8a54106700 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 -- 172.21.15.31:0/1171248974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 msgr2=0x7f8a54106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/1171248974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a500098a0 tx=0x7f8a50009bb0 comp rx=0 tx=0).stop 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 -- 172.21.15.31:0/1171248974 shutdown_connections 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/1171248974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a54106b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:51.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 -- 172.21.15.31:0/1171248974 >> 172.21.15.31:0/1171248974 conn(0x7f8a54101ce0 msgr2=0x7f8a54104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:51.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 -- 172.21.15.31:0/1171248974 shutdown_connections 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.088+0000 7f8a5bebe700 1 -- 172.21.15.31:0/1171248974 wait complete. 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.090+0000 7f8a5bebe700 1 Processor -- start 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.090+0000 7f8a5bebe700 1 -- start start 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.090+0000 7f8a5bebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.090+0000 7f8a5bebe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5407c180 con 0x7f8a54106700 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.090+0000 7f8a59c5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a59c5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46076/0 (socket says 172.21.15.31:46076) 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a59c5a700 1 -- 172.21.15.31:0/3370312811 learned_addr learned my addr 172.21.15.31:0/3370312811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:10:51.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a59c5a700 1 -- 172.21.15.31:0/3370312811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009550 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a59c5a700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006af0 tx=0x7f8a50011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a50003bf0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5407a290 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8a5001b8b0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.091+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a500197e0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.092+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5407a6b0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f8a5001ba20 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a540624e0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a4affd700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 0x7f8a4003ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a5004cac0 con 0x7f8a54106700 2023-10-06T21:10:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a59459700 1 -- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 msgr2=0x7f8a4003ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.093+0000 7f8a59459700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 0x7f8a4003ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.099+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8a50018920 con 0x7f8a54106700 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.294+0000 7f8a59459700 1 -- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 msgr2=0x7f8a4003ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.294+0000 7f8a59459700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 0x7f8a4003ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.365+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8a5410ab80 con 0x7f8a54106700 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.366+0000 7f8a4affd700 1 -- 172.21.15.31:0/3370312811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8a5001f250 con 0x7f8a54106700 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.368+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 msgr2=0x7f8a4003ac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.368+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 0x7f8a4003ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.368+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 msgr2=0x7f8a5407bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.368+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006af0 tx=0x7f8a50011a30 comp rx=0 tx=0).stop 2023-10-06T21:10:51.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 shutdown_connections 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f8a400387d0 0x7f8a4003ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 --2- 172.21.15.31:0/3370312811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a54106700 0x7f8a5407bc40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 >> 172.21.15.31:0/3370312811 conn(0x7f8a54101ce0 msgr2=0x7f8a54102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 shutdown_connections 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:51.369+0000 7f8a5bebe700 1 -- 172.21.15.31:0/3370312811 wait complete. 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2023-10-06T21:10:51.818 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2023-10-06T21:10:51.985 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:51.641+0000 7f41c49ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:10:51.986 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:51.771+0000 7f41c49ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:10:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:51 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3370312811' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T21:10:52.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:52.345+0000 7f41c49ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:10:53.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:53.217+0000 7f41c49ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:10:53.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:53.326+0000 7f41c49ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:10:54.693 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:54.333+0000 7f41c49ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:54.693 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:54.457+0000 7f41c49ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:10:55.375 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:54.691+0000 7f41c49ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:55.375 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:55.072+0000 7f41c49ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:10:55.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:55.372+0000 7f41c49ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:10:55.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:55.571+0000 7f41c49ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:10:56.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:55.929+0000 7f41c49ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:10:57.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:56 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:56.888+0000 7f41c49ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:10:57.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:57.067+0000 7f41c49ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:10:57.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:57.324+0000 7f41c49ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:10:57.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:57.474+0000 7f41c49ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:58.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:57.763+0000 7f41c49ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:10:59.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:10:58 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:10:58.879+0000 7f41c49ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:10:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: Active manager daemon x restarted 2023-10-06T21:10:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: Activating manager daemon x 2023-10-06T21:10:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: osdmap e2: 0 total, 0 up, 0 in 2023-10-06T21:10:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: mgrmap e6: x(active, starting, since 0.00393843s) 2023-10-06T21:10:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: Manager daemon x is now available 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: Found migration_current of "None". Setting to last migration. 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:59.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:10:59.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:10:59 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:11:00.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:11:00.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.349+0000 7f60b7674700 1 Processor -- start 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.349+0000 7f60b7674700 1 -- start start 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b7674700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b7674700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b007a8c0 con 0x7f60b007be90 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b5410700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b5410700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46080/0 (socket says 172.21.15.31:46080) 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b5410700 1 -- 172.21.15.31:0/2214735167 learned_addr learned my addr 172.21.15.31:0/2214735167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.350+0000 7f60b5410700 1 -- 172.21.15.31:0/2214735167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b007aa00 con 0x7f60b007be90 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.351+0000 7f60b5410700 1 --2- 172.21.15.31:0/2214735167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098a0 tx=0x7f60ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9108eaa9fcab3374 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.351+0000 7f60a7fff700 1 -- 172.21.15.31:0/2214735167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60ac004160 con 0x7f60b007be90 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.351+0000 7f60a7fff700 1 -- 172.21.15.31:0/2214735167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f60ac0042c0 con 0x7f60b007be90 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.351+0000 7f60a7fff700 1 -- 172.21.15.31:0/2214735167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60ac011940 con 0x7f60b007be90 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 -- 172.21.15.31:0/2214735167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 msgr2=0x7f60b007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 --2- 172.21.15.31:0/2214735167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098a0 tx=0x7f60ac009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 -- 172.21.15.31:0/2214735167 shutdown_connections 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 --2- 172.21.15.31:0/2214735167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b007a2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 -- 172.21.15.31:0/2214735167 >> 172.21.15.31:0/2214735167 conn(0x7f60b0101ce0 msgr2=0x7f60b0104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 -- 172.21.15.31:0/2214735167 shutdown_connections 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.352+0000 7f60b7674700 1 -- 172.21.15.31:0/2214735167 wait complete. 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.353+0000 7f60b7674700 1 Processor -- start 2023-10-06T21:11:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.353+0000 7f60b7674700 1 -- start start 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.353+0000 7f60b7674700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.353+0000 7f60b7674700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b019ca30 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.354+0000 7f60b5410700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.354+0000 7f60b5410700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46082/0 (socket says 172.21.15.31:46082) 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.354+0000 7f60b5410700 1 -- 172.21.15.31:0/1989633032 learned_addr learned my addr 172.21.15.31:0/1989633032 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.354+0000 7f60b5410700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009550 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.354+0000 7f60b5410700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.355+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60ac0036a0 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.355+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f60ac003c70 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.355+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60ac0193f0 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.355+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b019cc30 con 0x7f60b007be90 2023-10-06T21:11:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.355+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b019d050 con 0x7f60b007be90 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.356+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f60ac01b440 con 0x7f60b007be90 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.357+0000 7f60a67fc700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.357+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f609c03b440 con 0x7f609c038870 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.357+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f60ac04c650 con 0x7f60b007be90 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.357+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.357+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.558+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.558+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.959+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:52.959+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:11:00.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:53.760+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:53.760+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:55.360+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:55.360+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:58.563+0000 7f60b4c0f700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/461858081 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:58.563+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:58.884+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) v1 ==== 45346+0+0 (secure 0 0 0) 0x7f60ac01b790 con 0x7f60b007be90 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:58.884+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 msgr2=0x7f609c03ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:58.884+0000 7f60a67fc700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.885+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7f60ac04e170 con 0x7f60b007be90 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.885+0000 7f60a67fc700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 0x7f609c03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.886+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f609c03b440 con 0x7f609c03bfd0 2023-10-06T21:11:00.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.886+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 0x7f609c03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.887+0000 7f60b4c0f700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 0x7f609c03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60a0003a10 tx=0x7f60a00092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.888+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f609c03b440 con 0x7f609c03bfd0 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.899+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f60b019d830 con 0x7f609c03bfd0 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.899+0000 7f60a67fc700 1 -- 172.21.15.31:0/1989633032 <== mgr.14116 v2:172.21.15.31:6800/484008040 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f60b019d830 con 0x7f609c03bfd0 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 msgr2=0x7f609c03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 0x7f609c03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60a0003a10 tx=0x7f60a00092b0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 msgr2=0x7f60b019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac011e70 comp rx=0 tx=0).stop 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 shutdown_connections 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f609c03bfd0 0x7f609c03e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b007be90 0x7f60b019c4f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 --2- 172.21.15.31:0/1989633032 >> [v2:172.21.15.31:6800/461858081,v1:172.21.15.31:6801/461858081] conn(0x7f609c038870 0x7f609c03ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:00.417 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 >> 172.21.15.31:0/1989633032 conn(0x7f60b0101ce0 msgr2=0x7f60b0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:00.417 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 shutdown_connections 2023-10-06T21:11:00.417 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:10:59.900+0000 7f60b7674700 1 -- 172.21.15.31:0/1989633032 wait complete. 2023-10-06T21:11:00.417 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2023-10-06T21:11:00.417 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2023-10-06T21:11:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:00 smithi031 ceph-mon[98539]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T21:11:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:00 smithi031 ceph-mon[98539]: mgrmap e7: x(active, since 1.00655s) 2023-10-06T21:11:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:00 smithi031 ceph-mon[98539]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T21:11:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:00 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:00 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.984+0000 7fbf16348700 1 Processor -- start 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf16348700 1 -- start start 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf16348700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf16348700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf101092c0 con 0x7fbf101088d0 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:01.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47156/0 (socket says 172.21.15.31:47156) 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.985+0000 7fbf0ffff700 1 -- 172.21.15.31:0/1799348168 learned_addr learned my addr 172.21.15.31:0/1799348168 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.986+0000 7fbf0ffff700 1 -- 172.21.15.31:0/1799348168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10109ad0 con 0x7fbf101088d0 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.986+0000 7fbf0ffff700 1 --2- 172.21.15.31:0/1799348168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf000098a0 tx=0x7fbf00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fc9f8220a90d90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.986+0000 7fbf0effd700 1 -- 172.21.15.31:0/1799348168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf00004160 con 0x7fbf101088d0 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.986+0000 7fbf0effd700 1 -- 172.21.15.31:0/1799348168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbf000042c0 con 0x7fbf101088d0 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.986+0000 7fbf0effd700 1 -- 172.21.15.31:0/1799348168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf00011940 con 0x7fbf101088d0 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 -- 172.21.15.31:0/1799348168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 msgr2=0x7fbf10108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 --2- 172.21.15.31:0/1799348168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf000098a0 tx=0x7fbf00009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 -- 172.21.15.31:0/1799348168 shutdown_connections 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 --2- 172.21.15.31:0/1799348168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf10108cf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 -- 172.21.15.31:0/1799348168 >> 172.21.15.31:0/1799348168 conn(0x7fbf1007be10 msgr2=0x7fbf101064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 -- 172.21.15.31:0/1799348168 shutdown_connections 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.987+0000 7fbf16348700 1 -- 172.21.15.31:0/1799348168 wait complete. 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.988+0000 7fbf16348700 1 Processor -- start 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.988+0000 7fbf16348700 1 -- start start 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.988+0000 7fbf16348700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.988+0000 7fbf16348700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1019c970 con 0x7fbf101088d0 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47170/0 (socket says 172.21.15.31:47170) 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0ffff700 1 -- 172.21.15.31:0/1003041195 learned_addr learned my addr 172.21.15.31:0/1003041195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0ffff700 1 -- 172.21.15.31:0/1003041195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00009550 con 0x7fbf101088d0 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0ffff700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbf00000c00 tx=0x7fbf00011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf000036a0 con 0x7fbf101088d0 2023-10-06T21:11:01.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.989+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbf00003c70 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.990+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1019cb70 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.990+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf000193f0 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.990+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1019d010 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.991+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7fbf0001b440 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.991+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf100624e0 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.991+0000 7fbf0d7fa700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 0x7fbef803ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.991+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbf0004c520 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.991+0000 7fbf0f7fe700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 0x7fbef803ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.992+0000 7fbf0f7fe700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 0x7fbef803ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf04006fd0 tx=0x7fbf04006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:00.997+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf00022950 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.217+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fbf0001b790 con 0x7fbf101088d0 2023-10-06T21:11:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.225+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf10106500 con 0x7fbef8038740 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.232+0000 7fbf0d7fa700 1 -- 172.21.15.31:0/1003041195 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf10106500 con 0x7fbef8038740 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 msgr2=0x7fbef803ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 0x7fbef803ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf04006fd0 tx=0x7fbf04006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 msgr2=0x7fbf1019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbf00000c00 tx=0x7fbf00011e70 comp rx=0 tx=0).stop 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 shutdown_connections 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fbef8038740 0x7fbef803ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 --2- 172.21.15.31:0/1003041195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf101088d0 0x7fbf1019c430 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 >> 172.21.15.31:0/1003041195 conn(0x7fbf1007be10 msgr2=0x7fbf10105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 shutdown_connections 2023-10-06T21:11:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:01.235+0000 7fbf16348700 1 -- 172.21.15.31:0/1003041195 wait complete. 2023-10-06T21:11:01.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:00] ENGINE Bus STARTING 2023-10-06T21:11:01.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:01] ENGINE Serving on https://172.21.15.31:7150 2023-10-06T21:11:01.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:01] ENGINE Serving on http://172.21.15.31:8765 2023-10-06T21:11:01.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:01] ENGINE Bus STARTED 2023-10-06T21:11:01.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:01.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: mgrmap e8: x(active, since 2s) 2023-10-06T21:11:01.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:01.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:01 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:02.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.229+0000 7fe4bf122700 1 Processor -- start 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.229+0000 7fe4bf122700 1 -- start start 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bf122700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bf122700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b81070f0 con 0x7fe4b8106700 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bcebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bcebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47176/0 (socket says 172.21.15.31:47176) 2023-10-06T21:11:02.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bcebe700 1 -- 172.21.15.31:0/14240671 learned_addr learned my addr 172.21.15.31:0/14240671 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.230+0000 7fe4bcebe700 1 -- 172.21.15.31:0/14240671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8107900 con 0x7fe4b8106700 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4bcebe700 1 --2- 172.21.15.31:0/14240671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1d83444eb45fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4b77fe700 1 -- 172.21.15.31:0/14240671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac004160 con 0x7fe4b8106700 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4b77fe700 1 -- 172.21.15.31:0/14240671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe4ac0042c0 con 0x7fe4b8106700 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4b77fe700 1 -- 172.21.15.31:0/14240671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac011940 con 0x7fe4b8106700 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4bf122700 1 -- 172.21.15.31:0/14240671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 msgr2=0x7fe4b8106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.231+0000 7fe4bf122700 1 --2- 172.21.15.31:0/14240671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.232+0000 7fe4bf122700 1 -- 172.21.15.31:0/14240671 shutdown_connections 2023-10-06T21:11:02.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.232+0000 7fe4bf122700 1 --2- 172.21.15.31:0/14240671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b8106b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.232+0000 7fe4bf122700 1 -- 172.21.15.31:0/14240671 >> 172.21.15.31:0/14240671 conn(0x7fe4b8101ce0 msgr2=0x7fe4b8104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.232+0000 7fe4bf122700 1 -- 172.21.15.31:0/14240671 shutdown_connections 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.232+0000 7fe4bf122700 1 -- 172.21.15.31:0/14240671 wait complete. 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bf122700 1 Processor -- start 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bf122700 1 -- start start 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bf122700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bf122700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b819ca30 con 0x7fe4b8106700 2023-10-06T21:11:02.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bcebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bcebe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47180/0 (socket says 172.21.15.31:47180) 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.233+0000 7fe4bcebe700 1 -- 172.21.15.31:0/4041631061 learned_addr learned my addr 172.21.15.31:0/4041631061 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4bcebe700 1 -- 172.21.15.31:0/4041631061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac009550 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4bcebe700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac006af0 tx=0x7fe4ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac003bf0 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe4ac01b8b0 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b819cc30 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.234+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac0197e0 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.235+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b819d050 con 0x7fe4b8106700 2023-10-06T21:11:02.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.235+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b80624e0 con 0x7fe4b8106700 2023-10-06T21:11:02.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.236+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fe4ac01b440 con 0x7fe4b8106700 2023-10-06T21:11:02.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.236+0000 7fe4b5ffb700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 0x7fe4a803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.236+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe4ac04cb50 con 0x7fe4b8106700 2023-10-06T21:11:02.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.236+0000 7fe4b7fff700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 0x7fe4a803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.237+0000 7fe4b7fff700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 0x7fe4a803ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.241+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4ac018530 con 0x7fe4b8106700 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.454+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4b8103130 con 0x7fe4a8038820 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.455+0000 7fe4b5ffb700 1 -- 172.21.15.31:0/4041631061 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe4b8103130 con 0x7fe4a8038820 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 msgr2=0x7fe4a803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 0x7fe4a803ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 msgr2=0x7fe4b819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac006af0 tx=0x7fe4ac011a30 comp rx=0 tx=0).stop 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 shutdown_connections 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe4a8038820 0x7fe4a803ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 --2- 172.21.15.31:0/4041631061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8106700 0x7fe4b819c4f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 >> 172.21.15.31:0/4041631061 conn(0x7fe4b8101ce0 msgr2=0x7fe4b8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 shutdown_connections 2023-10-06T21:11:02.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:02.457+0000 7fe4bf122700 1 -- 172.21.15.31:0/4041631061 wait complete. 2023-10-06T21:11:02.874 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2023-10-06T21:11:03.153 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:02 smithi031 ceph-mon[98539]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:04.247 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.386+0000 7f96518f9700 1 Processor -- start 2023-10-06T21:11:04.248 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f96518f9700 1 -- start start 2023-10-06T21:11:04.248 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f96518f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:04.248 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f96518f9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c079ea0 con 0x7f964c104f50 2023-10-06T21:11:04.248 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f964affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:04.248 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f964affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59700/0 (socket says 172.21.15.31:59700) 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.387+0000 7f964affd700 1 -- 172.21.15.31:0/1860866192 learned_addr learned my addr 172.21.15.31:0/1860866192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.388+0000 7f964affd700 1 -- 172.21.15.31:0/1860866192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c1058b0 con 0x7f964c104f50 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.388+0000 7f964affd700 1 --2- 172.21.15.31:0/1860866192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f963c0098a0 tx=0x7f963c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7957d9db07e3175 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.388+0000 7f9649ffb700 1 -- 172.21.15.31:0/1860866192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f963c004160 con 0x7f964c104f50 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.388+0000 7f9649ffb700 1 -- 172.21.15.31:0/1860866192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f963c0042c0 con 0x7f964c104f50 2023-10-06T21:11:04.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.388+0000 7f9649ffb700 1 -- 172.21.15.31:0/1860866192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f963c011940 con 0x7f964c104f50 2023-10-06T21:11:04.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 -- 172.21.15.31:0/1860866192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 msgr2=0x7f964c105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:04.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 --2- 172.21.15.31:0/1860866192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f963c0098a0 tx=0x7f963c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:04.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 -- 172.21.15.31:0/1860866192 shutdown_connections 2023-10-06T21:11:04.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 --2- 172.21.15.31:0/1860866192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c105370 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:04.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 -- 172.21.15.31:0/1860866192 >> 172.21.15.31:0/1860866192 conn(0x7f964c100bb0 msgr2=0x7f964c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 -- 172.21.15.31:0/1860866192 shutdown_connections 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.389+0000 7f96518f9700 1 -- 172.21.15.31:0/1860866192 wait complete. 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.390+0000 7f96518f9700 1 Processor -- start 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.390+0000 7f96518f9700 1 -- start start 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f96518f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:04.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f96518f9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c19c9e0 con 0x7f964c104f50 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f964affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f964affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59704/0 (socket says 172.21.15.31:59704) 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f964affd700 1 -- 172.21.15.31:0/3747649693 learned_addr learned my addr 172.21.15.31:0/3747649693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f964affd700 1 -- 172.21.15.31:0/3747649693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c009550 con 0x7f964c104f50 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.391+0000 7f964affd700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f963c011dc0 tx=0x7f963c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.392+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f963c0037d0 con 0x7f964c104f50 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.392+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f963c01b440 con 0x7f964c104f50 2023-10-06T21:11:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.392+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c19cbe0 con 0x7f964c104f50 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.392+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f963c01a560 con 0x7f964c104f50 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.392+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c19d000 con 0x7f964c104f50 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.393+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c195d80 con 0x7f964c104f50 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.393+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f963c01b5b0 con 0x7f964c104f50 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.393+0000 7f96508f7700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 0x7f963403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:04.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.393+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f963c04c740 con 0x7f964c104f50 2023-10-06T21:11:04.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.394+0000 7f964a7fc700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 0x7f963403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:04.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.394+0000 7f964a7fc700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 0x7f963403ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:04.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.398+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f963c023590 con 0x7f964c104f50 2023-10-06T21:11:04.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.609+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f964c02d170 con 0x7f9634038870 2023-10-06T21:11:04.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.817+0000 7f96508f7700 1 -- 172.21.15.31:0/3747649693 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f964c02d170 con 0x7f9634038870 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 msgr2=0x7f963403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 0x7f963403ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 msgr2=0x7f964c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f963c011dc0 tx=0x7f963c011ea0 comp rx=0 tx=0).stop 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 shutdown_connections 2023-10-06T21:11:04.255 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f9634038870 0x7f963403ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 --2- 172.21.15.31:0/3747649693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c104f50 0x7f964c19c4a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 >> 172.21.15.31:0/3747649693 conn(0x7f964c100bb0 msgr2=0x7f964c190b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 shutdown_connections 2023-10-06T21:11:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:03.820+0000 7f96518f9700 1 -- 172.21.15.31:0/3747649693 wait complete. 2023-10-06T21:11:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-mon[98539]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:04.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: Generating public/private rsa key pair. 2023-10-06T21:11:04.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: Your identification has been saved in /tmp/tmp0s0zv_dl/key. 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: Your public key has been saved in /tmp/tmp0s0zv_dl/key.pub. 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: The key fingerprint is: 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: SHA256:QPztwCWWmqw280EyqP7wuo0e9gt5SugNM+vpRHNn5/U ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: The key's randomart image is: 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: +---[RSA 3072]----+ 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: | .. . | 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: | .. + . | 2023-10-06T21:11:04.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: | . ..* + | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: | . o =.+ . | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: | + . B .S+ | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: |+ + B + . o | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: |o% o + o E | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: |=.^ . | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: |+%+B. | 2023-10-06T21:11:04.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: +----[SHA256]-----+ 2023-10-06T21:11:05.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:04 smithi031 ceph-mon[98539]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:05.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:04 smithi031 ceph-mon[98539]: Generating ssh key... 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.785+0000 7f845eb17700 1 Processor -- start 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.785+0000 7f845eb17700 1 -- start start 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.786+0000 7f845eb17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.786+0000 7f845eb17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845807a8e0 con 0x7f845807beb0 2023-10-06T21:11:05.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.786+0000 7f845c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.786+0000 7f845c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59714/0 (socket says 172.21.15.31:59714) 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.786+0000 7f845c8b3700 1 -- 172.21.15.31:0/3406109180 learned_addr learned my addr 172.21.15.31:0/3406109180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f845c8b3700 1 -- 172.21.15.31:0/3406109180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845807aa20 con 0x7f845807beb0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f845c8b3700 1 --2- 172.21.15.31:0/3406109180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f844c0098a0 tx=0x7f844c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b535af88da6d99b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f84577fe700 1 -- 172.21.15.31:0/3406109180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844c004160 con 0x7f845807beb0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f84577fe700 1 -- 172.21.15.31:0/3406109180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f844c0042c0 con 0x7f845807beb0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f84577fe700 1 -- 172.21.15.31:0/3406109180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844c011940 con 0x7f845807beb0 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f845eb17700 1 -- 172.21.15.31:0/3406109180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 msgr2=0x7f845807a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.787+0000 7f845eb17700 1 --2- 172.21.15.31:0/3406109180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f844c0098a0 tx=0x7f844c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.788+0000 7f845eb17700 1 -- 172.21.15.31:0/3406109180 shutdown_connections 2023-10-06T21:11:05.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.788+0000 7f845eb17700 1 --2- 172.21.15.31:0/3406109180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845807a310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.788+0000 7f845eb17700 1 -- 172.21.15.31:0/3406109180 >> 172.21.15.31:0/3406109180 conn(0x7f8458101ce0 msgr2=0x7f8458104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.788+0000 7f845eb17700 1 -- 172.21.15.31:0/3406109180 shutdown_connections 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.788+0000 7f845eb17700 1 -- 172.21.15.31:0/3406109180 wait complete. 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.789+0000 7f845eb17700 1 Processor -- start 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.789+0000 7f845eb17700 1 -- start start 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.789+0000 7f845eb17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.789+0000 7f845eb17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845819c9e0 con 0x7f845807beb0 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.789+0000 7f845c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f845c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59730/0 (socket says 172.21.15.31:59730) 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f845c8b3700 1 -- 172.21.15.31:0/1732774313 learned_addr learned my addr 172.21.15.31:0/1732774313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:05.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f845c8b3700 1 -- 172.21.15.31:0/1732774313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c009550 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f845c8b3700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f844c000c00 tx=0x7f844c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844c0036a0 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f844c003c70 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845819cbe0 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.790+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844c0193f0 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.791+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845819d000 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.791+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845810ca50 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.792+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f844c01b440 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.792+0000 7f8455ffb700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 0x7f844803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.792+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f844c04c720 con 0x7f845807beb0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.792+0000 7f8457fff700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 0x7f844803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.793+0000 7f8457fff700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 0x7f844803ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8444006fd0 tx=0x7f8444006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:04.798+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f844c022950 con 0x7f845807beb0 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.010+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f845819fa40 con 0x7f8448038870 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.011+0000 7f8455ffb700 1 -- 172.21.15.31:0/1732774313 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f845819fa40 con 0x7f8448038870 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 msgr2=0x7f844803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 0x7f844803ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8444006fd0 tx=0x7f8444006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 msgr2=0x7f845819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f844c000c00 tx=0x7f844c011e70 comp rx=0 tx=0).stop 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 shutdown_connections 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f8448038870 0x7f844803ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 --2- 172.21.15.31:0/1732774313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845807beb0 0x7f845819c4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 >> 172.21.15.31:0/1732774313 conn(0x7f8458101ce0 msgr2=0x7f8458102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:05.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 shutdown_connections 2023-10-06T21:11:05.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.013+0000 7f845eb17700 1 -- 172.21.15.31:0/1732774313 wait complete. 2023-10-06T21:11:05.434 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:11:05.434 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2023-10-06T21:11:05.434 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2023-10-06T21:11:06.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:05 smithi031 ceph-mon[98539]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:07.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:07 smithi031 ceph-mon[98539]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:07.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:07 smithi031 ceph-mon[98539]: Deploying cephadm binary to smithi031 2023-10-06T21:11:10.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2023-10-06T21:11:10.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.955+0000 7fb4f284f700 1 Processor -- start 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.955+0000 7fb4f284f700 1 -- start start 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4f284f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4f284f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec079e00 con 0x7fb4ec1071f0 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59740/0 (socket says 172.21.15.31:59740) 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4ebfff700 1 -- 172.21.15.31:0/116261893 learned_addr learned my addr 172.21.15.31:0/116261893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.956+0000 7fb4ebfff700 1 -- 172.21.15.31:0/116261893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec079f40 con 0x7fb4ec1071f0 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4ebfff700 1 --2- 172.21.15.31:0/116261893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0098a0 tx=0x7fb4dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2216301138c14d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4eaffd700 1 -- 172.21.15.31:0/116261893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc004160 con 0x7fb4ec1071f0 2023-10-06T21:11:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4eaffd700 1 -- 172.21.15.31:0/116261893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb4dc0042c0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4eaffd700 1 -- 172.21.15.31:0/116261893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc011940 con 0x7fb4ec1071f0 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4f284f700 1 -- 172.21.15.31:0/116261893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 msgr2=0x7fb4ec1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.957+0000 7fb4f284f700 1 --2- 172.21.15.31:0/116261893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0098a0 tx=0x7fb4dc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.958+0000 7fb4f284f700 1 -- 172.21.15.31:0/116261893 shutdown_connections 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.958+0000 7fb4f284f700 1 --2- 172.21.15.31:0/116261893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec1095e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.958+0000 7fb4f284f700 1 -- 172.21.15.31:0/116261893 >> 172.21.15.31:0/116261893 conn(0x7fb4ec100ba0 msgr2=0x7fb4ec102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.958+0000 7fb4f284f700 1 -- 172.21.15.31:0/116261893 shutdown_connections 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.958+0000 7fb4f284f700 1 -- 172.21.15.31:0/116261893 wait complete. 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.959+0000 7fb4f284f700 1 Processor -- start 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.959+0000 7fb4f284f700 1 -- start start 2023-10-06T21:11:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.959+0000 7fb4f284f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.959+0000 7fb4f284f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec19a770 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.959+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59756/0 (socket says 172.21.15.31:59756) 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4ebfff700 1 -- 172.21.15.31:0/684664137 learned_addr learned my addr 172.21.15.31:0/684664137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4ebfff700 1 -- 172.21.15.31:0/684664137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009550 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4ebfff700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011dc0 tx=0x7fb4dc011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc0037d0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb4dc01b440 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec19a970 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc01a560 con 0x7fb4ec1071f0 2023-10-06T21:11:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.960+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec19ae10 con 0x7fb4ec1071f0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.961+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fb4dc01a7d0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.962+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec10a7f0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.962+0000 7fb4e97fa700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 0x7fb4d403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.962+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb4dc04c7a0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.962+0000 7fb4eb7fe700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 0x7fb4d403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.963+0000 7fb4eb7fe700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 0x7fb4d403ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:05.967+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb4dc01b5b0 con 0x7fb4ec1071f0 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:06.185+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ec101f90 con 0x7fb4d4038870 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.555+0000 7fb4e97fa700 1 -- 172.21.15.31:0/684664137 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fb4ec101f90 con 0x7fb4d4038870 2023-10-06T21:11:10.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 msgr2=0x7fb4d403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 0x7fb4d403ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 msgr2=0x7fb4ec19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011dc0 tx=0x7fb4dc011ea0 comp rx=0 tx=0).stop 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 shutdown_connections 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fb4d4038870 0x7fb4d403ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 --2- 172.21.15.31:0/684664137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4ec1071f0 0x7fb4ec19a230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 >> 172.21.15.31:0/684664137 conn(0x7fb4ec100ba0 msgr2=0x7fb4ec101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 shutdown_connections 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:09.558+0000 7fb4f284f700 1 -- 172.21.15.31:0/684664137 wait complete. 2023-10-06T21:11:10.025 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2023-10-06T21:11:10.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:10 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:10.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:10 smithi031 ceph-mon[98539]: Added host smithi031 2023-10-06T21:11:10.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:10 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.582+0000 7fe1d80c1700 1 Processor -- start 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.582+0000 7fe1d80c1700 1 -- start start 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d80c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d80c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d01070c0 con 0x7fe1d01066d0 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d5e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:11.324 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d5e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59764/0 (socket says 172.21.15.31:59764) 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d5e5d700 1 -- 172.21.15.31:0/2217239699 learned_addr learned my addr 172.21.15.31:0/2217239699 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.583+0000 7fe1d5e5d700 1 -- 172.21.15.31:0/2217239699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d01078d0 con 0x7fe1d01066d0 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d5e5d700 1 --2- 172.21.15.31:0/2217239699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0098a0 tx=0x7fe1cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2318f3f66a0e0e68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d4e5b700 1 -- 172.21.15.31:0/2217239699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc004160 con 0x7fe1d01066d0 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d4e5b700 1 -- 172.21.15.31:0/2217239699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe1cc0042c0 con 0x7fe1d01066d0 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d4e5b700 1 -- 172.21.15.31:0/2217239699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc011940 con 0x7fe1d01066d0 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d80c1700 1 -- 172.21.15.31:0/2217239699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 msgr2=0x7fe1d0106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.584+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/2217239699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0098a0 tx=0x7fe1cc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.585+0000 7fe1d80c1700 1 -- 172.21.15.31:0/2217239699 shutdown_connections 2023-10-06T21:11:11.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.585+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/2217239699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d0106af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.585+0000 7fe1d80c1700 1 -- 172.21.15.31:0/2217239699 >> 172.21.15.31:0/2217239699 conn(0x7fe1d0101cd0 msgr2=0x7fe1d01040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.585+0000 7fe1d80c1700 1 -- 172.21.15.31:0/2217239699 shutdown_connections 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.585+0000 7fe1d80c1700 1 -- 172.21.15.31:0/2217239699 wait complete. 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.586+0000 7fe1d80c1700 1 Processor -- start 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.586+0000 7fe1d80c1700 1 -- start start 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.586+0000 7fe1d80c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.586+0000 7fe1d80c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d019a7c0 con 0x7fe1d01066d0 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.586+0000 7fe1d5e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:11.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1d5e5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59776/0 (socket says 172.21.15.31:59776) 2023-10-06T21:11:11.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1d5e5d700 1 -- 172.21.15.31:0/1840986713 learned_addr learned my addr 172.21.15.31:0/1840986713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:11.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1d5e5d700 1 -- 172.21.15.31:0/1840986713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc009550 con 0x7fe1d01066d0 2023-10-06T21:11:11.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1d5e5d700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc000c00 tx=0x7fe1cc011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:11.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc0036a0 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe1cc003c70 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d019a9c0 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.587+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc0193f0 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.588+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d019ae60 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.589+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fe1cc01b440 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.589+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d00624e0 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.589+0000 7fe1c6ffd700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 0x7fe1bc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.589+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe1cc04c660 con 0x7fe1d01066d0 2023-10-06T21:11:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.589+0000 7fe1d565c700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 0x7fe1bc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.590+0000 7fe1d565c700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 0x7fe1bc03ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0006fd0 tx=0x7fe1c0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.595+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1cc022950 con 0x7fe1d01066d0 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.820+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe1d0103150 con 0x7fe1bc038870 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.828+0000 7fe1c6ffd700 1 -- 172.21.15.31:0/1840986713 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe1d0103150 con 0x7fe1bc038870 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 msgr2=0x7fe1bc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 0x7fe1bc03ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0006fd0 tx=0x7fe1c0006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 msgr2=0x7fe1d019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc000c00 tx=0x7fe1cc011e70 comp rx=0 tx=0).stop 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 shutdown_connections 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fe1bc038870 0x7fe1bc03ad30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:11.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 --2- 172.21.15.31:0/1840986713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1d01066d0 0x7fe1d019a280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.830+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 >> 172.21.15.31:0/1840986713 conn(0x7fe1d0101cd0 msgr2=0x7fe1d0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.831+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 shutdown_connections 2023-10-06T21:11:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:10.831+0000 7fe1d80c1700 1 -- 172.21.15.31:0/1840986713 wait complete. 2023-10-06T21:11:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2023-10-06T21:11:11.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:11 smithi031 ceph-mon[98539]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:11.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:11 smithi031 ceph-mon[98539]: Saving service mon spec with placement count:5 2023-10-06T21:11:11.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:11 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:12.757 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-06T21:11:12.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.055+0000 7f4f14b53700 1 Processor -- start 2023-10-06T21:11:12.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.055+0000 7f4f14b53700 1 -- start start 2023-10-06T21:11:12.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.055+0000 7f4f14b53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:12.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.055+0000 7f4f14b53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f101070c0 con 0x7f4f101066d0 2023-10-06T21:11:12.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.056+0000 7f4f0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:12.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.056+0000 7f4f0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59792/0 (socket says 172.21.15.31:59792) 2023-10-06T21:11:12.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.056+0000 7f4f0e59c700 1 -- 172.21.15.31:0/79922516 learned_addr learned my addr 172.21.15.31:0/79922516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.056+0000 7f4f0e59c700 1 -- 172.21.15.31:0/79922516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f101078d0 con 0x7f4f101066d0 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.056+0000 7f4f0e59c700 1 --2- 172.21.15.31:0/79922516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f000098a0 tx=0x7f4f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c8736329163c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f0d59a700 1 -- 172.21.15.31:0/79922516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f00004160 con 0x7f4f101066d0 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f0d59a700 1 -- 172.21.15.31:0/79922516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4f000042c0 con 0x7f4f101066d0 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f0d59a700 1 -- 172.21.15.31:0/79922516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f00011940 con 0x7f4f101066d0 2023-10-06T21:11:12.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 -- 172.21.15.31:0/79922516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 msgr2=0x7f4f10106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:12.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 --2- 172.21.15.31:0/79922516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f000098a0 tx=0x7f4f00009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:12.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 -- 172.21.15.31:0/79922516 shutdown_connections 2023-10-06T21:11:12.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 --2- 172.21.15.31:0/79922516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f10106af0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 -- 172.21.15.31:0/79922516 >> 172.21.15.31:0/79922516 conn(0x7f4f10101cd0 msgr2=0x7f4f101040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 -- 172.21.15.31:0/79922516 shutdown_connections 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.057+0000 7f4f14b53700 1 -- 172.21.15.31:0/79922516 wait complete. 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.059+0000 7f4f14b53700 1 Processor -- start 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.059+0000 7f4f14b53700 1 -- start start 2023-10-06T21:11:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f14b53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f14b53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1019a7c0 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59806/0 (socket says 172.21.15.31:59806) 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f0e59c700 1 -- 172.21.15.31:0/3548437125 learned_addr learned my addr 172.21.15.31:0/3548437125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f0e59c700 1 -- 172.21.15.31:0/3548437125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f00009550 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.060+0000 7f4f0e59c700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f00000c00 tx=0x7f4f00011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.061+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f000036a0 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.061+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4f00003c70 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.061+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f000193f0 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.061+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1019a9c0 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.061+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1019ae60 con 0x7f4f101066d0 2023-10-06T21:11:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.062+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f4f0001b440 con 0x7f4f101066d0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.062+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f100624e0 con 0x7f4f101066d0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.062+0000 7f4eff7fe700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 0x7f4ef803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.062+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4f0004c660 con 0x7f4f101066d0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.063+0000 7f4f0dd9b700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 0x7f4ef803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.063+0000 7f4f0dd9b700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 0x7f4ef803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f04006fd0 tx=0x7f4f04006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.067+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4f00022950 con 0x7f4f101066d0 2023-10-06T21:11:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.276+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f0004c070 con 0x7f4f101066d0 2023-10-06T21:11:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.299+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4f10103150 con 0x7f4ef8038870 2023-10-06T21:11:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.307+0000 7f4eff7fe700 1 -- 172.21.15.31:0/3548437125 <== mgr.14116 v2:172.21.15.31:6800/484008040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4f10103150 con 0x7f4ef8038870 2023-10-06T21:11:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 msgr2=0x7f4ef803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 0x7f4ef803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f04006fd0 tx=0x7f4f04006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 msgr2=0x7f4f1019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f00000c00 tx=0x7f4f00011e70 comp rx=0 tx=0).stop 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 shutdown_connections 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4ef8038870 0x7f4ef803ad30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 --2- 172.21.15.31:0/3548437125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f101066d0 0x7f4f1019a280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 >> 172.21.15.31:0/3548437125 conn(0x7f4f10101cd0 msgr2=0x7f4f10102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 shutdown_connections 2023-10-06T21:11:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:12.309+0000 7f4f14b53700 1 -- 172.21.15.31:0/3548437125 wait complete. 2023-10-06T21:11:13.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:13 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:13.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:13 smithi031 ceph-mon[98539]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:13.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:13 smithi031 ceph-mon[98539]: Saving service mgr spec with placement count:2 2023-10-06T21:11:13.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:13 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:13.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:13 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.284+0000 7f1077f76700 1 Processor -- start 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.284+0000 7f1077f76700 1 -- start start 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.284+0000 7f1077f76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.284+0000 7f1077f76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701070f0 con 0x7f1070106700 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.285+0000 7f1075d12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.285+0000 7f1075d12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44434/0 (socket says 172.21.15.31:44434) 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.285+0000 7f1075d12700 1 -- 172.21.15.31:0/1115548797 learned_addr learned my addr 172.21.15.31:0/1115548797 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.285+0000 7f1075d12700 1 -- 172.21.15.31:0/1115548797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070107900 con 0x7f1070106700 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.285+0000 7f1075d12700 1 --2- 172.21.15.31:0/1115548797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106c0098a0 tx=0x7f106c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a88cb02ecc7a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1074d10700 1 -- 172.21.15.31:0/1115548797 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c004160 con 0x7f1070106700 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1074d10700 1 -- 172.21.15.31:0/1115548797 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f106c0042c0 con 0x7f1070106700 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1074d10700 1 -- 172.21.15.31:0/1115548797 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0118e0 con 0x7f1070106700 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1077f76700 1 -- 172.21.15.31:0/1115548797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 msgr2=0x7f1070106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1077f76700 1 --2- 172.21.15.31:0/1115548797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106c0098a0 tx=0x7f106c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1077f76700 1 -- 172.21.15.31:0/1115548797 shutdown_connections 2023-10-06T21:11:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1077f76700 1 --2- 172.21.15.31:0/1115548797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f1070106b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.286+0000 7f1077f76700 1 -- 172.21.15.31:0/1115548797 >> 172.21.15.31:0/1115548797 conn(0x7f1070101ce0 msgr2=0x7f1070104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.287+0000 7f1077f76700 1 -- 172.21.15.31:0/1115548797 shutdown_connections 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.287+0000 7f1077f76700 1 -- 172.21.15.31:0/1115548797 wait complete. 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.288+0000 7f1077f76700 1 Processor -- start 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.288+0000 7f1077f76700 1 -- start start 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.288+0000 7f1077f76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.288+0000 7f1077f76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107019a8f0 con 0x7f1070106700 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1075d12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1075d12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44444/0 (socket says 172.21.15.31:44444) 2023-10-06T21:11:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1075d12700 1 -- 172.21.15.31:0/866381043 learned_addr learned my addr 172.21.15.31:0/866381043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1075d12700 1 -- 172.21.15.31:0/866381043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009550 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1075d12700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f106c003e30 tx=0x7f106c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.289+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0036a0 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.290+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107019aaf0 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.290+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f106c01b440 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.290+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c023400 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.290+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107019af10 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.291+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10700624e0 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.291+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f106c023630 con 0x7f1070106700 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.291+0000 7f1066ffd700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 0x7f105c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.292+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f106c055760 con 0x7f1070106700 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.292+0000 7f1075511700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 0x7f105c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.293+0000 7f1075511700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 0x7f105c03ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.300+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f106c052060 con 0x7f1070106700 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.517+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f107019d9d0 con 0x7f1070106700 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.522+0000 7f1066ffd700 1 -- 172.21.15.31:0/866381043 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f106c052070 con 0x7f1070106700 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.524+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 msgr2=0x7f105c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.524+0000 7f1077f76700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 0x7f105c03ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 msgr2=0x7f107019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f106c003e30 tx=0x7f106c0119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 shutdown_connections 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f105c038870 0x7f105c03ad30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 --2- 172.21.15.31:0/866381043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1070106700 0x7f107019a3b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 >> 172.21.15.31:0/866381043 conn(0x7f1070101ce0 msgr2=0x7f10701033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 shutdown_connections 2023-10-06T21:11:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:13.525+0000 7f1077f76700 1 -- 172.21.15.31:0/866381043 wait complete. 2023-10-06T21:11:14.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:14 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/866381043' entity='client.admin' 2023-10-06T21:11:15.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.575+0000 7fa9d6eef700 1 Processor -- start 2023-10-06T21:11:15.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.575+0000 7fa9d6eef700 1 -- start start 2023-10-06T21:11:15.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.576+0000 7fa9d6eef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.576+0000 7fa9d6eef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d01070c0 con 0x7fa9d01066d0 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.576+0000 7fa9d4c8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.576+0000 7fa9d4c8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44460/0 (socket says 172.21.15.31:44460) 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.576+0000 7fa9d4c8b700 1 -- 172.21.15.31:0/2483432953 learned_addr learned my addr 172.21.15.31:0/2483432953 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.577+0000 7fa9d4c8b700 1 -- 172.21.15.31:0/2483432953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d01078d0 con 0x7fa9d01066d0 2023-10-06T21:11:15.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.577+0000 7fa9d4c8b700 1 --2- 172.21.15.31:0/2483432953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40098a0 tx=0x7fa9c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d71c679bb8768d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.577+0000 7fa9cf7fe700 1 -- 172.21.15.31:0/2483432953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c4004160 con 0x7fa9d01066d0 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.577+0000 7fa9cf7fe700 1 -- 172.21.15.31:0/2483432953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9c40042c0 con 0x7fa9d01066d0 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.577+0000 7fa9cf7fe700 1 -- 172.21.15.31:0/2483432953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c40118e0 con 0x7fa9d01066d0 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2483432953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 msgr2=0x7fa9d0106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2483432953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40098a0 tx=0x7fa9c4009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2483432953 shutdown_connections 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2483432953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d0106af0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2483432953 >> 172.21.15.31:0/2483432953 conn(0x7fa9d0101cd0 msgr2=0x7fa9d01040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.578+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2483432953 shutdown_connections 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.579+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2483432953 wait complete. 2023-10-06T21:11:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.579+0000 7fa9d6eef700 1 Processor -- start 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d6eef700 1 -- start start 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d6eef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d6eef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d019ca50 con 0x7fa9d01066d0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d4c8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d4c8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44476/0 (socket says 172.21.15.31:44476) 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.580+0000 7fa9d4c8b700 1 -- 172.21.15.31:0/2196560743 learned_addr learned my addr 172.21.15.31:0/2196560743 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9d4c8b700 1 -- 172.21.15.31:0/2196560743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c4009550 con 0x7fa9d01066d0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9d4c8b700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003e30 tx=0x7fa9c40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c40036a0 con 0x7fa9d01066d0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9c401b440 con 0x7fa9d01066d0 2023-10-06T21:11:15.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c4023400 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.581+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d019cc50 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.582+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d019d0f0 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.583+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fa9c4003800 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.583+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d00624e0 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.583+0000 7fa9cdffb700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 0x7fa9c003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.583+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa9c404c7e0 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.584+0000 7fa9cffff700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 0x7fa9c003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.584+0000 7fa9cffff700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 0x7fa9c003ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006fd0 tx=0x7fa9bc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.589+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa9c4003ab0 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.807+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa9d010aad0 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.810+0000 7fa9cdffb700 1 -- 172.21.15.31:0/2196560743 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa9c402b350 con 0x7fa9d01066d0 2023-10-06T21:11:15.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 msgr2=0x7fa9c003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 0x7fa9c003ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006fd0 tx=0x7fa9bc006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 msgr2=0x7fa9d019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003e30 tx=0x7fa9c40119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 shutdown_connections 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa9c0038820 0x7fa9c003ace0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 --2- 172.21.15.31:0/2196560743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d01066d0 0x7fa9d019c510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 >> 172.21.15.31:0/2196560743 conn(0x7fa9d0101cd0 msgr2=0x7fa9d0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 shutdown_connections 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:14.813+0000 7fa9d6eef700 1 -- 172.21.15.31:0/2196560743 wait complete. 2023-10-06T21:11:15.536 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2023-10-06T21:11:16.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:15 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2196560743' entity='client.admin' 2023-10-06T21:11:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:15 smithi031 ceph-mon[98539]: from='mgr.14116 172.21.15.31:0/3190373628' entity='mgr.x' 2023-10-06T21:11:17.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:16 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/249366282' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-06T21:11:17.132 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:16 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setuser ceph since I am not root 2023-10-06T21:11:17.132 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:16 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setgroup ceph since I am not root 2023-10-06T21:11:17.132 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:17.128+0000 7fa2e83b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:11:17.317 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.068+0000 7f4d63fff700 1 Processor -- start 2023-10-06T21:11:17.322 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.069+0000 7f4d63fff700 1 -- start start 2023-10-06T21:11:17.325 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.069+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.069+0000 7f4d63fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6407a8a0 con 0x7f4d6407be70 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d62ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d62ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44480/0 (socket says 172.21.15.31:44480) 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d62ffd700 1 -- 172.21.15.31:0/3261441046 learned_addr learned my addr 172.21.15.31:0/3261441046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d62ffd700 1 -- 172.21.15.31:0/3261441046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6407a9e0 con 0x7f4d6407be70 2023-10-06T21:11:17.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d62ffd700 1 --2- 172.21.15.31:0/3261441046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f0aa5e56ed110 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.070+0000 7f4d61ffb700 1 -- 172.21.15.31:0/3261441046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54004160 con 0x7f4d6407be70 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d61ffb700 1 -- 172.21.15.31:0/3261441046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d540042c0 con 0x7f4d6407be70 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d61ffb700 1 -- 172.21.15.31:0/3261441046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540118e0 con 0x7f4d6407be70 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d63fff700 1 -- 172.21.15.31:0/3261441046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 msgr2=0x7f4d6407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d63fff700 1 --2- 172.21.15.31:0/3261441046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d63fff700 1 -- 172.21.15.31:0/3261441046 shutdown_connections 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.071+0000 7f4d63fff700 1 --2- 172.21.15.31:0/3261441046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6407a2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.072+0000 7f4d63fff700 1 -- 172.21.15.31:0/3261441046 >> 172.21.15.31:0/3261441046 conn(0x7f4d64101ae0 msgr2=0x7f4d64103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.072+0000 7f4d63fff700 1 -- 172.21.15.31:0/3261441046 shutdown_connections 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.072+0000 7f4d63fff700 1 -- 172.21.15.31:0/3261441046 wait complete. 2023-10-06T21:11:17.327 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.073+0000 7f4d63fff700 1 Processor -- start 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.073+0000 7f4d63fff700 1 -- start start 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.073+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.073+0000 7f4d63fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6419c8c0 con 0x7f4d6407be70 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.074+0000 7f4d62ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.074+0000 7f4d62ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44494/0 (socket says 172.21.15.31:44494) 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.074+0000 7f4d62ffd700 1 -- 172.21.15.31:0/249366282 learned_addr learned my addr 172.21.15.31:0/249366282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.074+0000 7f4d62ffd700 1 -- 172.21.15.31:0/249366282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009550 con 0x7f4d6407be70 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.074+0000 7f4d62ffd700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006af0 tx=0x7f4d54011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.075+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540037d0 con 0x7f4d6407be70 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.075+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d5401b440 con 0x7f4d6407be70 2023-10-06T21:11:17.328 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.075+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6419cac0 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.075+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54023400 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.075+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6419cee0 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.076+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f4d54003930 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.076+0000 7f4d4bfff700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 0x7f4d4c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.076+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640624e0 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.076+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4d5404c9a0 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.077+0000 7f4d627fc700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 0x7f4d4c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.077+0000 7f4d627fc700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 0x7f4d4c03ac90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006fd0 tx=0x7f4d58006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.081+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4d5402b350 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.406+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f4d6410c900 con 0x7f4d6407be70 2023-10-06T21:11:17.329 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.818+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4d5402b350 con 0x7f4d6407be70 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.820+0000 7f4d4bfff700 1 -- 172.21.15.31:0/249366282 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f4d54048020 con 0x7f4d6407be70 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.822+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 msgr2=0x7f4d4c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.822+0000 7f4d63fff700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 0x7f4d4c03ac90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006fd0 tx=0x7f4d58006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.822+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 msgr2=0x7f4d6419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.822+0000 7f4d63fff700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006af0 tx=0x7f4d54011ed0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 shutdown_connections 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7f4d4c0387d0 0x7f4d4c03ac90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 --2- 172.21.15.31:0/249366282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6407be70 0x7f4d6419c380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 >> 172.21.15.31:0/249366282 conn(0x7f4d64101ae0 msgr2=0x7f4d641028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 shutdown_connections 2023-10-06T21:11:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:16.823+0000 7f4d63fff700 1 -- 172.21.15.31:0/249366282 wait complete. 2023-10-06T21:11:17.495 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:17.237+0000 7fa2e83b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:11:17.495 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:17.360+0000 7fa2e83b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:11:17.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:17.491+0000 7fa2e83b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:11:17.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:17.730+0000 7fa2e83b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:11:18.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/249366282' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-06T21:11:18.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:17 smithi031 ceph-mon[98539]: mgrmap e9: x(active, since 17s) 2023-10-06T21:11:18.506 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:18.210+0000 7fa2e83b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:11:18.507 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:18.383+0000 7fa2e83b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:11:18.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:11:18.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.907+0000 7fc278e29700 1 Processor -- start 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.907+0000 7fc278e29700 1 -- start start 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.908+0000 7fc278e29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.908+0000 7fc278e29700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27407a8e0 con 0x7fc27407beb0 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.908+0000 7fc27259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.908+0000 7fc27259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44510/0 (socket says 172.21.15.31:44510) 2023-10-06T21:11:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.908+0000 7fc27259c700 1 -- 172.21.15.31:0/1010521001 learned_addr learned my addr 172.21.15.31:0/1010521001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.909+0000 7fc27259c700 1 -- 172.21.15.31:0/1010521001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27407aa20 con 0x7fc27407beb0 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.909+0000 7fc27259c700 1 --2- 172.21.15.31:0/1010521001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c96b26141324a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.909+0000 7fc27159a700 1 -- 172.21.15.31:0/1010521001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc264004160 con 0x7fc27407beb0 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.909+0000 7fc27159a700 1 -- 172.21.15.31:0/1010521001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2640042c0 con 0x7fc27407beb0 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.909+0000 7fc27159a700 1 -- 172.21.15.31:0/1010521001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2640118e0 con 0x7fc27407beb0 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 -- 172.21.15.31:0/1010521001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 msgr2=0x7fc27407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 --2- 172.21.15.31:0/1010521001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 -- 172.21.15.31:0/1010521001 shutdown_connections 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 --2- 172.21.15.31:0/1010521001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27407a310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 -- 172.21.15.31:0/1010521001 >> 172.21.15.31:0/1010521001 conn(0x7fc274101ce0 msgr2=0x7fc274104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 -- 172.21.15.31:0/1010521001 shutdown_connections 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.910+0000 7fc278e29700 1 -- 172.21.15.31:0/1010521001 wait complete. 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.911+0000 7fc278e29700 1 Processor -- start 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.911+0000 7fc278e29700 1 -- start start 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc278e29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc278e29700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27419ca30 con 0x7fc27407beb0 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc27259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc27259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44512/0 (socket says 172.21.15.31:44512) 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc27259c700 1 -- 172.21.15.31:0/3257695452 learned_addr learned my addr 172.21.15.31:0/3257695452 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.912+0000 7fc27259c700 1 -- 172.21.15.31:0/3257695452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264009550 con 0x7fc27407beb0 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc27259c700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc264003e30 tx=0x7fc2640119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2640036a0 con 0x7fc27407beb0 2023-10-06T21:11:18.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc26401b440 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc264023400 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27419cc30 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.913+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27419d050 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.914+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fc264003800 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.914+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2740624e0 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.915+0000 7fc2637fe700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 0x7fc25c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.915+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc26404c7f0 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.915+0000 7fc271d9b700 1 -- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 msgr2=0x7fc25c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.915+0000 7fc271d9b700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 0x7fc25c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:17.920+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc26402b350 con 0x7fc27407beb0 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.116+0000 7fc271d9b700 1 -- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 msgr2=0x7fc25c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:18.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.116+0000 7fc271d9b700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 0x7fc25c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.187+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc27410caa0 con 0x7fc27407beb0 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.188+0000 7fc2637fe700 1 -- 172.21.15.31:0/3257695452 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc26402b350 con 0x7fc27407beb0 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 msgr2=0x7fc25c03ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 0x7fc25c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 msgr2=0x7fc27419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc264003e30 tx=0x7fc2640119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 shutdown_connections 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fc25c0388c0 0x7fc25c03ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 --2- 172.21.15.31:0/3257695452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27407beb0 0x7fc27419c4f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 >> 172.21.15.31:0/3257695452 conn(0x7fc274101ce0 msgr2=0x7fc274103330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 shutdown_connections 2023-10-06T21:11:18.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:18.190+0000 7fc278e29700 1 -- 172.21.15.31:0/3257695452 wait complete. 2023-10-06T21:11:18.611 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2023-10-06T21:11:18.611 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2023-10-06T21:11:18.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:18.503+0000 7fa2e83b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:11:18.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:18.634+0000 7fa2e83b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:11:19.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:18 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3257695452' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T21:11:19.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:19.231+0000 7fa2e83b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:11:20.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:20.102+0000 7fa2e83b3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:11:20.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:20.210+0000 7fa2e83b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:11:21.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:21.194+0000 7fa2e83b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:11:21.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:21.318+0000 7fa2e83b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:11:21.930 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:21.548+0000 7fa2e83b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:11:22.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:21.927+0000 7fa2e83b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:11:22.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:22.210+0000 7fa2e83b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:11:22.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:22.409+0000 7fa2e83b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:11:23.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:22.769+0000 7fa2e83b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:11:24.009 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:23.736+0000 7fa2e83b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:11:24.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:23.917+0000 7fa2e83b3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:11:24.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:24.172+0000 7fa2e83b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:11:24.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:24.322+0000 7fa2e83b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:11:25.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:24.606+0000 7fa2e83b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:11:25.713+0000 7fa2e83b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: Active manager daemon x restarted 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: Activating manager daemon x 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: osdmap e3: 0 total, 0 up, 0 in 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: mgrmap e10: x(active, starting, since 0.00418133s) 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:11:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: Manager daemon x is now available 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:26.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:11:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:11:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:26 smithi031 ceph-mon[98539]: mgrmap e11: x(active, since 1.00766s) 2023-10-06T21:11:27.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.166+0000 7fa7a94e4700 1 Processor -- start 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.166+0000 7fa7a94e4700 1 -- start start 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.167+0000 7fa7a94e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.167+0000 7fa7a94e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a407a8e0 con 0x7fa7a407beb0 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.167+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.167+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44528/0 (socket says 172.21.15.31:44528) 2023-10-06T21:11:27.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.167+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2102561234 learned_addr learned my addr 172.21.15.31:0/2102561234 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:27.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.168+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2102561234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a407aa20 con 0x7fa7a407beb0 2023-10-06T21:11:27.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.168+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2102561234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7940098a0 tx=0x7fa794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b258d82d045b0d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:27.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.168+0000 7fa7a1ffb700 1 -- 172.21.15.31:0/2102561234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa794004160 con 0x7fa7a407beb0 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.168+0000 7fa7a1ffb700 1 -- 172.21.15.31:0/2102561234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7940042c0 con 0x7fa7a407beb0 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.168+0000 7fa7a1ffb700 1 -- 172.21.15.31:0/2102561234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7940118e0 con 0x7fa7a407beb0 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2102561234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 msgr2=0x7fa7a407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2102561234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7940098a0 tx=0x7fa794009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2102561234 shutdown_connections 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2102561234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a407a310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2102561234 >> 172.21.15.31:0/2102561234 conn(0x7fa7a4101ce0 msgr2=0x7fa7a4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2102561234 shutdown_connections 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.169+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2102561234 wait complete. 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.171+0000 7fa7a94e4700 1 Processor -- start 2023-10-06T21:11:27.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.171+0000 7fa7a94e4700 1 -- start start 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.171+0000 7fa7a94e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.171+0000 7fa7a94e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a419caa0 con 0x7fa7a407beb0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.172+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.172+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44532/0 (socket says 172.21.15.31:44532) 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.172+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2400295556 learned_addr learned my addr 172.21.15.31:0/2400295556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.172+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794009550 con 0x7fa7a407beb0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.172+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa794003e30 tx=0x7fa7940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.173+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7940036a0 con 0x7fa7a407beb0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.173+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa79401b440 con 0x7fa7a407beb0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.173+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa794023400 con 0x7fa7a407beb0 2023-10-06T21:11:27.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.173+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a419cca0 con 0x7fa7a407beb0 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.173+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a419d0c0 con 0x7fa7a407beb0 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.174+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fa79401b5b0 con 0x7fa7a407beb0 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.175+0000 7fa78bfff700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.175+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa78c03b4e0 con 0x7fa78c038910 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.175+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa79404c8c0 con 0x7fa7a407beb0 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.175+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.175+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.376+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.376+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.777+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:19.777+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:20.578+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:20.578+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:22.180+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:22.180+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:25.383+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/484008040 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:25.383+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:25.718+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) v1 ==== 45359+0+0 (secure 0 0 0) 0x7fa794003800 con 0x7fa7a407beb0 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:25.718+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 msgr2=0x7fa78c03add0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:25.718+0000 7fa78bfff700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.720+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) v1 ==== 45486+0+0 (secure 0 0 0) 0x7fa79404e340 con 0x7fa7a407beb0 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.720+0000 7fa78bfff700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 0x7fa78c03e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.720+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa78c03b4e0 con 0x7fa78c03c3b0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.721+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 0x7fa78c03e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.722+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 0x7fa78c03e7a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa798003a10 tx=0x7fa7980092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.723+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa78c03b4e0 con 0x7fa78c03c3b0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.730+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa7a419d8a0 con 0x7fa78c03c3b0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.730+0000 7fa78bfff700 1 -- 172.21.15.31:0/2400295556 <== mgr.14148 v2:172.21.15.31:6800/964869793 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa7a419d8a0 con 0x7fa78c03c3b0 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 msgr2=0x7fa78c03e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 0x7fa78c03e7a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa798003a10 tx=0x7fa7980092b0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 msgr2=0x7fa7a419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa794003e30 tx=0x7fa7940119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 shutdown_connections 2023-10-06T21:11:27.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa78c03c3b0 0x7fa78c03e7a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a407beb0 0x7fa7a419c560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 --2- 172.21.15.31:0/2400295556 >> [v2:172.21.15.31:6800/484008040,v1:172.21.15.31:6801/484008040] conn(0x7fa78c038910 0x7fa78c03add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 >> 172.21.15.31:0/2400295556 conn(0x7fa7a4101ce0 msgr2=0x7fa7a4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 shutdown_connections 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:26.731+0000 7fa7a94e4700 1 -- 172.21.15.31:0/2400295556 wait complete. 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2023-10-06T21:11:27.130 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:26] ENGINE Bus STARTING 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:26] ENGINE Serving on https://172.21.15.31:7150 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:26] ENGINE Serving on http://172.21.15.31:8765 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: [06/Oct/2023:21:11:26] ENGINE Bus STARTED 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T21:11:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:27 smithi031 ceph-mon[98539]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.873+0000 7ff28a1d5700 1 Processor -- start 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.874+0000 7ff28a1d5700 1 -- start start 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.874+0000 7ff28a1d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.874+0000 7ff28a1d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841070f0 con 0x7ff284106700 2023-10-06T21:11:28.704 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.875+0000 7ff2837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.875+0000 7ff2837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52036/0 (socket says 172.21.15.31:52036) 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.875+0000 7ff2837fe700 1 -- 172.21.15.31:0/4120531014 learned_addr learned my addr 172.21.15.31:0/4120531014 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.875+0000 7ff2837fe700 1 -- 172.21.15.31:0/4120531014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284107900 con 0x7ff284106700 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.875+0000 7ff2837fe700 1 --2- 172.21.15.31:0/4120531014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098a0 tx=0x7ff274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5f09fff59794d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff2827fc700 1 -- 172.21.15.31:0/4120531014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff274004160 con 0x7ff284106700 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff2827fc700 1 -- 172.21.15.31:0/4120531014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2740042c0 con 0x7ff284106700 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff2827fc700 1 -- 172.21.15.31:0/4120531014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2740118e0 con 0x7ff284106700 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 -- 172.21.15.31:0/4120531014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 msgr2=0x7ff284106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 --2- 172.21.15.31:0/4120531014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098a0 tx=0x7ff274009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 -- 172.21.15.31:0/4120531014 shutdown_connections 2023-10-06T21:11:28.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 --2- 172.21.15.31:0/4120531014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff284106b20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 -- 172.21.15.31:0/4120531014 >> 172.21.15.31:0/4120531014 conn(0x7ff284101ce0 msgr2=0x7ff284104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 -- 172.21.15.31:0/4120531014 shutdown_connections 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.876+0000 7ff28a1d5700 1 -- 172.21.15.31:0/4120531014 wait complete. 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.877+0000 7ff28a1d5700 1 Processor -- start 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.877+0000 7ff28a1d5700 1 -- start start 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff28a1d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff28a1d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419cac0 con 0x7ff284106700 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff2837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff2837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52040/0 (socket says 172.21.15.31:52040) 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff2837fe700 1 -- 172.21.15.31:0/389297995 learned_addr learned my addr 172.21.15.31:0/389297995 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.878+0000 7ff2837fe700 1 -- 172.21.15.31:0/389297995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009550 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff2837fe700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff274003e30 tx=0x7ff2740119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2740036a0 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff27401b440 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff28a1d5700 1 -- 172.21.15.31:0/389297995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28419ccc0 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff274023400 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.879+0000 7ff28a1d5700 1 -- 172.21.15.31:0/389297995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28419d0e0 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.880+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7ff274023630 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.880+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2640052f0 con 0x7ff284106700 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.880+0000 7ff280ff9700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 0x7ff26c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.881+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff27404cb80 con 0x7ff284106700 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.881+0000 7ff282ffd700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 0x7ff26c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.881+0000 7ff282ffd700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 0x7ff26c03ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:27.887+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff274003bc0 con 0x7ff284106700 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.119+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff264000bc0 con 0x7ff26c0388c0 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.162+0000 7ff280ff9700 1 -- 172.21.15.31:0/389297995 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff264000bc0 con 0x7ff26c0388c0 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 msgr2=0x7ff26c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 0x7ff26c03ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 msgr2=0x7ff28419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff274003e30 tx=0x7ff2740119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 shutdown_connections 2023-10-06T21:11:28.708 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff26c0388c0 0x7ff26c03ad80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.709 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 --2- 172.21.15.31:0/389297995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff284106700 0x7ff28419c580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.709 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 >> 172.21.15.31:0/389297995 conn(0x7ff284101ce0 msgr2=0x7ff284102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.709 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 shutdown_connections 2023-10-06T21:11:28.709 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:28.164+0000 7ff26a7fc700 1 -- 172.21.15.31:0/389297995 wait complete. 2023-10-06T21:11:28.709 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2023-10-06T21:11:28.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:28 smithi031 ceph-mon[98539]: mgrmap e12: x(active, since 2s) 2023-10-06T21:11:28.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:28.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:28.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:29 smithi031 ceph-mon[98539]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:30.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$es46zWrWSFDPPh06nVSa2Ot9rGHG9y4UmMexIHRWwVslAmJi.TxqS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696626689, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.274+0000 7f85674c4700 1 Processor -- start 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.274+0000 7f85674c4700 1 -- start start 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.274+0000 7f85674c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.274+0000 7f85674c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85601070f0 con 0x7f8560106700 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8565260700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8565260700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52052/0 (socket says 172.21.15.31:52052) 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8565260700 1 -- 172.21.15.31:0/2034347820 learned_addr learned my addr 172.21.15.31:0/2034347820 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8565260700 1 -- 172.21.15.31:0/2034347820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560107900 con 0x7f8560106700 2023-10-06T21:11:30.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8565260700 1 --2- 172.21.15.31:0/2034347820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f855c0098a0 tx=0x7f855c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b950c0ade3fcc2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.275+0000 7f8557fff700 1 -- 172.21.15.31:0/2034347820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855c004160 con 0x7f8560106700 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f8557fff700 1 -- 172.21.15.31:0/2034347820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f855c0042c0 con 0x7f8560106700 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f8557fff700 1 -- 172.21.15.31:0/2034347820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855c0118e0 con 0x7f8560106700 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f85674c4700 1 -- 172.21.15.31:0/2034347820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 msgr2=0x7f8560106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f85674c4700 1 --2- 172.21.15.31:0/2034347820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f855c0098a0 tx=0x7f855c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f85674c4700 1 -- 172.21.15.31:0/2034347820 shutdown_connections 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f85674c4700 1 --2- 172.21.15.31:0/2034347820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f8560106b20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.276+0000 7f85674c4700 1 -- 172.21.15.31:0/2034347820 >> 172.21.15.31:0/2034347820 conn(0x7f8560101ce0 msgr2=0x7f8560104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.277+0000 7f85674c4700 1 -- 172.21.15.31:0/2034347820 shutdown_connections 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.277+0000 7f85674c4700 1 -- 172.21.15.31:0/2034347820 wait complete. 2023-10-06T21:11:30.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f85674c4700 1 Processor -- start 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f85674c4700 1 -- start start 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f85674c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f85674c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856019cac0 con 0x7f8560106700 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f8565260700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f8565260700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52068/0 (socket says 172.21.15.31:52068) 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.278+0000 7f8565260700 1 -- 172.21.15.31:0/1403989652 learned_addr learned my addr 172.21.15.31:0/1403989652 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.279+0000 7f8565260700 1 -- 172.21.15.31:0/1403989652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c009550 con 0x7f8560106700 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.279+0000 7f8565260700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f855c003e30 tx=0x7f855c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.279+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855c0036a0 con 0x7f8560106700 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.279+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f855c01b440 con 0x7f8560106700 2023-10-06T21:11:30.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.279+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856019ccc0 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.280+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855c023400 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.280+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856019d0e0 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.281+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f855c023630 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.281+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85600624e0 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.281+0000 7f85567fc700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 0x7f854c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.281+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f855c04c930 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.282+0000 7f8564a5f700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 0x7f854c03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.282+0000 7f8564a5f700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 0x7f854c03add0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8550006fd0 tx=0x7f8550006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.291+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f855c018350 con 0x7f8560106700 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.533+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8560108750 con 0x7f854c038910 2023-10-06T21:11:30.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.787+0000 7f85567fc700 1 -- 172.21.15.31:0/1403989652 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8550009e20 con 0x7f854c038910 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 msgr2=0x7f854c03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 0x7f854c03add0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8550006fd0 tx=0x7f8550006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 msgr2=0x7f856019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f855c003e30 tx=0x7f855c0119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 shutdown_connections 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f854c038910 0x7f854c03add0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 --2- 172.21.15.31:0/1403989652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8560106700 0x7f856019c580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 >> 172.21.15.31:0/1403989652 conn(0x7f8560101ce0 msgr2=0x7f8560102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 shutdown_connections 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:29.789+0000 7f85674c4700 1 -- 172.21.15.31:0/1403989652 wait complete. 2023-10-06T21:11:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2023-10-06T21:11:30.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:30 smithi031 ceph-mon[98539]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:30.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.862+0000 7f44fef9b700 1 Processor -- start 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.862+0000 7f44fef9b700 1 -- start start 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.863+0000 7f44fef9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.863+0000 7f44fef9b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8079e20 con 0x7f44f81071d0 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.863+0000 7f44fcd37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.863+0000 7f44fcd37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52076/0 (socket says 172.21.15.31:52076) 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.863+0000 7f44fcd37700 1 -- 172.21.15.31:0/3390534440 learned_addr learned my addr 172.21.15.31:0/3390534440 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.864+0000 7f44fcd37700 1 -- 172.21.15.31:0/3390534440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8109b00 con 0x7f44f81071d0 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.864+0000 7f44fcd37700 1 --2- 172.21.15.31:0/3390534440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0098a0 tx=0x7f44ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94df11112e3ad1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.864+0000 7f44f77fe700 1 -- 172.21.15.31:0/3390534440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44ec004160 con 0x7f44f81071d0 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.864+0000 7f44f77fe700 1 -- 172.21.15.31:0/3390534440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44ec0042c0 con 0x7f44f81071d0 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.864+0000 7f44f77fe700 1 -- 172.21.15.31:0/3390534440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44ec0118e0 con 0x7f44f81071d0 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 -- 172.21.15.31:0/3390534440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 msgr2=0x7f44f81095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 --2- 172.21.15.31:0/3390534440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0098a0 tx=0x7f44ec009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 -- 172.21.15.31:0/3390534440 shutdown_connections 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 --2- 172.21.15.31:0/3390534440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f81095c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 -- 172.21.15.31:0/3390534440 >> 172.21.15.31:0/3390534440 conn(0x7f44f8100ba0 msgr2=0x7f44f8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 -- 172.21.15.31:0/3390534440 shutdown_connections 2023-10-06T21:11:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.865+0000 7f44fef9b700 1 -- 172.21.15.31:0/3390534440 wait complete. 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.866+0000 7f44fef9b700 1 Processor -- start 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fef9b700 1 -- start start 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fef9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fef9b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f819ca20 con 0x7f44f81071d0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fcd37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fcd37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52086/0 (socket says 172.21.15.31:52086) 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.867+0000 7f44fcd37700 1 -- 172.21.15.31:0/893191330 learned_addr learned my addr 172.21.15.31:0/893191330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44fcd37700 1 -- 172.21.15.31:0/893191330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009550 con 0x7f44f81071d0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44fcd37700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44ec003e30 tx=0x7f44ec0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44ec0036a0 con 0x7f44f81071d0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44ec01b440 con 0x7f44f81071d0 2023-10-06T21:11:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f819cc20 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.868+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44ec023400 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.869+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f819d0c0 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.870+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f44ec023630 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.870+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f810a830 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.870+0000 7f44f5ffb700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 0x7f44e803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.871+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f44ec04c990 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.871+0000 7f44f7fff700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 0x7f44e803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.872+0000 7f44f7fff700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 0x7f44e803ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44e4006fd0 tx=0x7f44e4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:30.876+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f44ec018350 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.096+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f44f819fae0 con 0x7f44f81071d0 2023-10-06T21:11:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.097+0000 7f44f5ffb700 1 -- 172.21.15.31:0/893191330 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f44ec02bb40 con 0x7f44f81071d0 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 msgr2=0x7f44e803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 0x7f44e803ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44e4006fd0 tx=0x7f44e4006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 msgr2=0x7f44f819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44ec003e30 tx=0x7f44ec0119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 shutdown_connections 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f44e80388c0 0x7f44e803ad80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 --2- 172.21.15.31:0/893191330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44f81071d0 0x7f44f819c4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.099+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 >> 172.21.15.31:0/893191330 conn(0x7f44f8100ba0 msgr2=0x7f44f8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.100+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 shutdown_connections 2023-10-06T21:11:31.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:31.100+0000 7f44fef9b700 1 -- 172.21.15.31:0/893191330 wait complete. 2023-10-06T21:11:31.788 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:11:31.788 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-10-06T21:11:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:11:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-10-06T21:11:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-10-06T21:11:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-06T21:11:31.808 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: Password: ymz9htf8gx 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:31.809 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config directory 2023-10-06T21:11:31.810 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2023-10-06T21:11:32.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:31 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/893191330' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.342+0000 7f48ed5c8700 1 Processor -- start 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48ed5c8700 1 -- start start 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48ed5c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48ed5c8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81070c0 con 0x7f48e81066d0 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52090/0 (socket says 172.21.15.31:52090) 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.343+0000 7f48e6ffd700 1 -- 172.21.15.31:0/207565815 learned_addr learned my addr 172.21.15.31:0/207565815 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.344+0000 7f48e6ffd700 1 -- 172.21.15.31:0/207565815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e81078d0 con 0x7f48e81066d0 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.344+0000 7f48e6ffd700 1 --2- 172.21.15.31:0/207565815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f48d80098a0 tx=0x7f48d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254a260138e228d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.344+0000 7f48e5ffb700 1 -- 172.21.15.31:0/207565815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48d8004160 con 0x7f48e81066d0 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.344+0000 7f48e5ffb700 1 -- 172.21.15.31:0/207565815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d80042c0 con 0x7f48e81066d0 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.344+0000 7f48e5ffb700 1 -- 172.21.15.31:0/207565815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48d80118e0 con 0x7f48e81066d0 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 -- 172.21.15.31:0/207565815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 msgr2=0x7f48e8106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/207565815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f48d80098a0 tx=0x7f48d8009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:32.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 -- 172.21.15.31:0/207565815 shutdown_connections 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/207565815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e8106af0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 -- 172.21.15.31:0/207565815 >> 172.21.15.31:0/207565815 conn(0x7f48e8101cd0 msgr2=0x7f48e81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 -- 172.21.15.31:0/207565815 shutdown_connections 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.345+0000 7f48ed5c8700 1 -- 172.21.15.31:0/207565815 wait complete. 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.346+0000 7f48ed5c8700 1 Processor -- start 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.346+0000 7f48ed5c8700 1 -- start start 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48ed5c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48ed5c8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819a860 con 0x7f48e81066d0 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52102/0 (socket says 172.21.15.31:52102) 2023-10-06T21:11:32.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48e6ffd700 1 -- 172.21.15.31:0/2165821055 learned_addr learned my addr 172.21.15.31:0/2165821055 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48e6ffd700 1 -- 172.21.15.31:0/2165821055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8009550 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.347+0000 7f48e6ffd700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006af0 tx=0x7f48d8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.348+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48d80037d0 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.348+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d801b440 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.348+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48d8023400 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.348+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e819aa60 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.348+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e819af00 con 0x7f48e81066d0 2023-10-06T21:11:32.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.349+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e80624e0 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.349+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f48d801b5b0 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.349+0000 7f48cffff700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 0x7f48d003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.349+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48d804cab0 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.350+0000 7f48e67fc700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 0x7f48d003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.350+0000 7f48e67fc700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 0x7f48d003add0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48dc006fd0 tx=0x7f48dc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.354+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f48d801b8d0 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.563+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f48e810aad0 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.563+0000 7f48cffff700 1 -- 172.21.15.31:0/2165821055 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f48d8016260 con 0x7f48e81066d0 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 msgr2=0x7f48d003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 0x7f48d003add0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48dc006fd0 tx=0x7f48dc006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 msgr2=0x7f48e819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006af0 tx=0x7f48d8011ed0 comp rx=0 tx=0).stop 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 shutdown_connections 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f48d0038910 0x7f48d003add0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 --2- 172.21.15.31:0/2165821055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48e81066d0 0x7f48e819a320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 >> 172.21.15.31:0/2165821055 conn(0x7f48e8101cd0 msgr2=0x7f48e8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 shutdown_connections 2023-10-06T21:11:32.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:32.566+0000 7f48ed5c8700 1 -- 172.21.15.31:0/2165821055 wait complete. 2023-10-06T21:11:33.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:32 smithi031 ceph-mon[98539]: mgrmap e13: x(active, since 6s) 2023-10-06T21:11:34.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.517+0000 7ff52c9b1700 1 Processor -- start 2023-10-06T21:11:34.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.517+0000 7ff52c9b1700 1 -- start start 2023-10-06T21:11:34.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52c9b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:34.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52c9b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5281070c0 con 0x7ff5281066d0 2023-10-06T21:11:34.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45004/0 (socket says 172.21.15.31:45004) 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52659c700 1 -- 172.21.15.31:0/2428559231 learned_addr learned my addr 172.21.15.31:0/2428559231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.518+0000 7ff52659c700 1 -- 172.21.15.31:0/2428559231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5281078d0 con 0x7ff5281066d0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52659c700 1 --2- 172.21.15.31:0/2428559231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5180098a0 tx=0x7ff518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900c5b41764744bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52559a700 1 -- 172.21.15.31:0/2428559231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff518004160 con 0x7ff5281066d0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52559a700 1 -- 172.21.15.31:0/2428559231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5180042c0 con 0x7ff5281066d0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52559a700 1 -- 172.21.15.31:0/2428559231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5180118e0 con 0x7ff5281066d0 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52c9b1700 1 -- 172.21.15.31:0/2428559231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 msgr2=0x7ff528106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.519+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/2428559231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5180098a0 tx=0x7ff518009bb0 comp rx=0 tx=0).stop 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.520+0000 7ff52c9b1700 1 -- 172.21.15.31:0/2428559231 shutdown_connections 2023-10-06T21:11:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.520+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/2428559231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff528106af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.520+0000 7ff52c9b1700 1 -- 172.21.15.31:0/2428559231 >> 172.21.15.31:0/2428559231 conn(0x7ff528101cd0 msgr2=0x7ff5281040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.520+0000 7ff52c9b1700 1 -- 172.21.15.31:0/2428559231 shutdown_connections 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.520+0000 7ff52c9b1700 1 -- 172.21.15.31:0/2428559231 wait complete. 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52c9b1700 1 Processor -- start 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52c9b1700 1 -- start start 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52c9b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52c9b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52819ca50 con 0x7ff5281066d0 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45018/0 (socket says 172.21.15.31:45018) 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.521+0000 7ff52659c700 1 -- 172.21.15.31:0/1490943914 learned_addr learned my addr 172.21.15.31:0/1490943914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff52659c700 1 -- 172.21.15.31:0/1490943914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009550 con 0x7ff5281066d0 2023-10-06T21:11:34.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff52659c700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff518003e30 tx=0x7ff5180119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5180036a0 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff51801b440 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52819cc50 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff518023400 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.522+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52819d0f0 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.523+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7ff51801b5b0 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.524+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5280624e0 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.524+0000 7ff5177fe700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 0x7ff51003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.524+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff51804ca10 con 0x7ff5281066d0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.524+0000 7ff525d9b700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 0x7ff51003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:34.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.525+0000 7ff525d9b700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 0x7ff51003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.529+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff51800bc10 con 0x7ff5281066d0 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.851+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ff52810aad0 con 0x7ff5281066d0 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.854+0000 7ff5177fe700 1 -- 172.21.15.31:0/1490943914 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7ff518016260 con 0x7ff5281066d0 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 msgr2=0x7ff51003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 0x7ff51003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 msgr2=0x7ff52819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff518003e30 tx=0x7ff5180119a0 comp rx=0 tx=0).stop 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 shutdown_connections 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff510038910 0x7ff51003add0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 --2- 172.21.15.31:0/1490943914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5281066d0 0x7ff52819c510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.855+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 >> 172.21.15.31:0/1490943914 conn(0x7ff528101cd0 msgr2=0x7ff528102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:34.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.856+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 shutdown_connections 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-10-06T21:11:33.856+0000 7ff52c9b1700 1 -- 172.21.15.31:0/1490943914 wait complete. 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.382 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:34.383 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2023-10-06T21:11:34.435 INFO:tasks.cephadm:Fetching config... 2023-10-06T21:11:34.435 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:11:34.435 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-06T21:11:34.479 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-06T21:11:34.479 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:11:34.479 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-06T21:11:34.563 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-06T21:11:34.563 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:11:34.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-06T21:11:34.676 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-06T21:11:34.677 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:11:34.677 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-06T21:11:34.764 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-06T21:11:34.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:11:34.907 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:11:34.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:34 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1490943914' entity='client.admin' 2023-10-06T21:11:34.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:11:35.050 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:11:35.079 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:11:35.186 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC0DSq5Zb0Ho2IgZ77k1Crgn8GhBwQC5vsDSWlDvne5Qs/u6zOZzChfzD2xWDwLhFJRX2Zg2S2IsZS6CNLyeQapzJSAOBGCkIaOjsXT8XGCJyCfP4TkEyHYic6YpHZE7hhPyXiMWxgzlAsdTp18yrjwk/vw31+Z4grDa5QGyxurvVPISL7daPVz5yIPp9VykhnC+tsa6QvbmpK8m43rqIw4CVyqVXZOyK6Y3r5Dm2lt8apunYAxAXb+JPRsLqX4u58SSMufyU7LfxJCGBFGdN8bE/BZdFM6H95Y9+jtGnszglguHg51fX8rA40u8pQxp+QoaPwFFPnfbP/vxZzj9JBlOF06AfGHpN0sVCG/xQcsY9xU041zmS/UpImY3QXvSlbeoKfn65nEYH/EXcovZKTye7LfaZQ3mYqSrD3dDsq2ean9/Ouzhep1Sr3Zml1itMwLzQe5LbZyWX/K6EuqVNHEnaTQbPsRG1wcIu03DFNJ8ZllDgOuDGXOi1nqP82Qw8= ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:11:35.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-06T21:11:35.972 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 -- 172.21.15.31:0/2151060989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 msgr2=0x7f8870100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 --2- 172.21.15.31:0/2151060989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870100e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8860009910 tx=0x7f8860009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 -- 172.21.15.31:0/2151060989 shutdown_connections 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 --2- 172.21.15.31:0/2151060989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870100e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 -- 172.21.15.31:0/2151060989 >> 172.21.15.31:0/2151060989 conn(0x7f88700fbf80 msgr2=0x7f88700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 -- 172.21.15.31:0/2151060989 shutdown_connections 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.796+0000 7f88757fc700 1 -- 172.21.15.31:0/2151060989 wait complete. 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.797+0000 7f88757fc700 1 Processor -- start 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f88757fc700 1 -- start start 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f88757fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f88757fc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870195f50 con 0x7f8870100a00 2023-10-06T21:11:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f886effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f886effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45022/0 (socket says 172.21.15.31:45022) 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f886effd700 1 -- 172.21.15.31:0/2763695814 learned_addr learned my addr 172.21.15.31:0/2763695814 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.798+0000 7f886effd700 1 -- 172.21.15.31:0/2763695814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88600095f0 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f886effd700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8860005950 tx=0x7f8860005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886000ba90 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f886000bbf0 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8860017440 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8870196150 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.799+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88701965f0 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.800+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887018f3c0 con 0x7f8870100a00 2023-10-06T21:11:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.800+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f886000f460 con 0x7f8870100a00 2023-10-06T21:11:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.801+0000 7f8857fff700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 0x7f885803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.801+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f886004ca90 con 0x7f8870100a00 2023-10-06T21:11:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.801+0000 7f886e7fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 0x7f885803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.801+0000 7f886e7fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 0x7f885803add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8864006fd0 tx=0x7f8864006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:37.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:37.806+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f886002ad30 con 0x7f8870100a00 2023-10-06T21:11:38.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.016+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8870106600 con 0x7f8870100a00 2023-10-06T21:11:38.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.019+0000 7f8857fff700 1 -- 172.21.15.31:0/2763695814 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8860027070 con 0x7f8870100a00 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.021+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 msgr2=0x7f885803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.021+0000 7f88757fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 0x7f885803add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8864006fd0 tx=0x7f8864006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.021+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 msgr2=0x7f8870195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.021+0000 7f88757fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8860005950 tx=0x7f8860005810 comp rx=0 tx=0).stop 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 shutdown_connections 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8858038910 0x7f885803add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 --2- 172.21.15.31:0/2763695814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8870100a00 0x7f8870195a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 >> 172.21.15.31:0/2763695814 conn(0x7f88700fbf80 msgr2=0x7f88700fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 shutdown_connections 2023-10-06T21:11:38.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:38.022+0000 7f88757fc700 1 -- 172.21.15.31:0/2763695814 wait complete. 2023-10-06T21:11:38.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:38.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:38.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:11:38.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:38.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:38.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:38.554 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-06T21:11:38.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-06T21:11:39.285 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:39 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2763695814' entity='client.admin' 2023-10-06T21:11:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 -- 172.21.15.31:0/2583414014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 msgr2=0x7f16e0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 --2- 172.21.15.31:0/2583414014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0073340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009910 tx=0x7f16d0009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 -- 172.21.15.31:0/2583414014 shutdown_connections 2023-10-06T21:11:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 --2- 172.21.15.31:0/2583414014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0073340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 -- 172.21.15.31:0/2583414014 >> 172.21.15.31:0/2583414014 conn(0x7f16e00fbf80 msgr2=0x7f16e00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 -- 172.21.15.31:0/2583414014 shutdown_connections 2023-10-06T21:11:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.840+0000 7f16e5340700 1 -- 172.21.15.31:0/2583414014 wait complete. 2023-10-06T21:11:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.841+0000 7f16e5340700 1 Processor -- start 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.841+0000 7f16e5340700 1 -- start start 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.841+0000 7f16e5340700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.841+0000 7f16e5340700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e01981b0 con 0x7f16e0074ee0 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.841+0000 7f16deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45042/0 (socket says 172.21.15.31:45042) 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16deffd700 1 -- 172.21.15.31:0/1837345908 learned_addr learned my addr 172.21.15.31:0/1837345908 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16deffd700 1 -- 172.21.15.31:0/1837345908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00095f0 con 0x7f16e0074ee0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16deffd700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16d0005950 tx=0x7f16d0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16d000ba90 con 0x7f16e0074ee0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16d000bbf0 con 0x7f16e0074ee0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e01983b0 con 0x7f16e0074ee0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.842+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16d0017440 con 0x7f16e0074ee0 2023-10-06T21:11:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.843+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e0198850 con 0x7f16e0074ee0 2023-10-06T21:11:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.843+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f16d000f460 con 0x7f16e0074ee0 2023-10-06T21:11:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.844+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e01915d0 con 0x7f16e0074ee0 2023-10-06T21:11:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.844+0000 7f16c7fff700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 0x7f16c803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.844+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f16d004ca70 con 0x7f16e0074ee0 2023-10-06T21:11:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.844+0000 7f16de7fc700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 0x7f16c803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.845+0000 7f16de7fc700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 0x7f16c803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16d4006fd0 tx=0x7f16d4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:40.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:40.850+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f16d002ad30 con 0x7f16e0074ee0 2023-10-06T21:11:41.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.063+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f16e002d170 con 0x7f16c8038910 2023-10-06T21:11:41.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.066+0000 7f16c7fff700 1 -- 172.21.15.31:0/1837345908 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f16e002d170 con 0x7f16c8038910 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.068+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 msgr2=0x7f16c803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.068+0000 7f16e5340700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 0x7f16c803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16d4006fd0 tx=0x7f16d4006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.068+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 msgr2=0x7f16e0197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.068+0000 7f16e5340700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16d0005950 tx=0x7f16d0005810 comp rx=0 tx=0).stop 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 shutdown_connections 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16c8038910 0x7f16c803add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 --2- 172.21.15.31:0/1837345908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e0074ee0 0x7f16e0197c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 >> 172.21.15.31:0/1837345908 conn(0x7f16e00fbf80 msgr2=0x7f16e00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 shutdown_connections 2023-10-06T21:11:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:41.069+0000 7f16e5340700 1 -- 172.21.15.31:0/1837345908 wait complete. 2023-10-06T21:11:41.567 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2023-10-06T21:11:41.567 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:11:41.567 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2023-10-06T21:11:41.615 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:11:41.615 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:11:41.700 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2023-10-06T21:11:41.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi142 2023-10-06T21:11:42.463 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:11:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:42 smithi031 ceph-mon[98539]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:11:43.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:43 smithi031 ceph-mon[98539]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:11:43.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:43 smithi031 ceph-mon[98539]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:11:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 -- 172.21.15.31:0/2959502217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 msgr2=0x7f2ba4106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/2959502217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4106d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009910 tx=0x7f2ba0009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 -- 172.21.15.31:0/2959502217 shutdown_connections 2023-10-06T21:11:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/2959502217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4106d50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 -- 172.21.15.31:0/2959502217 >> 172.21.15.31:0/2959502217 conn(0x7f2ba40fc4a0 msgr2=0x7f2ba40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 -- 172.21.15.31:0/2959502217 shutdown_connections 2023-10-06T21:11:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.978+0000 7f2bab4f5700 1 -- 172.21.15.31:0/2959502217 wait complete. 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.979+0000 7f2bab4f5700 1 Processor -- start 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.979+0000 7f2bab4f5700 1 -- start start 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2bab4f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2bab4f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba41962a0 con 0x7f2ba4106970 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2ba9291700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2ba9291700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35592/0 (socket says 172.21.15.31:35592) 2023-10-06T21:11:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2ba9291700 1 -- 172.21.15.31:0/3510546565 learned_addr learned my addr 172.21.15.31:0/3510546565 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2ba9291700 1 -- 172.21.15.31:0/3510546565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00095f0 con 0x7f2ba4106970 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.980+0000 7f2ba9291700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0006010 tx=0x7f2ba0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.981+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba001d070 con 0x7f2ba4106970 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.981+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ba000bdf0 con 0x7f2ba4106970 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.981+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba0022d80 con 0x7f2ba4106970 2023-10-06T21:11:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.981+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4196500 con 0x7f2ba4106970 2023-10-06T21:11:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.981+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba4196980 con 0x7f2ba4106970 2023-10-06T21:11:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.982+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f2ba0022570 con 0x7f2ba4106970 2023-10-06T21:11:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.982+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba40624e0 con 0x7f2ba4106970 2023-10-06T21:11:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.982+0000 7f2b9a7fc700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 0x7f2b9003ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.982+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ba004caa0 con 0x7f2ba4106970 2023-10-06T21:11:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.983+0000 7f2ba8a90700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 0x7f2b9003ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.983+0000 7f2ba8a90700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 0x7f2b9003ae20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006fd0 tx=0x7f2b94006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:43.988+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ba002bba0 con 0x7f2ba4106970 2023-10-06T21:11:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:44.202+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ba40008d0 con 0x7f2b90038960 2023-10-06T21:11:44.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:45.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:45 smithi031 ceph-mon[98539]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:47.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:46 smithi031 ceph-mon[98539]: Deploying cephadm binary to smithi142 2023-10-06T21:11:47.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:46 smithi031 ceph-mon[98539]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:47.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.519+0000 7f2b9a7fc700 1 -- 172.21.15.31:0/3510546565 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2ba40008d0 con 0x7f2b90038960 2023-10-06T21:11:47.521 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi142' with addr '172.21.15.142' 2023-10-06T21:11:47.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 msgr2=0x7f2b9003ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:47.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 0x7f2b9003ae20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006fd0 tx=0x7f2b94006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:47.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 msgr2=0x7f2ba4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:47.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0006010 tx=0x7f2ba0004ff0 comp rx=0 tx=0).stop 2023-10-06T21:11:47.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 shutdown_connections 2023-10-06T21:11:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b90038960 0x7f2b9003ae20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 --2- 172.21.15.31:0/3510546565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4106970 0x7f2ba4195d60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 >> 172.21.15.31:0/3510546565 conn(0x7f2ba40fc4a0 msgr2=0x7f2ba4108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 shutdown_connections 2023-10-06T21:11:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:47.522+0000 7f2bab4f5700 1 -- 172.21.15.31:0/3510546565 wait complete. 2023-10-06T21:11:48.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T21:11:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:48 smithi031 ceph-mon[98539]: Added host smithi142 2023-10-06T21:11:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:48 smithi031 ceph-mon[98539]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:49 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 -- 172.21.15.31:0/2102648627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 msgr2=0x7f4da4106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/2102648627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4106d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009910 tx=0x7f4da0009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 -- 172.21.15.31:0/2102648627 shutdown_connections 2023-10-06T21:11:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/2102648627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4106d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 -- 172.21.15.31:0/2102648627 >> 172.21.15.31:0/2102648627 conn(0x7f4da40fc4a0 msgr2=0x7f4da40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 -- 172.21.15.31:0/2102648627 shutdown_connections 2023-10-06T21:11:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.318+0000 7f4dabd0d700 1 -- 172.21.15.31:0/2102648627 wait complete. 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.319+0000 7f4dabd0d700 1 Processor -- start 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.319+0000 7f4dabd0d700 1 -- start start 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.319+0000 7f4dabd0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.319+0000 7f4dabd0d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da41962a0 con 0x7f4da4106970 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4da9aa9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4da9aa9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35612/0 (socket says 172.21.15.31:35612) 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4da9aa9700 1 -- 172.21.15.31:0/1527346253 learned_addr learned my addr 172.21.15.31:0/1527346253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4da9aa9700 1 -- 172.21.15.31:0/1527346253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00095f0 con 0x7f4da4106970 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4da9aa9700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006010 tx=0x7f4da0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.320+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da001d070 con 0x7f4da4106970 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.321+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da4196500 con 0x7f4da4106970 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.321+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4da000bdf0 con 0x7f4da4106970 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.321+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da0022d80 con 0x7f4da4106970 2023-10-06T21:11:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.321+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da4196980 con 0x7f4da4106970 2023-10-06T21:11:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.322+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da40624e0 con 0x7f4da4106970 2023-10-06T21:11:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.322+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f4da0022570 con 0x7f4da4106970 2023-10-06T21:11:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.322+0000 7f4d9affd700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 0x7f4d9003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.323+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4da004cb00 con 0x7f4da4106970 2023-10-06T21:11:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.323+0000 7f4da92a8700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 0x7f4d9003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.323+0000 7f4da92a8700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 0x7f4d9003ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d94006fd0 tx=0x7f4d94006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.328+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4da004ee60 con 0x7f4da4106970 2023-10-06T21:11:50.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.546+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4da40008d0 con 0x7f4d900388c0 2023-10-06T21:11:50.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.547+0000 7f4d9affd700 1 -- 172.21.15.31:0/1527346253 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f4da40008d0 con 0x7f4d900388c0 2023-10-06T21:11:50.553 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:50.553 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 msgr2=0x7f4d9003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 0x7f4d9003ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d94006fd0 tx=0x7f4d94006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 msgr2=0x7f4da4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006010 tx=0x7f4da0004ff0 comp rx=0 tx=0).stop 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 shutdown_connections 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4d900388c0 0x7f4d9003ad80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 --2- 172.21.15.31:0/1527346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4da4106970 0x7f4da4195d60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 >> 172.21.15.31:0/1527346253 conn(0x7f4da40fc4a0 msgr2=0x7f4da4108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 shutdown_connections 2023-10-06T21:11:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:50.550+0000 7f4dabd0d700 1 -- 172.21.15.31:0/1527346253 wait complete. 2023-10-06T21:11:51.076 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2023-10-06T21:11:51.076 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:11:51.076 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2023-10-06T21:11:51.124 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:11:51.124 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:11:51.209 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2023-10-06T21:11:51.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi146 2023-10-06T21:11:51.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:50 smithi031 ceph-mon[98539]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:52.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:51 smithi031 ceph-mon[98539]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:11:53.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:52 smithi031 ceph-mon[98539]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:53.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 -- 172.21.15.31:0/84150845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 msgr2=0x7fe3a4106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 --2- 172.21.15.31:0/84150845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4106d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe394009910 tx=0x7fe394009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 -- 172.21.15.31:0/84150845 shutdown_connections 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 --2- 172.21.15.31:0/84150845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4106d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 -- 172.21.15.31:0/84150845 >> 172.21.15.31:0/84150845 conn(0x7fe3a40fc4a0 msgr2=0x7fe3a40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 -- 172.21.15.31:0/84150845 shutdown_connections 2023-10-06T21:11:53.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.418+0000 7fe3aa923700 1 -- 172.21.15.31:0/84150845 wait complete. 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.419+0000 7fe3aa923700 1 Processor -- start 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.419+0000 7fe3aa923700 1 -- start start 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3aa923700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3aa923700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41984d0 con 0x7fe3a4106970 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:53.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39506/0 (socket says 172.21.15.31:39506) 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3a3fff700 1 -- 172.21.15.31:0/3277790442 learned_addr learned my addr 172.21.15.31:0/3277790442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3a3fff700 1 -- 172.21.15.31:0/3277790442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3940095f0 con 0x7fe3a4106970 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.420+0000 7fe3a3fff700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe394006010 tx=0x7fe394004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.421+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe39401d070 con 0x7fe3a4106970 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.421+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a4198730 con 0x7fe3a4106970 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.421+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe39400bdf0 con 0x7fe3a4106970 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.421+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe394022d80 con 0x7fe3a4106970 2023-10-06T21:11:53.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.421+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4198bb0 con 0x7fe3a4106970 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.422+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fe394022570 con 0x7fe3a4106970 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.422+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a40624e0 con 0x7fe3a4106970 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.422+0000 7fe3a17fa700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 0x7fe38c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.422+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe39404cb00 con 0x7fe3a4106970 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.423+0000 7fe3a37fe700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 0x7fe38c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:53.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.423+0000 7fe3a37fe700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 0x7fe38c03ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe398006fd0 tx=0x7fe398006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:53.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.427+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe39404ee60 con 0x7fe3a4106970 2023-10-06T21:11:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:53.639+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3a40008d0 con 0x7fe38c0388c0 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:11:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:11:55.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:11:56.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:55 smithi031 ceph-mon[98539]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:11:56.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:55 smithi031 ceph-mon[98539]: Deploying cephadm binary to smithi146 2023-10-06T21:11:56.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:55 smithi031 ceph-mon[98539]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:11:56.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.955+0000 7fe3a17fa700 1 -- 172.21.15.31:0/3277790442 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe3a40008d0 con 0x7fe38c0388c0 2023-10-06T21:11:56.957 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi146' with addr '172.21.15.146' 2023-10-06T21:11:56.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 msgr2=0x7fe38c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:56.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 0x7fe38c03ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe398006fd0 tx=0x7fe398006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 msgr2=0x7fe3a4197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe394006010 tx=0x7fe394004ff0 comp rx=0 tx=0).stop 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 shutdown_connections 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe38c0388c0 0x7fe38c03ad80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 --2- 172.21.15.31:0/3277790442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3a4106970 0x7fe3a4197f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.957+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 >> 172.21.15.31:0/3277790442 conn(0x7fe3a40fc4a0 msgr2=0x7fe3a4108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.958+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 shutdown_connections 2023-10-06T21:11:56.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:56.958+0000 7fe3aa923700 1 -- 172.21.15.31:0/3277790442 wait complete. 2023-10-06T21:11:56.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:56 smithi031 ceph-mon[98539]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:11:56.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:56 smithi031 ceph-mon[98539]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:57.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T21:11:58.157 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: Added host smithi146 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:59.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:58 smithi031 ceph-mon[98539]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:11:59.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:11:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:11:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 -- 172.21.15.31:0/2448977229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 msgr2=0x7f07fc0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 --2- 172.21.15.31:0/2448977229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc0736c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009910 tx=0x7f07ec009c20 comp rx=0 tx=0).stop 2023-10-06T21:11:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 -- 172.21.15.31:0/2448977229 shutdown_connections 2023-10-06T21:11:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 --2- 172.21.15.31:0/2448977229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc0736c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 -- 172.21.15.31:0/2448977229 >> 172.21.15.31:0/2448977229 conn(0x7f07fc0fc4a0 msgr2=0x7f07fc0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:59.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 -- 172.21.15.31:0/2448977229 shutdown_connections 2023-10-06T21:11:59.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.618+0000 7f0802037700 1 -- 172.21.15.31:0/2448977229 wait complete. 2023-10-06T21:11:59.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.619+0000 7f0802037700 1 Processor -- start 2023-10-06T21:11:59.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f0802037700 1 -- start start 2023-10-06T21:11:59.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f0802037700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:59.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f0802037700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc198560 con 0x7f07fc0732e0 2023-10-06T21:11:59.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f07fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:59.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f07fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39512/0 (socket says 172.21.15.31:39512) 2023-10-06T21:11:59.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.620+0000 7f07fb7fe700 1 -- 172.21.15.31:0/3681012111 learned_addr learned my addr 172.21.15.31:0/3681012111 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:11:59.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f07fb7fe700 1 -- 172.21.15.31:0/3681012111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0095f0 con 0x7f07fc0732e0 2023-10-06T21:11:59.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f07fb7fe700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:59.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07ec01d070 con 0x7f07fc0732e0 2023-10-06T21:11:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07ec00bdf0 con 0x7f07fc0732e0 2023-10-06T21:11:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc198760 con 0x7f07fc0732e0 2023-10-06T21:11:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.621+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07ec022d80 con 0x7f07fc0732e0 2023-10-06T21:11:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.622+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc198b80 con 0x7f07fc0732e0 2023-10-06T21:11:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.623+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc0624e0 con 0x7f07fc0732e0 2023-10-06T21:11:59.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.623+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f07ec022570 con 0x7f07fc0732e0 2023-10-06T21:11:59.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.623+0000 7f07f8ff9700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 0x7f07e403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:59.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.623+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07ec04caa0 con 0x7f07fc0732e0 2023-10-06T21:11:59.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.624+0000 7f07faffd700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 0x7f07e403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.624+0000 7f07faffd700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 0x7f07e403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006fd0 tx=0x7f07f0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.628+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f07ec02bba0 con 0x7f07fc0732e0 2023-10-06T21:11:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.844+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07fc10cea0 con 0x7f07e4038910 2023-10-06T21:11:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.845+0000 7f07f8ff9700 1 -- 172.21.15.31:0/3681012111 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f07fc10cea0 con 0x7f07e4038910 2023-10-06T21:11:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:11:59.848 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 msgr2=0x7f07e403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 0x7f07e403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006fd0 tx=0x7f07f0006e40 comp rx=0 tx=0).stop 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 msgr2=0x7f07fc198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec004f40 comp rx=0 tx=0).stop 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 shutdown_connections 2023-10-06T21:11:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f07e4038910 0x7f07e403add0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:59.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 --2- 172.21.15.31:0/3681012111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07fc0732e0 0x7f07fc198020 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:59.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.847+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 >> 172.21.15.31:0/3681012111 conn(0x7f07fc0fc4a0 msgr2=0x7f07fc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:59.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.848+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 shutdown_connections 2023-10-06T21:11:59.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:11:59.848+0000 7f0802037700 1 -- 172.21.15.31:0/3681012111 wait complete. 2023-10-06T21:12:00.334 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-06T21:12:00.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-06T21:12:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:12:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:00 smithi031 ceph-mon[98539]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:00 smithi031 ceph-mon[98539]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:12:01.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 -- 172.21.15.31:0/1509709667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 msgr2=0x7fef6c106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 --2- 172.21.15.31:0/1509709667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c106d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009910 tx=0x7fef5c009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 -- 172.21.15.31:0/1509709667 shutdown_connections 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 --2- 172.21.15.31:0/1509709667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c106d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 -- 172.21.15.31:0/1509709667 >> 172.21.15.31:0/1509709667 conn(0x7fef6c0fc4a0 msgr2=0x7fef6c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 -- 172.21.15.31:0/1509709667 shutdown_connections 2023-10-06T21:12:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.574+0000 7fef724c1700 1 -- 172.21.15.31:0/1509709667 wait complete. 2023-10-06T21:12:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.575+0000 7fef724c1700 1 Processor -- start 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef724c1700 1 -- start start 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef724c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef724c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c1962a0 con 0x7fef6c106970 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39532/0 (socket says 172.21.15.31:39532) 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.576+0000 7fef6bfff700 1 -- 172.21.15.31:0/2239466352 learned_addr learned my addr 172.21.15.31:0/2239466352 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef6bfff700 1 -- 172.21.15.31:0/2239466352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c0095f0 con 0x7fef6c106970 2023-10-06T21:12:02.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef6bfff700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef5c01d070 con 0x7fef6c106970 2023-10-06T21:12:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c196500 con 0x7fef6c106970 2023-10-06T21:12:02.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef5c00bdf0 con 0x7fef6c106970 2023-10-06T21:12:02.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef5c022d80 con 0x7fef6c106970 2023-10-06T21:12:02.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.577+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c196980 con 0x7fef6c106970 2023-10-06T21:12:02.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.578+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0624e0 con 0x7fef6c106970 2023-10-06T21:12:02.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.578+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fef5c022570 con 0x7fef6c106970 2023-10-06T21:12:02.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.579+0000 7fef697fa700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 0x7fef5403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:02.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.579+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef5c04cab0 con 0x7fef6c106970 2023-10-06T21:12:02.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.579+0000 7fef6b7fe700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 0x7fef5403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:02.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.580+0000 7fef6b7fe700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 0x7fef5403ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.584+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef5c02bba0 con 0x7fef6c106970 2023-10-06T21:12:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.800+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fef6c0008d0 con 0x7fef6c106970 2023-10-06T21:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.960+0000 7fef697fa700 1 -- 172.21.15.31:0/2239466352 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fef5c027030 con 0x7fef6c106970 2023-10-06T21:12:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 msgr2=0x7fef5403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 0x7fef5403ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60006e40 comp rx=0 tx=0).stop 2023-10-06T21:12:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 msgr2=0x7fef6c195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c004ff0 comp rx=0 tx=0).stop 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 shutdown_connections 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fef540388c0 0x7fef5403ad80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 --2- 172.21.15.31:0/2239466352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef6c106970 0x7fef6c195d60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 >> 172.21.15.31:0/2239466352 conn(0x7fef6c0fc4a0 msgr2=0x7fef6c108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 shutdown_connections 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:02.963+0000 7fef724c1700 1 -- 172.21.15.31:0/2239466352 wait complete. 2023-10-06T21:12:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2023-10-06T21:12:03.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:02 smithi031 ceph-mon[98539]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:03.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T21:12:03.454 INFO:tasks.cephadm:Adding mon.a on smithi031 2023-10-06T21:12:03.454 INFO:tasks.cephadm:Adding mon.b on smithi142 2023-10-06T21:12:03.454 INFO:tasks.cephadm:Adding mon.c on smithi146 2023-10-06T21:12:03.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c' 2023-10-06T21:12:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:03 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T21:12:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:03 smithi031 ceph-mon[98539]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:04.324 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /etc/ceph/ceph.conf 2023-10-06T21:12:04.672 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /etc/ceph/ceph.conf 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:06.227 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.225+0000 7fe3738db700 1 -- 172.21.15.146:0/1034503097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 msgr2=0x7fe36c106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.225+0000 7fe3738db700 1 --2- 172.21.15.146:0/1034503097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c106d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe368009910 tx=0x7fe368009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.225+0000 7fe3738db700 1 -- 172.21.15.146:0/1034503097 shutdown_connections 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.225+0000 7fe3738db700 1 --2- 172.21.15.146:0/1034503097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c106d50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.225+0000 7fe3738db700 1 -- 172.21.15.146:0/1034503097 >> 172.21.15.146:0/1034503097 conn(0x7fe36c0fc4a0 msgr2=0x7fe36c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.226+0000 7fe3738db700 1 -- 172.21.15.146:0/1034503097 shutdown_connections 2023-10-06T21:12:06.228 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.226+0000 7fe3738db700 1 -- 172.21.15.146:0/1034503097 wait complete. 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.226+0000 7fe3738db700 1 Processor -- start 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe3738db700 1 -- start start 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe3738db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe3738db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1962a0 con 0x7fe36c106970 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe371677700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe371677700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.146:37694/0 (socket says 172.21.15.146:37694) 2023-10-06T21:12:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.227+0000 7fe371677700 1 -- 172.21.15.146:0/2930008419 learned_addr learned my addr 172.21.15.146:0/2930008419 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe371677700 1 -- 172.21.15.146:0/2930008419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680095f0 con 0x7fe36c106970 2023-10-06T21:12:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe371677700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe368006010 tx=0x7fe368004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe36801d070 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c196500 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe36800bdf0 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe368022d80 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.228+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c196980 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.229+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c0624e0 con 0x7fe36c106970 2023-10-06T21:12:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.230+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fe368022570 con 0x7fe36c106970 2023-10-06T21:12:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.230+0000 7fe3627fc700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 0x7fe35803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.230+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe36804cb00 con 0x7fe36c106970 2023-10-06T21:12:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.230+0000 7fe370e76700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 0x7fe35803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:06.233 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.231+0000 7fe370e76700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 0x7fe35803ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:06.237 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.236+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe36804ee60 con 0x7fe36c106970 2023-10-06T21:12:06.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:05 smithi031 ceph-mon[98539]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:06.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:05 smithi031 ceph-mon[98539]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:06.451 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.449+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe36c000990 con 0x7fe3580388c0 2023-10-06T21:12:06.457 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.455+0000 7fe3627fc700 1 -- 172.21.15.146:0/2930008419 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe36c000990 con 0x7fe3580388c0 2023-10-06T21:12:06.458 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mon update... 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 msgr2=0x7fe35803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 0x7fe35803ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c006e40 comp rx=0 tx=0).stop 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 msgr2=0x7fe36c195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe368006010 tx=0x7fe368004ff0 comp rx=0 tx=0).stop 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 shutdown_connections 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3580388c0 0x7fe35803ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 --2- 172.21.15.146:0/2930008419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c106970 0x7fe36c195d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:06.460 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 >> 172.21.15.146:0/2930008419 conn(0x7fe36c0fc4a0 msgr2=0x7fe36c108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 shutdown_connections 2023-10-06T21:12:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:06.458+0000 7fe3738db700 1 -- 172.21.15.146:0/2930008419 wait complete. 2023-10-06T21:12:06.955 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-06T21:12:06.960 DEBUG:teuthology.orchestra.run.smithi146:mon.c> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-06T21:12:06.965 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:12:06.965 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:12:07.035 INFO:journalctl@ceph.mon.c.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:12:07.036 INFO:journalctl@ceph.mon.b.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:07.725 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c;count:3 2023-10-06T21:12:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:12:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:08.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:08.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:08.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:08.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:08.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:08 smithi031 ceph-mon[98539]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:09.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.269+0000 7fb03ebd2700 1 -- 172.21.15.146:0/1510290735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 msgr2=0x7fb038101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.269+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/1510290735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038101f10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009910 tx=0x7fb02c009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:09.272 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.269+0000 7fb03ebd2700 1 -- 172.21.15.146:0/1510290735 shutdown_connections 2023-10-06T21:12:09.272 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.269+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/1510290735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038101f10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.272 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.269+0000 7fb03ebd2700 1 -- 172.21.15.146:0/1510290735 >> 172.21.15.146:0/1510290735 conn(0x7fb0380fb380 msgr2=0x7fb0380fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:09.272 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.270+0000 7fb03ebd2700 1 -- 172.21.15.146:0/1510290735 shutdown_connections 2023-10-06T21:12:09.272 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.270+0000 7fb03ebd2700 1 -- 172.21.15.146:0/1510290735 wait complete. 2023-10-06T21:12:09.273 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.271+0000 7fb03ebd2700 1 Processor -- start 2023-10-06T21:12:09.273 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.271+0000 7fb03ebd2700 1 -- start start 2023-10-06T21:12:09.274 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.272+0000 7fb03ebd2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:09.274 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.272+0000 7fb03ebd2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb038196380 con 0x7fb038101b30 2023-10-06T21:12:09.274 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.272+0000 7fb03c96e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:09.274 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.272+0000 7fb03c96e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.146:37712/0 (socket says 172.21.15.146:37712) 2023-10-06T21:12:09.274 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.273+0000 7fb03c96e700 1 -- 172.21.15.146:0/2396999779 learned_addr learned my addr 172.21.15.146:0/2396999779 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:09.275 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.273+0000 7fb03c96e700 1 -- 172.21.15.146:0/2396999779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0095f0 con 0x7fb038101b30 2023-10-06T21:12:09.275 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.273+0000 7fb03c96e700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006010 tx=0x7fb02c004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:09.276 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.274+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c01d070 con 0x7fb038101b30 2023-10-06T21:12:09.276 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.274+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb038196580 con 0x7fb038101b30 2023-10-06T21:12:09.276 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.274+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02c00bdf0 con 0x7fb038101b30 2023-10-06T21:12:09.276 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.274+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c022d80 con 0x7fb038101b30 2023-10-06T21:12:09.276 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.274+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0381969a0 con 0x7fb038101b30 2023-10-06T21:12:09.277 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.275+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0380624e0 con 0x7fb038101b30 2023-10-06T21:12:09.277 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.275+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fb02c00b960 con 0x7fb038101b30 2023-10-06T21:12:09.277 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.275+0000 7fb035ffb700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 0x7fb02803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:09.277 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.276+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb02c04ca90 con 0x7fb038101b30 2023-10-06T21:12:09.278 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.276+0000 7fb037fff700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 0x7fb02803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:09.278 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.277+0000 7fb037fff700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 0x7fb02803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb024006fd0 tx=0x7fb024006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:09.283 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.281+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb02c02a540 con 0x7fb038101b30 2023-10-06T21:12:09.604 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.602+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb03810ac20 con 0x7fb038101b30 2023-10-06T21:12:09.604 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.603+0000 7fb035ffb700 1 -- 172.21.15.146:0/2396999779 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fb02c027030 con 0x7fb038101b30 2023-10-06T21:12:09.605 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:12:09.605 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":1,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:10:24.983904Z","created":"2023-10-06T21:10:24.983904Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-06T21:12:09.607 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 msgr2=0x7fb02803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.607 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 0x7fb02803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb024006fd0 tx=0x7fb024006e40 comp rx=0 tx=0).stop 2023-10-06T21:12:09.607 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 msgr2=0x7fb038195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.607 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006010 tx=0x7fb02c004f40 comp rx=0 tx=0).stop 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 shutdown_connections 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb028038860 0x7fb02803ad20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 --2- 172.21.15.146:0/2396999779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038101b30 0x7fb038195e40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 >> 172.21.15.146:0/2396999779 conn(0x7fb0380fb380 msgr2=0x7fb038105220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 shutdown_connections 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:09.605+0000 7fb03ebd2700 1 -- 172.21.15.146:0/2396999779 wait complete. 2023-10-06T21:12:09.608 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 1 2023-10-06T21:12:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:09 smithi031 ceph-mon[98539]: Deploying daemon mon.c on smithi146 2023-10-06T21:12:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:09 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/2396999779' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:11.216 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:12:11.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:12:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:10 smithi031 ceph-mon[98539]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:11 smithi031 ceph-mon[98539]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:12.378 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:12 smithi146 systemd[1]: Starting Ceph mon.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:12:12.476 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:12:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:12 smithi146 podman[102725]: 2023-10-06T21:12:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 bash[102725]: a059c332cf6da98c6ff6c8c2f06c2ff277529e9cf5c2181369c47c90fd426bb9 2023-10-06T21:12:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T21:12:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pidfile_write: ignore empty --pid-file 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: load: jerasure load: lrc 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: RocksDB version: 7.9.2 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Git sha 0 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: DB SUMMARY 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: DB Session ID: 0JRUYC58YECPGX1ITVLL 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: CURRENT file: CURRENT 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-06T21:12:13.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.error_if_exists: 0 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.create_if_missing: 0 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.paranoid_checks: 1 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.env: 0x557ce0837da0 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.info_log: 0x557ce220b5c0 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T21:12:13.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.statistics: (nil) 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.use_fsync: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_log_file_size: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_fallocate: 1 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.use_direct_reads: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T21:12:13.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.db_log_dir: 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.wal_dir: 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.write_buffer_manager: 0x557ce21c3360 2023-10-06T21:12:13.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.unordered_write: 0 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T21:12:13.804 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.row_cache: None 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.wal_filter: None 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.two_write_queues: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.wal_compression: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.atomic_flush: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.log_readahead_size: 0 2023-10-06T21:12:13.805 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_background_jobs: 2 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_background_compactions: -1 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_subcompactions: 1 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T21:12:13.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_open_files: -1 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T21:12:13.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_background_flushes: -1 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Compression algorithms supported: 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kZSTD supported: 0 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kXpressCompression supported: 0 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kZlibCompression supported: 1 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kSnappyCompression supported: 1 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kLZ4Compression supported: 1 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: kBZip2Compression supported: 0 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-10-06T21:12:13.808 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.merge_operator: 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_filter: None 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557ce220b6c0) 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout: cache_index_and_filter_blocks: 1 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T21:12:13.809 INFO:journalctl@ceph.mon.c.smithi146.stdout: pin_top_level_index_and_filter: 1 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_type: 0 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: data_block_index_type: 0 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_shortening: 1 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: checksum: 4 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: no_block_cache: 0 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache: 0x557ce22611f0 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_name: BinnedLRUCache 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_options: 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: capacity : 536870912 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: num_shard_bits : 4 2023-10-06T21:12:13.810 INFO:journalctl@ceph.mon.c.smithi146.stdout: strict_capacity_limit : 0 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: high_pri_pool_ratio: 0.000 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_compressed: (nil) 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: persistent_cache: (nil) 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_size: 4096 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_size_deviation: 10 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_restart_interval: 16 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_block_restart_interval: 1 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: metadata_block_size: 4096 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: partition_filters: 0 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: use_delta_encoding: 1 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: filter_policy: bloomfilter 2023-10-06T21:12:13.811 INFO:journalctl@ceph.mon.c.smithi146.stdout: whole_key_filtering: 1 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: verify_compression: 0 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: read_amp_bytes_per_bit: 0 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: format_version: 5 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: enable_index_compression: 1 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_align: 0 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: max_auto_readahead_size: 262144 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: prepopulate_block_cache: 0 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: initial_auto_readahead_size: 8192 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression: NoCompression 2023-10-06T21:12:13.812 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.num_levels: 7 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T21:12:13.813 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T21:12:13.814 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:12:13.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T21:12:13.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T21:12:13.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.table_properties_collectors: 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.inplace_update_support: 0 2023-10-06T21:12:13.818 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.bloom_locality: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.max_successive_merges: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.ttl: 2592000 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T21:12:13.819 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enable_blob_files: false 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.min_blob_size: 0 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-06T21:12:13.820 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5392cf70-02a1-44dd-865b-7aa14e199bcb 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626733367945, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626733369021, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696626733, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5392cf70-02a1-44dd-865b-7aa14e199bcb", "db_session_id": "0JRUYC58YECPGX1ITVLL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626733369230, "job": 1, "event": "recovery_finished"} 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557ce22fe000 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: DB pointer 0x557ce22ea000 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** DB Stats ** 2023-10-06T21:12:13.821 INFO:journalctl@ceph.mon.c.smithi146.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** Compaction Stats [default] ** 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:13.822 INFO:journalctl@ceph.mon.c.smithi146.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** Compaction Stats [default] ** 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T21:12:13.823 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: Block cache BinnedLRUCache@0x557ce22611f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: using public_addr v2:172.21.15.146:0/0 -> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] 2023-10-06T21:12:13.824 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: starting mon.c rank -1 at public addrs [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] at bind addrs [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(???) e0 preinit fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).mds e1 new map 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: e1 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: legacy client fscid: -1 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout: No filesystems configured 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-06T21:12:13.825 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.826 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Deploying cephadm binary to smithi146 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Added host smithi146 2023-10-06T21:12:13.827 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T21:12:13.828 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:13.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:13.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c;count:3 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.831 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: Deploying daemon mon.c on smithi146 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/2396999779' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expand map: {default=false} 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta from 'false' to 'false' 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expanded map: {default=false} 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expand map: {default=info} 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta from 'info' to 'info' 2023-10-06T21:12:13.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expanded map: {default=info} 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expand map: {default=daemon} 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expand map: {default=debug} 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 ceph-mon[102809]: expand_channel_meta expanded map: {default=debug} 2023-10-06T21:12:13.833 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:13 smithi146 systemd[1]: Started Ceph mon.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:12:18.553 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.549+0000 7fd7485ed700 1 -- 172.21.15.146:0/3516776166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0035c0 msgr2=0x7fd72c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:18.553 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.549+0000 7fd7485ed700 1 --2- 172.21.15.146:0/3516776166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0035c0 0x7fd72c005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7401009a0 tx=0x7fd73c00fe60 comp rx=0 tx=0).stop 2023-10-06T21:12:18.553 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.550+0000 7fd7485ed700 1 -- 172.21.15.146:0/3516776166 shutdown_connections 2023-10-06T21:12:18.553 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.550+0000 7fd7485ed700 1 --2- 172.21.15.146:0/3516776166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd72c0035c0 0x7fd72c005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:18.554 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.550+0000 7fd7485ed700 1 -- 172.21.15.146:0/3516776166 >> 172.21.15.146:0/3516776166 conn(0x7fd7400fc540 msgr2=0x7fd7400fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:18.554 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.550+0000 7fd7485ed700 1 -- 172.21.15.146:0/3516776166 shutdown_connections 2023-10-06T21:12:18.554 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.550+0000 7fd7485ed700 1 -- 172.21.15.146:0/3516776166 wait complete. 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.551+0000 7fd7485ed700 1 Processor -- start 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.552+0000 7fd7485ed700 1 -- start start 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.552+0000 7fd7485ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.552+0000 7fd7485ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 0x7fd740073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.552+0000 7fd7485ed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740075130 con 0x7fd7400736f0 2023-10-06T21:12:18.555 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.552+0000 7fd7485ed700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401a2050 con 0x7fd72c0035c0 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50314/0 (socket says 172.21.15.146:50314) 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 -- 172.21.15.146:0/1832970694 learned_addr learned my addr 172.21.15.146:0/1832970694 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd745b88700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 0x7fd740073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 -- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 msgr2=0x7fd7400731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-06T21:12:18.557 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 -- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 msgr2=0x7fd7400731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T21:12:18.558 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T21:12:18.558 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd746389700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:12:18.558 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd745b88700 1 -- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 msgr2=0x7fd7400731b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:12:18.558 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd745b88700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:18.558 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.553+0000 7fd745b88700 1 -- 172.21.15.146:0/1832970694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c00f8b0 con 0x7fd7400736f0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd745b88700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 0x7fd740073b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b4d0 tx=0x7fd73000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd730011630 con 0x7fd7400736f0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd730011c20 con 0x7fd7400736f0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7401a2250 con 0x7fd7400736f0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd730010c60 con 0x7fd7400736f0 2023-10-06T21:12:18.559 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.554+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7401a2770 con 0x7fd7400736f0 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.555+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fd73001a400 con 0x7fd7400736f0 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.555+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7400624e0 con 0x7fd7400736f0 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.556+0000 7fd7377fe700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 0x7fd72403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.556+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd73004af00 con 0x7fd7400736f0 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.556+0000 7fd746389700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 0x7fd72403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:18.560 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.557+0000 7fd746389700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 0x7fd72403abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005d10 tx=0x7fd73c005c80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:18.565 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.561+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd7300178b0 con 0x7fd7400736f0 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: mon.a calling monitor election 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: mon.c calling monitor election 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: fsmap 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: mgrmap e13: x(active, since 52s) 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: overall HEALTH_OK 2023-10-06T21:12:18.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:18.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: mon.a calling monitor election 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: mon.c calling monitor election 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:12:18.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: fsmap 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: mgrmap e13: x(active, since 52s) 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: overall HEALTH_OK 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:18.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:18.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.843+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd740108940 con 0x7fd7400736f0 2023-10-06T21:12:18.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.844+0000 7fd7377fe700 1 -- 172.21.15.146:0/1832970694 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fd7300172c0 con 0x7fd7400736f0 2023-10-06T21:12:18.847 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:12:18.847 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:12:13.412677Z","created":"2023-10-06T21:10:24.983904Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T21:12:18.850 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.846+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 msgr2=0x7fd72403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:18.850 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.846+0000 7fd7485ed700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 0x7fd72403abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005d10 tx=0x7fd73c005c80 comp rx=0 tx=0).stop 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.846+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 msgr2=0x7fd740073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.846+0000 7fd7485ed700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 0x7fd740073b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b4d0 tx=0x7fd73000b890 comp rx=0 tx=0).stop 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 shutdown_connections 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd724038700 0x7fd72403abc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd72c0035c0 0x7fd7400731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 --2- 172.21.15.146:0/1832970694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7400736f0 0x7fd740073b70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 >> 172.21.15.146:0/1832970694 conn(0x7fd7400fc540 msgr2=0x7fd740102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 shutdown_connections 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:18.847+0000 7fd7485ed700 1 -- 172.21.15.146:0/1832970694 wait complete. 2023-10-06T21:12:18.851 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 2 2023-10-06T21:12:20.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:19 smithi031 ceph-mon[98539]: Deploying daemon mon.b on smithi142 2023-10-06T21:12:20.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:19 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/1832970694' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:20.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:19 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:19 smithi146 ceph-mon[102809]: Deploying daemon mon.b on smithi142 2023-10-06T21:12:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:19 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/1832970694' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:19 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:20.273 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:12:20.273 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:12:20.674 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:12:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:12:20.412+0000 7fa2a1a3a700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T21:12:21.007 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:12:21.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:20 smithi031 ceph-mon[98539]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:21.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:20 smithi146 ceph-mon[102809]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:22 smithi142 systemd[1]: Starting Ceph mon.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 -- 172.21.15.146:0/474908442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a0073250 msgr2=0x7f29a0073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 --2- 172.21.15.146:0/474908442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a0073250 0x7f29a0073630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2990009910 tx=0x7f2990009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 -- 172.21.15.146:0/474908442 shutdown_connections 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 --2- 172.21.15.146:0/474908442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a0073250 0x7f29a0073630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 -- 172.21.15.146:0/474908442 >> 172.21.15.146:0/474908442 conn(0x7f29a00fc540 msgr2=0x7f29a00fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 -- 172.21.15.146:0/474908442 shutdown_connections 2023-10-06T21:12:22.497 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.493+0000 7f29a52aa700 1 -- 172.21.15.146:0/474908442 wait complete. 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.494+0000 7f29a52aa700 1 Processor -- start 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.495+0000 7f29a52aa700 1 -- start start 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.495+0000 7f29a52aa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.495+0000 7f29a52aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.495+0000 7f29a52aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a019d060 con 0x7f29a019cad0 2023-10-06T21:12:22.498 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.495+0000 7f29a52aa700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a019d1d0 con 0x7f29a0073250 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.146:44892/0 (socket says 172.21.15.146:44892) 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 -- 172.21.15.146:0/3762556161 learned_addr learned my addr 172.21.15.146:0/3762556161 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299effd700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299effd700 1 -- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 msgr2=0x7f29a019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299effd700 1 -- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 msgr2=0x7f29a019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299effd700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299effd700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 -- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 msgr2=0x7f29a019c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:12:22.500 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 -- 172.21.15.146:0/3762556161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900095f0 con 0x7f29a019cad0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.496+0000 7f299e7fc700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f299400b700 tx=0x7f299400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.497+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f2994011630 con 0x7f29a019cad0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.497+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2994011c20 con 0x7f29a019cad0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.497+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f2994010c60 con 0x7f29a019cad0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.497+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a01a14e0 con 0x7f29a019cad0 2023-10-06T21:12:22.501 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.497+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a01a1a30 con 0x7f29a019cad0 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.498+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a01a1d60 con 0x7f29a019cad0 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.498+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f2994011790 con 0x7f29a019cad0 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.498+0000 7f2987fff700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 0x7f298803ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.498+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f299404b100 con 0x7f29a019cad0 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.499+0000 7f299effd700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 0x7f298803ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.503 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.500+0000 7f299effd700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 0x7f298803ab60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2990006010 tx=0x7f299000b580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.507 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.504+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2994010380 con 0x7f29a019cad0 2023-10-06T21:12:22.785 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.782+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f29a0111210 con 0x7f29a019cad0 2023-10-06T21:12:22.787 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.785+0000 7f2987fff700 1 -- 172.21.15.146:0/3762556161 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f2994010380 con 0x7f29a019cad0 2023-10-06T21:12:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:12:22.788 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:12:13.412677Z","created":"2023-10-06T21:10:24.983904Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T21:12:22.790 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 msgr2=0x7f298803ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.790 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 0x7f298803ab60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2990006010 tx=0x7f299000b580 comp rx=0 tx=0).stop 2023-10-06T21:12:22.790 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 msgr2=0x7f29a01a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f299400b700 tx=0x7f299400bac0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 shutdown_connections 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f29880386a0 0x7f298803ab60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29a0073250 0x7f29a019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 --2- 172.21.15.146:0/3762556161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a019cad0 0x7f29a01a0f40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 >> 172.21.15.146:0/3762556161 conn(0x7f29a00fc540 msgr2=0x7f29a00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 shutdown_connections 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:22.787+0000 7f29a52aa700 1 -- 172.21.15.146:0/3762556161 wait complete. 2023-10-06T21:12:22.791 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 2 2023-10-06T21:12:23.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:22 smithi142 podman[102911]: 2023-10-06T21:12:23.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:22 smithi146 ceph-mon[102809]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:22 smithi031 ceph-mon[98539]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 bash[102911]: 10acf701570c9e35ee1bd647d17139d744e54daa6c99470f36496b2456634b5e 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pidfile_write: ignore empty --pid-file 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: load: jerasure load: lrc 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: RocksDB version: 7.9.2 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Git sha 0 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: DB SUMMARY 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: DB Session ID: RSFPK36BITZGU0629JO4 2023-10-06T21:12:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: CURRENT file: CURRENT 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.error_if_exists: 0 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.create_if_missing: 0 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.paranoid_checks: 1 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.env: 0x55bf1f5b0da0 2023-10-06T21:12:23.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.info_log: 0x55bf202615c0 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.statistics: (nil) 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.use_fsync: 0 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_log_file_size: 0 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_fallocate: 1 2023-10-06T21:12:23.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.use_direct_reads: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.db_log_dir: 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.wal_dir: 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T21:12:23.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.write_buffer_manager: 0x55bf20219360 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T21:12:23.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.unordered_write: 0 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.row_cache: None 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.wal_filter: None 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.two_write_queues: 0 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T21:12:23.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.wal_compression: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.atomic_flush: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.log_readahead_size: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T21:12:23.409 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_background_jobs: 2 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_background_compactions: -1 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_subcompactions: 1 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T21:12:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_open_files: -1 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_background_flushes: -1 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Compression algorithms supported: 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kZSTD supported: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kXpressCompression supported: 0 2023-10-06T21:12:23.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kZlibCompression supported: 1 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kSnappyCompression supported: 1 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kLZ4Compression supported: 1 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: kBZip2Compression supported: 0 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T21:12:23.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.merge_operator: 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_filter: None 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bf202616c0) 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T21:12:23.413 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x55bf202b71f0 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2023-10-06T21:12:23.414 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2023-10-06T21:12:23.415 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T21:12:23.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression: NoCompression 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.num_levels: 7 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T21:12:23.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T21:12:23.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T21:12:23.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T21:12:23.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T21:12:23.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.table_properties_collectors: 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.inplace_update_support: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.bloom_locality: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.max_successive_merges: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T21:12:23.423 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.ttl: 2592000 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enable_blob_files: false 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.min_blob_size: 0 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T21:12:23.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f9ee8e08-97a4-44e1-924c-99a5b7216bc5 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626743052459, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626743053199, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696626743, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f9ee8e08-97a4-44e1-924c-99a5b7216bc5", "db_session_id": "RSFPK36BITZGU0629JO4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626743053445, "job": 1, "event": "recovery_finished"} 2023-10-06T21:12:23.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bf20354000 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: DB pointer 0x55bf20340000 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:12:23.426 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2023-10-06T21:12:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:12:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x55bf202b71f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(???) e0 preinit fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).mds e1 new map 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-06T21:12:23.429 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-06T21:12:23.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Deploying cephadm binary to smithi146 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Added host smithi146 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.432 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2239466352' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T21:12:23.433 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:23.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 systemd[1]: Started Ceph mon.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c;count:3 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:23.436 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Deploying daemon mon.c on smithi146 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/2396999779' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.a calling monitor election 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.c calling monitor election 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: fsmap 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mgrmap e13: x(active, since 52s) 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: overall HEALTH_OK 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: Deploying daemon mon.b on smithi142 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/1832970694' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expand map: {default=false} 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta from 'false' to 'false' 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expanded map: {default=false} 2023-10-06T21:12:23.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expand map: {default=info} 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta from 'info' to 'info' 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expanded map: {default=info} 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expand map: {default=daemon} 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expand map: {default=debug} 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T21:12:23.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:23 smithi142 ceph-mon[102951]: expand_channel_meta expanded map: {default=debug} 2023-10-06T21:12:24.282 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:12:24.283 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:12:25.014 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: mon.a calling monitor election 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: mon.c calling monitor election 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: mon.b calling monitor election 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:28.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: fsmap 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: mgrmap e13: x(active, since 62s) 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: overall HEALTH_OK 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: mon.a calling monitor election 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: mon.c calling monitor election 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: mon.b calling monitor election 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: fsmap 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:28.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: mgrmap e13: x(active, since 62s) 2023-10-06T21:12:28.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: overall HEALTH_OK 2023-10-06T21:12:28.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:12:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: mon.a calling monitor election 2023-10-06T21:12:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: mon.c calling monitor election 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: mon.b calling monitor election 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: fsmap 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: mgrmap e13: x(active, since 62s) 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: overall HEALTH_OK 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:28.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:29.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 -- 172.21.15.146:0/1302810198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c1068a0 msgr2=0x7f2084005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:29.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 --2- 172.21.15.146:0/1302810198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c1068a0 0x7f2084005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fa0 tx=0x7f208c00ffc0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 -- 172.21.15.146:0/1302810198 shutdown_connections 2023-10-06T21:12:29.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 --2- 172.21.15.146:0/1302810198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c1068a0 0x7f2084005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 -- 172.21.15.146:0/1302810198 >> 172.21.15.146:0/1302810198 conn(0x7f209c0fc390 msgr2=0x7f209c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:29.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 -- 172.21.15.146:0/1302810198 shutdown_connections 2023-10-06T21:12:29.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.429+0000 7f20a0907700 1 -- 172.21.15.146:0/1302810198 wait complete. 2023-10-06T21:12:29.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 Processor -- start 2023-10-06T21:12:29.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 -- start start 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209c1068a0 0x7f209c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c198570 0x7f209c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c198c60 con 0x7f209c198570 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1a4c90 con 0x7f209c1068a0 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.431+0000 7f20a0907700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1a4dd0 con 0x7f209c19cf20 2023-10-06T21:12:29.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c198570 0x7f209c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45720/0 (socket says 172.21.15.146:45720) 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 -- 172.21.15.146:0/1804510165 learned_addr learned my addr 172.21.15.146:0/1804510165 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209b7fe700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209c1068a0 0x7f209c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 -- 172.21.15.146:0/1804510165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209c1068a0 msgr2=0x7f209c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209c1068a0 0x7f209c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 -- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c198570 msgr2=0x7f209c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c198570 0x7f209c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 -- 172.21.15.146:0/1804510165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c00f9c0 con 0x7f209c19cf20 2023-10-06T21:12:29.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f209bfff700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f209400ba50 tx=0x7f209400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:29.437 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.432+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400c9b0 con 0x7f209c19cf20 2023-10-06T21:12:29.437 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.433+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1a50b0 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.433+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2094014440 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.433+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20940075b0 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.433+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c1a5560 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.434+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f2094007710 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.434+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c0624e0 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.434+0000 7f2098ff9700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 0x7f208803b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.434+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f209404aca0 con 0x7f209c19cf20 2023-10-06T21:12:29.438 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.435+0000 7f209b7fe700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 0x7f208803b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:29.439 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.435+0000 7f209b7fe700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 0x7f208803b200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f208c0059d0 tx=0x7f208c005940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:29.443 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.440+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f209401b8b0 con 0x7f209c19cf20 2023-10-06T21:12:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:29 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:29 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:29.737 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.734+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f209c108560 con 0x7f209c19cf20 2023-10-06T21:12:29.737 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.734+0000 7f2098ff9700 1 -- 172.21.15.146:0/1804510165 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f209401b2c0 con 0x7f209c19cf20 2023-10-06T21:12:29.738 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-10-06T21:12:29.738 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":3,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:12:23.098189Z","created":"2023-10-06T21:10:24.983904Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 msgr2=0x7f208803b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 0x7f208803b200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f208c0059d0 tx=0x7f208c005940 comp rx=0 tx=0).stop 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 msgr2=0x7f209c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f209400ba50 tx=0x7f209400bd60 comp rx=0 tx=0).stop 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 shutdown_connections 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2088038d40 0x7f208803b200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f209c19cf20 0x7f209c19d3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.740 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209c1068a0 0x7f209c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 --2- 172.21.15.146:0/1804510165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f209c198570 0x7f209c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 >> 172.21.15.146:0/1804510165 conn(0x7f209c0fc390 msgr2=0x7f209c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 shutdown_connections 2023-10-06T21:12:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:29.737+0000 7f20a0907700 1 -- 172.21.15.146:0/1804510165 wait complete. 2023-10-06T21:12:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 3 2023-10-06T21:12:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:29.761 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:29 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:29.761 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:29 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:12:30.399 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-06T21:12:30.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-06T21:12:30.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:30.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/1804510165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:30.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/1804510165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/1804510165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:31.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:31.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:31.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:32.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:32.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/1861560335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c58106a70 msgr2=0x7f0c58106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/1861560335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c58106a70 0x7f0c58106e50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009910 tx=0x7f0c48009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/1861560335 shutdown_connections 2023-10-06T21:12:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/1861560335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c58106a70 0x7f0c58106e50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/1861560335 >> 172.21.15.31:0/1861560335 conn(0x7f0c580fc540 msgr2=0x7f0c580fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/1861560335 shutdown_connections 2023-10-06T21:12:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.635+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/1861560335 wait complete. 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.636+0000 7f0c5e5f9700 1 Processor -- start 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.636+0000 7f0c5e5f9700 1 -- start start 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c58106a70 0x7f0c581960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c58196610 0x7f0c5819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c58196d00 con 0x7f0c5819afc0 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c581a2d30 con 0x7f0c58196610 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5e5f9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c581a2e70 con 0x7f0c58106a70 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5cb96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5cb96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40134/0 (socket says 172.21.15.31:40134) 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c5cb96700 1 -- 172.21.15.31:0/433702257 learned_addr learned my addr 172.21.15.31:0/433702257 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c57fff700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c58106a70 0x7f0c581960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.637+0000 7f0c577fe700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c58196610 0x7f0c5819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 -- 172.21.15.31:0/433702257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c58106a70 msgr2=0x7f0c581960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c58106a70 0x7f0c581960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 -- 172.21.15.31:0/433702257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c58196610 msgr2=0x7f0c5819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c58196610 0x7f0c5819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 -- 172.21.15.31:0/433702257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480095f0 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5cb96700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000ba50 tx=0x7f0c5000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000c9b0 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c50014440 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500075b0 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c581a3150 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.638+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c581a3600 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.639+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f0c500145b0 con 0x7f0c5819afc0 2023-10-06T21:12:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.639+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c580624e0 con 0x7f0c5819afc0 2023-10-06T21:12:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.640+0000 7f0c557fa700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 0x7f0c4003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.640+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0c5004ac80 con 0x7f0c5819afc0 2023-10-06T21:12:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.640+0000 7f0c57fff700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 0x7f0c4003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.641+0000 7f0c57fff700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 0x7f0c4003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009fd0 tx=0x7f0c480058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.646+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0c5000cb10 con 0x7f0c5819afc0 2023-10-06T21:12:32.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.863+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0c5810ad20 con 0x7f0c5819afc0 2023-10-06T21:12:32.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.864+0000 7f0c557fa700 1 -- 172.21.15.31:0/433702257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f0c500126c0 con 0x7f0c5819afc0 2023-10-06T21:12:32.867 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:32.867 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-10-06T21:12:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:12:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] 2023-10-06T21:12:32.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 msgr2=0x7f0c4003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:32.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 0x7f0c4003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009fd0 tx=0x7f0c480058e0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 msgr2=0x7f0c5819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000ba50 tx=0x7f0c5000bd60 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 shutdown_connections 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c40038d90 0x7f0c4003b250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5819afc0 0x7f0c5819b470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c58106a70 0x7f0c581960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 --2- 172.21.15.31:0/433702257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c58196610 0x7f0c5819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.866+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 >> 172.21.15.31:0/433702257 conn(0x7f0c580fc540 msgr2=0x7f0c58102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.867+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 shutdown_connections 2023-10-06T21:12:32.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:32.867+0000 7f0c5e5f9700 1 -- 172.21.15.31:0/433702257 wait complete. 2023-10-06T21:12:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:32 smithi031 ceph-mon[98539]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:32 smithi031 ceph-mon[98539]: Reconfiguring mon.a (monmap changed)... 2023-10-06T21:12:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:32 smithi031 ceph-mon[98539]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:32 smithi146 ceph-mon[102809]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:32 smithi146 ceph-mon[102809]: Reconfiguring mon.a (monmap changed)... 2023-10-06T21:12:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:32 smithi146 ceph-mon[102809]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:32 smithi142 ceph-mon[102951]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:32 smithi142 ceph-mon[102951]: Reconfiguring mon.a (monmap changed)... 2023-10-06T21:12:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:32 smithi142 ceph-mon[102951]: Reconfiguring daemon mon.a on smithi031 2023-10-06T21:12:33.561 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-06T21:12:33.561 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:12:33.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:12:33.641 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:12:33.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:33.751 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:12:33.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:12:33.831 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:12:33.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:33.939 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:12:33.940 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:12:34.018 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:12:34.018 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/433702257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.119 INFO:tasks.cephadm:Adding mgr.x on smithi031 2023-10-06T21:12:34.120 INFO:tasks.cephadm:Adding mgr.y on smithi142 2023-10-06T21:12:34.120 INFO:tasks.cephadm:Adding mgr.z on smithi146 2023-10-06T21:12:34.120 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi031=x;smithi142=y;smithi146=z' 2023-10-06T21:12:34.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/433702257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/433702257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:34.878 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:12:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:34 smithi146 ceph-mon[102809]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:12:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:34 smithi146 ceph-mon[102809]: Reconfiguring daemon mon.b on smithi142 2023-10-06T21:12:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:34 smithi146 ceph-mon[102809]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:34 smithi142 ceph-mon[102951]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:12:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:34 smithi142 ceph-mon[102951]: Reconfiguring daemon mon.b on smithi142 2023-10-06T21:12:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:34 smithi142 ceph-mon[102951]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:34 smithi031 ceph-mon[98539]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:12:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:34 smithi031 ceph-mon[98539]: Reconfiguring daemon mon.b on smithi142 2023-10-06T21:12:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:34 smithi031 ceph-mon[98539]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: Reconfiguring daemon mon.c on smithi146 2023-10-06T21:12:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:36 smithi146 ceph-mon[102809]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:12:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: Reconfiguring daemon mon.c on smithi146 2023-10-06T21:12:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:36 smithi142 ceph-mon[102951]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3525094822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c100af0 msgr2=0x7f8b7c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3525094822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c100af0 0x7f8b7c100f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009910 tx=0x7f8b78009c20 comp rx=0 tx=0).stop 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3525094822 shutdown_connections 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3525094822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c100af0 0x7f8b7c100f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3525094822 >> 172.21.15.146:0/3525094822 conn(0x7f8b7c0fc0b0 msgr2=0x7f8b7c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:36.462 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3525094822 shutdown_connections 2023-10-06T21:12:36.463 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.458+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3525094822 wait complete. 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.459+0000 7f8b83d9b700 1 Processor -- start 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.459+0000 7f8b83d9b700 1 -- start start 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 0x7f8b7c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 0x7f8b7c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:36.467 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1a02f0 con 0x7f8b7c073c60 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1a0430 con 0x7f8b7c0732a0 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b83d9b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1a0570 con 0x7f8b7c100af0 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b81b37700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b81b37700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:32816/0 (socket says 172.21.15.146:32816) 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b81b37700 1 -- 172.21.15.146:0/3603669628 learned_addr learned my addr 172.21.15.146:0/3603669628 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:12:36.468 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b82338700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 0x7f8b7c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b81336700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 0x7f8b7c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.460+0000 7f8b81b37700 1 -- 172.21.15.146:0/3603669628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 msgr2=0x7f8b7c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81b37700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 0x7f8b7c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81b37700 1 -- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 msgr2=0x7f8b7c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81b37700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 0x7f8b7c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.469 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81b37700 1 -- 172.21.15.146:0/3603669628 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780095f0 con 0x7f8b7c100af0 2023-10-06T21:12:36.470 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b82338700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 0x7f8b7c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:36.470 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81336700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 0x7f8b7c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:12:36.470 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b81b37700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b78006010 tx=0x7f8b7800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:36.470 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7801d070 con 0x7f8b7c100af0 2023-10-06T21:12:36.470 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b78004db0 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c1a07f0 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78022e10 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.461+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c1a0d60 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.462+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f8b78022570 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.462+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c18f4b0 con 0x7f8b7c100af0 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.463+0000 7f8b72ffd700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 0x7f8b6803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:36.471 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.463+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b7804c960 con 0x7f8b7c100af0 2023-10-06T21:12:36.472 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.463+0000 7f8b81336700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 0x7f8b6803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:36.472 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.464+0000 7f8b81336700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 0x7f8b6803b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0079b0 tx=0x7f8b6c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:36.472 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.469+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8b7800e5c0 con 0x7f8b7c100af0 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: Reconfiguring daemon mon.c on smithi146 2023-10-06T21:12:36.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:36 smithi031 ceph-mon[98539]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:36.690 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.687+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi142=y;smithi146=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b7c02d170 con 0x7f8b68038d90 2023-10-06T21:12:36.697 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.694+0000 7f8b72ffd700 1 -- 172.21.15.146:0/3603669628 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8b7c02d170 con 0x7f8b68038d90 2023-10-06T21:12:36.697 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mgr update... 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 msgr2=0x7f8b6803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 0x7f8b6803b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0079b0 tx=0x7f8b6c008040 comp rx=0 tx=0).stop 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 msgr2=0x7f8b7c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b78006010 tx=0x7f8b7800bda0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 shutdown_connections 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8b68038d90 0x7f8b6803b250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b7c073c60 0x7f8b7c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.701 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b7c100af0 0x7f8b7c074c50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 --2- 172.21.15.146:0/3603669628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c0732a0 0x7f8b7c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 >> 172.21.15.146:0/3603669628 conn(0x7f8b7c0fc0b0 msgr2=0x7f8b7c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 shutdown_connections 2023-10-06T21:12:36.702 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:12:36.697+0000 7f8b83d9b700 1 -- 172.21.15.146:0/3603669628 wait complete. 2023-10-06T21:12:37.414 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-06T21:12:37.419 DEBUG:teuthology.orchestra.run.smithi146:mgr.z> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-06T21:12:37.425 INFO:tasks.cephadm:Deploying OSDs... 2023-10-06T21:12:37.426 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:12:37.426 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:12:37.475 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:12:37.475 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-10-06T21:12:37.493 INFO:journalctl@ceph.mgr.z.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:12:37.494 INFO:journalctl@ceph.mgr.y.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 21:11:35.690373578 +0000 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 21:05:12.475864360 +0000 2023-10-06T21:12:37.524 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 21:05:12.475864360 +0000 2023-10-06T21:12:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 21:05:12.447864787 +0000 2023-10-06T21:12:37.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:12:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T21:12:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T21:12:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000280599 s, 1.8 MB/s 2023-10-06T21:12:37.597 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:12:37.642 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-10-06T21:12:37.727 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 21:11:35.772372310 +0000 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 21:05:13.302851743 +0000 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 21:05:13.302851743 +0000 2023-10-06T21:12:37.728 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 21:05:13.275852155 +0000 2023-10-06T21:12:37.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:12:37.836 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T21:12:37.836 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T21:12:37.836 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000299597 s, 1.7 MB/s 2023-10-06T21:12:37.838 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:12:37.883 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-10-06T21:12:37.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi142=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:37.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: Saving service mgr spec with placement smithi031=x;smithi142=y;smithi146=z;count:3 2023-10-06T21:12:37.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:37.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:37.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:37.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:12:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:37.968 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T21:12:37.968 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:37.969 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:37.969 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 21:11:35.855371028 +0000 2023-10-06T21:12:37.969 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 21:05:14.138838989 +0000 2023-10-06T21:12:37.969 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 21:05:14.138838989 +0000 2023-10-06T21:12:37.969 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 21:05:14.111839401 +0000 2023-10-06T21:12:37.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi142=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: Saving service mgr spec with placement smithi031=x;smithi142=y;smithi146=z;count:3 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi142=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: Saving service mgr spec with placement smithi031=x;smithi142=y;smithi146=z;count:3 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:38.076 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T21:12:38.076 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T21:12:38.076 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000301203 s, 1.7 MB/s 2023-10-06T21:12:38.078 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:12:38.125 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-10-06T21:12:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 21:11:35.932369837 +0000 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 21:05:14.961826433 +0000 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 21:05:14.961826433 +0000 2023-10-06T21:12:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 21:05:14.934826845 +0000 2023-10-06T21:12:38.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:12:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T21:12:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T21:12:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000317787 s, 1.6 MB/s 2023-10-06T21:12:38.285 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:12:38.330 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-10-06T21:12:38.330 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:12:38.378 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:12:38.379 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2023-10-06T21:12:38.465 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:12:38.465 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:38.465 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-10-06 21:12:22.198959527 +0000 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-10-06 21:04:58.929828628 +0000 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-10-06 21:04:58.929828628 +0000 2023-10-06T21:12:38.466 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-10-06 21:04:58.903829091 +0000 2023-10-06T21:12:38.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:12:38.533 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-10-06T21:12:38.533 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-10-06T21:12:38.533 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000484694 s, 1.1 MB/s 2023-10-06T21:12:38.536 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:12:38.583 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2023-10-06T21:12:38.666 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:12:38.666 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-10-06 21:12:22.199959509 +0000 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-10-06 21:04:59.756813919 +0000 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-10-06 21:04:59.756813919 +0000 2023-10-06T21:12:38.667 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-10-06 21:04:59.728814417 +0000 2023-10-06T21:12:38.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:12:38.772 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-10-06T21:12:38.772 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-10-06T21:12:38.772 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00030755 s, 1.7 MB/s 2023-10-06T21:12:38.774 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:12:38.820 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2023-10-06T21:12:38.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:38.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:38.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:38.904 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-10-06 21:12:22.199959509 +0000 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-10-06 21:05:00.594799014 +0000 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-10-06 21:05:00.594799014 +0000 2023-10-06T21:12:38.905 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-10-06 21:05:00.569799458 +0000 2023-10-06T21:12:38.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:39.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:39.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:39.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:39.011 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-10-06T21:12:39.011 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-10-06T21:12:39.011 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000301772 s, 1.7 MB/s 2023-10-06T21:12:39.013 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:39.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:39.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:39.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:39.058 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2023-10-06T21:12:39.141 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:12:39.141 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:39.141 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T21:12:39.142 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:39.142 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:39.142 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-10-06 21:12:22.199959509 +0000 2023-10-06T21:12:39.142 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-10-06 21:05:01.422784287 +0000 2023-10-06T21:12:39.143 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-10-06 21:05:01.422784287 +0000 2023-10-06T21:12:39.143 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-10-06 21:05:01.396784749 +0000 2023-10-06T21:12:39.143 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:12:39.247 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-10-06T21:12:39.247 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-10-06T21:12:39.247 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000319881 s, 1.6 MB/s 2023-10-06T21:12:39.250 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:12:39.295 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-10-06T21:12:39.295 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:12:39.340 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:12:39.341 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2023-10-06T21:12:39.427 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:12:39.427 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:39.427 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T21:12:39.427 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:39.428 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:39.428 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-10-06 21:12:12.218054850 +0000 2023-10-06T21:12:39.428 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-10-06 21:04:58.921959424 +0000 2023-10-06T21:12:39.428 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-10-06 21:04:58.921959424 +0000 2023-10-06T21:12:39.428 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-10-06 21:04:58.893959842 +0000 2023-10-06T21:12:39.428 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:12:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-10-06T21:12:39.503 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-10-06T21:12:39.503 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000594825 s, 861 kB/s 2023-10-06T21:12:39.505 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:12:39.550 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-10-06 21:12:12.218054850 +0000 2023-10-06T21:12:39.635 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-10-06 21:04:59.767946807 +0000 2023-10-06T21:12:39.636 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-10-06 21:04:59.767946807 +0000 2023-10-06T21:12:39.636 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-10-06 21:04:59.739947225 +0000 2023-10-06T21:12:39.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:12:39.735 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:39 smithi146 ceph-mon[102809]: Deploying daemon mgr.z on smithi146 2023-10-06T21:12:39.741 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-10-06T21:12:39.741 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-10-06T21:12:39.741 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000309031 s, 1.7 MB/s 2023-10-06T21:12:39.745 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:12:39.794 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:39.837 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-10-06 21:12:12.218054850 +0000 2023-10-06T21:12:39.838 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-10-06 21:05:00.609934250 +0000 2023-10-06T21:12:39.838 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-10-06 21:05:00.609934250 +0000 2023-10-06T21:12:39.838 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-10-06 21:05:00.581934668 +0000 2023-10-06T21:12:39.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:12:40.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:39 smithi031 ceph-mon[98539]: Deploying daemon mgr.z on smithi146 2023-10-06T21:12:40.113 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-10-06T21:12:40.113 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-10-06T21:12:40.113 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000328221 s, 1.6 MB/s 2023-10-06T21:12:40.116 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:12:40.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:39 smithi142 ceph-mon[102951]: Deploying daemon mgr.z on smithi146 2023-10-06T21:12:40.164 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-10-06 21:12:39.981636616 +0000 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-10-06 21:05:01.419922171 +0000 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-10-06 21:05:01.419922171 +0000 2023-10-06T21:12:40.256 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-10-06 21:05:01.391922589 +0000 2023-10-06T21:12:40.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:12:40.455 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-10-06T21:12:40.455 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-10-06T21:12:40.455 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000313721 s, 1.6 MB/s 2023-10-06T21:12:40.458 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:12:40.507 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2023-10-06T21:12:40.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:12:40.712 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:40 smithi146 systemd[1]: Starting Ceph mgr.z for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:12:41.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:40 smithi031 ceph-mon[98539]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:41.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:40 smithi146 ceph-mon[102809]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:41.049 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:40 smithi146 podman[105029]: 2023-10-06T21:12:41.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:40 smithi142 ceph-mon[102951]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:41.245 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:12:41.464 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-mgr[105069]: -- 172.21.15.146:0/538796783 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b442ce71e0 con 0x55b442cbcc00 2023-10-06T21:12:41.464 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 bash[105029]: ff57dd51b63c31a065dce91e5100c91ebc5d3ae9f9d191a0adef680768a5a495 2023-10-06T21:12:41.464 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 systemd[1]: Started Ceph mgr.z for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:12:41.464 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:41.357+0000 7fb12b428200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:12:41.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:41.461+0000 7fb12b428200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:12:41.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:41.574+0000 7fb12b428200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:12:41.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:41.692+0000 7fb12b428200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:12:42.299 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:41.913+0000 7fb12b428200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:42.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:42.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:42.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: Deploying daemon mgr.y on smithi142 2023-10-06T21:12:42.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:42 smithi142 ceph-mon[102951]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:42.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: Deploying daemon mgr.y on smithi142 2023-10-06T21:12:42.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:42 smithi031 ceph-mon[98539]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:42.658 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:42.372+0000 7fb12b428200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:12:42.658 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:42.539+0000 7fb12b428200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:12:42.658 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:42.654+0000 7fb12b428200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:12:42.659 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.660 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.661 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:42.661 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:42.661 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:12:42.662 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:42.663 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:42.664 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: Deploying daemon mgr.y on smithi142 2023-10-06T21:12:42.664 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-mon[102809]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:43.049 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:42.778+0000 7fb12b428200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0296871 s, 353 MB/s 2023-10-06T21:12:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-10-06T21:12:43.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:43 smithi142 systemd[1]: Starting Ceph mgr.y for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:12:43.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2023-10-06T21:12:43.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:43.316+0000 7fb12b428200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:12:44.070 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:43 smithi142 podman[104395]: 2023-10-06T21:12:44.070 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 bash[104395]: a531a27205923a4ace71ddeb2b8e0f30765b11d039e717714ebcb0d3e8465b39 2023-10-06T21:12:44.071 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mgr[104435]: -- 172.21.15.142:0/1886248614 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561c7aa8b1e0 con 0x561c7aa4c800 2023-10-06T21:12:44.071 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mgr[104435]: -- 172.21.15.142:0/1886248614 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c7aa771e0 con 0x561c7aa4c800 2023-10-06T21:12:44.309 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 systemd[1]: Started Ceph mgr.y for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:12:44.309 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:44.202+0000 7f6d834ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:12:44.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:44.306+0000 7f6d834ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:12:44.462 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:12:44.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:44.203+0000 7fb12b428200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:12:44.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:44.309+0000 7fb12b428200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:12:44.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:44.433+0000 7f6d834ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:12:44.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:44.554+0000 7f6d834ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:45.152 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:44.791+0000 7f6d834ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:12:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:45.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:45.573 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:45.282+0000 7f6d834ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:12:45.573 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:45.450+0000 7f6d834ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:12:45.681 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:45.316+0000 7fb12b428200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:12:45.681 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:45.444+0000 7fb12b428200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:12:45.681 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:45.678+0000 7fb12b428200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:12:45.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:45.571+0000 7f6d834ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:12:45.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:45.701+0000 7f6d834ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 -- 172.21.15.31:0/608265622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 msgr2=0x7fb3d01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/608265622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01025d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00099c0 tx=0x7fb3c0009cd0 comp rx=0 tx=0).stop 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 -- 172.21.15.31:0/608265622 shutdown_connections 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/608265622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 0x7fb3d01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/608265622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/608265622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01025d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 -- 172.21.15.31:0/608265622 >> 172.21.15.31:0/608265622 conn(0x7fb3d00fc4d0 msgr2=0x7fb3d00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 -- 172.21.15.31:0/608265622 shutdown_connections 2023-10-06T21:12:45.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.994+0000 7fb3d7b42700 1 -- 172.21.15.31:0/608265622 wait complete. 2023-10-06T21:12:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.995+0000 7fb3d7b42700 1 Processor -- start 2023-10-06T21:12:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 -- start start 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 0x7fb3d019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d019e730 con 0x7fb3d0102170 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d019e870 con 0x7fb3d0100f70 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d7b42700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d019ea00 con 0x7fb3d0102b10 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d50dd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d50dd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42448/0 (socket says 172.21.15.31:42448) 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d50dd700 1 -- 172.21.15.31:0/1150765563 learned_addr learned my addr 172.21.15.31:0/1150765563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d58de700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.996+0000 7fb3d60df700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 0x7fb3d019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 -- 172.21.15.31:0/1150765563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 msgr2=0x7fb3d019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 0x7fb3d019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 -- 172.21.15.31:0/1150765563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 msgr2=0x7fb3d0196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 -- 172.21.15.31:0/1150765563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009520 con 0x7fb3d0102170 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d58de700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d50dd700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009fd0 tx=0x7fb3c000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c001d070 con 0x7fb3d0102170 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c000fdb0 con 0x7fb3d0102170 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c00095f0 con 0x7fb3d0102170 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.997+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0004b10 con 0x7fb3d0102170 2023-10-06T21:12:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.998+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d019ef60 con 0x7fb3d0102170 2023-10-06T21:12:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.999+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fb3c0004c70 con 0x7fb3d0102170 2023-10-06T21:12:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.999+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d010c670 con 0x7fb3d0102170 2023-10-06T21:12:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.999+0000 7fb3c6ffd700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 0x7fb3bc03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.999+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb3c004c280 con 0x7fb3d0102170 2023-10-06T21:12:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:45.999+0000 7fb3d58de700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 0x7fb3bc03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:46.000+0000 7fb3d58de700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 0x7fb3bc03afb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009fd0 tx=0x7fb3cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:46.005+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb3c0029930 con 0x7fb3d0102170 2023-10-06T21:12:46.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:46.223+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d00612f0 con 0x7fb3bc038af0 2023-10-06T21:12:46.361 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:46.062+0000 7fb12b428200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:12:46.653 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:46.280+0000 7f6d834ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:12:46.788 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:46.358+0000 7fb12b428200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:12:46.788 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:46.552+0000 7fb12b428200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:12:47.049 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:46.901+0000 7fb12b428200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:12:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:12:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:47.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:47.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:47.186+0000 7f6d834ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:12:47.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:47.291+0000 7f6d834ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:12:48.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:48.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:12:48.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:48.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:48.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:48.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:47 smithi031 ceph-mon[98539]: Reconfiguring daemon mgr.x on smithi031 2023-10-06T21:12:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:12:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:48.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-mon[102809]: Reconfiguring daemon mgr.x on smithi031 2023-10-06T21:12:48.050 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:47.826+0000 7fb12b428200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:12:48.050 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:47.996+0000 7fb12b428200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:12:48.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:12:48.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:12:48.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:12:48.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:12:48.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:48.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:47 smithi142 ceph-mon[102951]: Reconfiguring daemon mgr.x on smithi031 2023-10-06T21:12:48.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:48.241+0000 7fb12b428200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:12:48.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:48.386+0000 7fb12b428200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:12:48.587 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:48.245+0000 7f6d834ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:12:48.587 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:48.362+0000 7f6d834ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:12:48.587 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:48.584+0000 7f6d834ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:12:48.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:48.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:48.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:48.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:48.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:48.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:48.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:49.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:49.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.049 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:48.660+0000 7fb12b428200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:12:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:49.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:49.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:49.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:12:49.241 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:48.961+0000 7f6d834ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:12:49.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:49.238+0000 7f6d834ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:12:49.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:49.428+0000 7f6d834ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:12:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:49.804+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7fb3c002b440 con 0x7fb3d0102170 2023-10-06T21:12:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-mon[102809]: Standby manager daemon z started 2023-10-06T21:12:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:12:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:12:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:50.050 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:12:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:12:49.734+0000 7fb12b428200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:12:50.151 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:49.767+0000 7f6d834ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:12:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-mon[102951]: Standby manager daemon z started 2023-10-06T21:12:50.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:12:50.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:50.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:12:50.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:49 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:49 smithi031 ceph-mon[98539]: Standby manager daemon z started 2023-10-06T21:12:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:49 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:12:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:49 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:49 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:12:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:49 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/2022151857' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:51.124 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:50.701+0000 7f6d834ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:50.876+0000 7f6d834ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:51.122+0000 7f6d834ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: mgrmap e14: x(active, since 84s), standbys: z 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]: dispatch 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]': finished 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:12:51.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:50 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: mgrmap e14: x(active, since 84s), standbys: z 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]: dispatch 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]': finished 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:12:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: mgrmap e14: x(active, since 84s), standbys: z 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]: dispatch 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3130142553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e79b590-506f-4710-912e-2de1d329bdcb"}]': finished 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:12:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:50 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:12:51.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:51.267+0000 7f6d834ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:12:51.811 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:51.546+0000 7f6d834ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:12:52.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:51 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2983709624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:12:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:51 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2983709624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:12:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:51 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2983709624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:12:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:12:52.820+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fb3c00053d0 con 0x7fb3d0102170 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:12:52.621+0000 7f6d834ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: Standby manager daemon y started 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:12:52.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:52 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: Standby manager daemon y started 2023-10-06T21:12:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:12:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:53.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:12:53.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:52 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: Standby manager daemon y started 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:12:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:52 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/2598872308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:12:54.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:53 smithi031 ceph-mon[98539]: mgrmap e15: x(active, since 87s), standbys: z, y 2023-10-06T21:12:54.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:53 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:12:54.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:53 smithi142 ceph-mon[102951]: mgrmap e15: x(active, since 87s), standbys: z, y 2023-10-06T21:12:54.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:53 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:12:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:53 smithi146 ceph-mon[102809]: mgrmap e15: x(active, since 87s), standbys: z, y 2023-10-06T21:12:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:53 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:12:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:54 smithi031 ceph-mon[98539]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:54 smithi142 ceph-mon[102951]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:54 smithi146 ceph-mon[102809]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:56 smithi142 ceph-mon[102951]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:57.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:56 smithi031 ceph-mon[98539]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:56 smithi146 ceph-mon[102809]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:12:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:12:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:12:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:58 smithi142 ceph-mon[102951]: Deploying daemon osd.0 on smithi031 2023-10-06T21:12:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:12:58 smithi142 ceph-mon[102951]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:59.188 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:58 smithi031 ceph-mon[98539]: Deploying daemon osd.0 on smithi031 2023-10-06T21:12:59.188 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:12:58 smithi031 ceph-mon[98539]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:12:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:58 smithi146 ceph-mon[102809]: Deploying daemon osd.0 on smithi031 2023-10-06T21:12:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:12:58 smithi146 ceph-mon[102809]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:01.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:00 smithi031 ceph-mon[98539]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:00 smithi142 ceph-mon[102951]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:00 smithi146 ceph-mon[102809]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:03.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:02 smithi031 ceph-mon[98539]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:03.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:02 smithi142 ceph-mon[102951]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:02 smithi146 ceph-mon[102809]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:04 smithi142 ceph-mon[102951]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:04 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:04 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:04 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:05.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:04 smithi031 ceph-mon[98539]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:05.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:04 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:04 smithi146 ceph-mon[102809]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:04 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:04 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:04 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:07.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:07.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:07.389 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2023-10-06T21:13:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.385+0000 7fb3c6ffd700 1 -- 172.21.15.31:0/1150765563 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb3d00612f0 con 0x7fb3bc038af0 2023-10-06T21:13:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.387+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 msgr2=0x7fb3bc03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.387+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 0x7fb3bc03afb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009fd0 tx=0x7fb3cc009260 comp rx=0 tx=0).stop 2023-10-06T21:13:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 msgr2=0x7fb3d01970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009fd0 tx=0x7fb3c000fa00 comp rx=0 tx=0).stop 2023-10-06T21:13:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 shutdown_connections 2023-10-06T21:13:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3bc038af0 0x7fb3bc03afb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0102b10 0x7fb3d019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d0100f70 0x7fb3d0196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 --2- 172.21.15.31:0/1150765563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d0102170 0x7fb3d01970e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 >> 172.21.15.31:0/1150765563 conn(0x7fb3d00fc4d0 msgr2=0x7fb3d00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:07.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 shutdown_connections 2023-10-06T21:13:07.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:13:07.388+0000 7fb3d7b42700 1 -- 172.21.15.31:0/1150765563 wait complete. 2023-10-06T21:13:07.791 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-06T21:13:07.797 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2023-10-06T21:13:07.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:13:07.884 INFO:journalctl@ceph.osd.0.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:13:08.260 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:13:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:13:07.884+0000 7f85e767b640 -1 osd.0 0 log_to_monitors true 2023-10-06T21:13:08.532 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:13:08.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:08.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:08 smithi031 ceph-mon[98539]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:08.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:08 smithi031 ceph-mon[98539]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:13:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:08.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:08 smithi142 ceph-mon[102951]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:08.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:08 smithi142 ceph-mon[102951]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:13:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:08 smithi146 ceph-mon[102809]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:08 smithi146 ceph-mon[102809]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:13:09.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:09 smithi142 ceph-mon[102951]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:13:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:09 smithi142 ceph-mon[102951]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:13:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:09 smithi142 ceph-mon[102951]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:13:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:09 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-mon[98539]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:13:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-mon[98539]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:13:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-mon[98539]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:13:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:09.762 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:13:09.349+0000 7f85dccf2700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T21:13:09.762 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:13:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:13:09.353+0000 7f85d82e6700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:13:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:09 smithi146 ceph-mon[102809]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:13:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:09 smithi146 ceph-mon[102809]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:13:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:09 smithi146 ceph-mon[102809]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:13:09.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:09 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0294242 s, 356 MB/s 2023-10-06T21:13:10.402 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-10-06T21:13:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:10 smithi142 ceph-mon[102951]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:13:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:10 smithi142 ceph-mon[102951]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:13:10.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:10 smithi142 ceph-mon[102951]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:10 smithi031 ceph-mon[98539]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:13:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:10 smithi031 ceph-mon[98539]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:13:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:10 smithi031 ceph-mon[98539]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:10.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:10 smithi146 ceph-mon[102809]: from='osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:13:10.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:10 smithi146 ceph-mon[102809]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:13:10.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:10.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:10 smithi146 ceph-mon[102809]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:13:10.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] boot 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:11.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.629 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:13:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:13:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] boot 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: osd.0 [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] boot 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:11.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:11.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:11.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:11.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:12 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi031 2023-10-06T21:13:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:12 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 17272M 2023-10-06T21:13:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:12 smithi142 ceph-mon[102951]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:13:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:12 smithi142 ceph-mon[102951]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:12.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:12 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi031 2023-10-06T21:13:12.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:12 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 17272M 2023-10-06T21:13:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:12 smithi031 ceph-mon[98539]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:13:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:12 smithi031 ceph-mon[98539]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:12 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi031 2023-10-06T21:13:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:12 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 17272M 2023-10-06T21:13:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:12 smithi146 ceph-mon[102809]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:13:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:12 smithi146 ceph-mon[102809]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:12.997 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 -- 172.21.15.142:0/1282410542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 msgr2=0x7fb9241025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/1282410542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb9241025d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb9140099c0 tx=0x7fb914009cd0 comp rx=0 tx=0).stop 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 -- 172.21.15.142:0/1282410542 shutdown_connections 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/1282410542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924102b10 0x7fb9241093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/1282410542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924100f70 0x7fb924101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/1282410542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb9241025d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.994+0000 7fb92c7f3700 1 -- 172.21.15.142:0/1282410542 >> 172.21.15.142:0/1282410542 conn(0x7fb9240fc4d0 msgr2=0x7fb9240fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.995+0000 7fb92c7f3700 1 -- 172.21.15.142:0/1282410542 shutdown_connections 2023-10-06T21:13:12.998 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.995+0000 7fb92c7f3700 1 -- 172.21.15.142:0/1282410542 wait complete. 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 Processor -- start 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 -- start start 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924100f70 0x7fb924196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924102b10 0x7fb92419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92419e6b0 con 0x7fb924100f70 2023-10-06T21:13:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92419e7f0 con 0x7fb924102170 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb92c7f3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92419e980 con 0x7fb924102b10 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.996+0000 7fb929d8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35494/0 (socket says 172.21.15.142:35494) 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 -- 172.21.15.142:0/564802263 learned_addr learned my addr 172.21.15.142:0/564802263 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb92a58f700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924100f70 0x7fb924196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb92ad90700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924102b10 0x7fb92419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 -- 172.21.15.142:0/564802263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924102b10 msgr2=0x7fb92419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924102b10 0x7fb92419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 -- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924100f70 msgr2=0x7fb924196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924100f70 0x7fb924196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 -- 172.21.15.142:0/564802263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140095f0 con 0x7fb924102170 2023-10-06T21:13:13.001 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb929d8e700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb914005910 tx=0x7fb914003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91401d070 con 0x7fb924102170 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb91400bd30 con 0x7fb924102170 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.997+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92419ec00 con 0x7fb924102170 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.998+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914022c80 con 0x7fb924102170 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.998+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92419f070 con 0x7fb924102170 2023-10-06T21:13:13.002 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:12.999+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92410c670 con 0x7fb924102170 2023-10-06T21:13:13.003 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.001+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fb914022470 con 0x7fb924102170 2023-10-06T21:13:13.004 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.002+0000 7fb91b7fe700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 0x7fb9100a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:13.004 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.002+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fb9140ccdd0 con 0x7fb924102170 2023-10-06T21:13:13.005 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.003+0000 7fb92a58f700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 0x7fb9100a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:13.005 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.003+0000 7fb92a58f700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 0x7fb9100a39b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb924101fd0 tx=0x7fb92000a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:13.007 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.005+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb914097e90 con 0x7fb924102170 2023-10-06T21:13:13.213 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:13.211+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9240612f0 con 0x7fb9100a14f0 2023-10-06T21:13:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:13.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:14.852 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:14 smithi142 ceph-mon[102951]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:14.852 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:14 smithi142 ceph-mon[102951]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:14 smithi031 ceph-mon[98539]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:14 smithi031 ceph-mon[98539]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:14 smithi146 ceph-mon[102809]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:14 smithi146 ceph-mon[102809]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/1345701799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]': finished 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:13:17.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/1345701799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]': finished 2023-10-06T21:13:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:13:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/1345701799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]: dispatch 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ed69e82-9719-4e1c-9a4c-3c78add6dd0f"}]': finished 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:13:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:18.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:17 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/600432704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:17 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/600432704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:18.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:17 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/600432704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:18 smithi142 ceph-mon[102951]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:18 smithi031 ceph-mon[98539]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:18 smithi146 ceph-mon[102809]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:20.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:20 smithi142 ceph-mon[102951]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:20 smithi031 ceph-mon[98539]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:20 smithi146 ceph-mon[102809]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:23.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:22 smithi142 ceph-mon[102951]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:23.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:22 smithi031 ceph-mon[98539]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:22 smithi146 ceph-mon[102809]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:24.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:13:24.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:23 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:24.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:23 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:13:24.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:23 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:23 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:13:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:23 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:24.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:24 smithi142 ceph-mon[102951]: Deploying daemon osd.1 on smithi142 2023-10-06T21:13:24.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:24 smithi142 ceph-mon[102951]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:25.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:24 smithi031 ceph-mon[98539]: Deploying daemon osd.1 on smithi142 2023-10-06T21:13:25.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:24 smithi031 ceph-mon[98539]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:25.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:24 smithi146 ceph-mon[102809]: Deploying daemon osd.1 on smithi142 2023-10-06T21:13:25.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:24 smithi146 ceph-mon[102809]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:26 smithi142 ceph-mon[102951]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:27.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:26 smithi031 ceph-mon[98539]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:27.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:26 smithi146 ceph-mon[102809]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:28 smithi142 ceph-mon[102951]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:29.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:28 smithi031 ceph-mon[98539]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:29.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:28 smithi146 ceph-mon[102809]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:30.698 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.698 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.698 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:30.698 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:30 smithi142 ceph-mon[102951]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:30.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:30 smithi031 ceph-mon[98539]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:30 smithi146 ceph-mon[102809]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:32.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:32.889 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2023-10-06T21:13:32.889 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.885+0000 7fb91b7fe700 1 -- 172.21.15.142:0/564802263 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb9240612f0 con 0x7fb9100a14f0 2023-10-06T21:13:32.891 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 msgr2=0x7fb9100a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:32.891 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 0x7fb9100a39b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb924101fd0 tx=0x7fb92000a3b0 comp rx=0 tx=0).stop 2023-10-06T21:13:32.891 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 msgr2=0x7fb924197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:32.891 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb914005910 tx=0x7fb914003a50 comp rx=0 tx=0).stop 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 shutdown_connections 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb9100a14f0 0x7fb9100a39b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924102b10 0x7fb92419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb924100f70 0x7fb924196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 --2- 172.21.15.142:0/564802263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924102170 0x7fb924197060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 >> 172.21.15.142:0/564802263 conn(0x7fb9240fc4d0 msgr2=0x7fb9240fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 shutdown_connections 2023-10-06T21:13:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:13:32.888+0000 7fb92c7f3700 1 -- 172.21.15.142:0/564802263 wait complete. 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:33.362 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-06T21:13:33.367 INFO:tasks.cephadm:Deploying osd.2 on smithi146 with /dev/vg_nvme/lv_4... 2023-10-06T21:13:33.367 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:13:33.450 INFO:journalctl@ceph.osd.1.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:13:34.101 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:13:34.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:33 smithi142 ceph-mon[102951]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:34.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:33 smithi142 ceph-mon[102951]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:33 smithi031 ceph-mon[98539]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:33 smithi031 ceph-mon[98539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:33 smithi146 ceph-mon[102809]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:33 smithi146 ceph-mon[102809]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:35.028 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:34 smithi146 ceph-mon[102809]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:35.151 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1[107995]: 2023-10-06T21:13:34.882+0000 7fdcad3d5700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T21:13:35.151 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1[107995]: 2023-10-06T21:13:34.887+0000 7fdca71c6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:34 smithi142 ceph-mon[102951]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: from='osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:34 smithi031 ceph-mon[98539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0294029 s, 357 MB/s 2023-10-06T21:13:36.057 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2023-10-06T21:13:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:35 smithi142 ceph-mon[102951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:13:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:35 smithi142 ceph-mon[102951]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:13:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:35 smithi146 ceph-mon[102809]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:13:36.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:35 smithi146 ceph-mon[102809]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:13:36.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:35 smithi031 ceph-mon[98539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:13:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:35 smithi031 ceph-mon[98539]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:13:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:36.579 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2023-10-06T21:13:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:13:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] boot 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:37.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] boot 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:13:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: osd.1 [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] boot 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:37.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:37.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:37.306 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:13:38.196 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:37 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi142 2023-10-06T21:13:38.196 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:37 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 17267M 2023-10-06T21:13:38.196 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:37 smithi146 ceph-mon[102809]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:13:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:37 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi142 2023-10-06T21:13:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:37 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 17267M 2023-10-06T21:13:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:37 smithi031 ceph-mon[98539]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:13:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:37 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi142 2023-10-06T21:13:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:37 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 17267M 2023-10-06T21:13:38.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:37 smithi142 ceph-mon[102951]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:13:38.803 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 -- 172.21.15.146:0/1516866066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 msgr2=0x7ff5ac1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 --2- 172.21.15.146:0/1516866066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1025d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 -- 172.21.15.146:0/1516866066 shutdown_connections 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 --2- 172.21.15.146:0/1516866066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac102b10 0x7ff5ac1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 --2- 172.21.15.146:0/1516866066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac100f70 0x7ff5ac101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 --2- 172.21.15.146:0/1516866066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1025d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 -- 172.21.15.146:0/1516866066 >> 172.21.15.146:0/1516866066 conn(0x7ff5ac0fc4d0 msgr2=0x7ff5ac0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:38.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 -- 172.21.15.146:0/1516866066 shutdown_connections 2023-10-06T21:13:38.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.800+0000 7ff5b321b700 1 -- 172.21.15.146:0/1516866066 wait complete. 2023-10-06T21:13:38.807 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.801+0000 7ff5b321b700 1 Processor -- start 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.801+0000 7ff5b321b700 1 -- start start 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 0x7ff5ac196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac102b10 0x7ff5ac19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac19e710 con 0x7ff5ac100f70 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac19e850 con 0x7ff5ac102b10 2023-10-06T21:13:38.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b321b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac19e9e0 con 0x7ff5ac102170 2023-10-06T21:13:38.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:38.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:54278/0 (socket says 172.21.15.146:54278) 2023-10-06T21:13:38.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 -- 172.21.15.146:0/2609158265 learned_addr learned my addr 172.21.15.146:0/2609158265 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:13:38.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b17b8700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac102b10 0x7ff5ac19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b0fb7700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 0x7ff5ac196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 -- 172.21.15.146:0/2609158265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac102b10 msgr2=0x7ff5ac19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac102b10 0x7ff5ac19e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 -- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 msgr2=0x7ff5ac196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 0x7ff5ac196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5abfff700 1 -- 172.21.15.146:0/2609158265 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980095f0 con 0x7ff5ac102170 2023-10-06T21:13:38.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.802+0000 7ff5b0fb7700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 0x7ff5ac196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5abfff700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff598005f50 tx=0x7ff598005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59801d070 con 0x7ff5ac102170 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff598004040 con 0x7ff5ac102170 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598022d70 con 0x7ff5ac102170 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac19ec60 con 0x7ff5ac102170 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.803+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac19f020 con 0x7ff5ac102170 2023-10-06T21:13:38.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.804+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac108610 con 0x7ff5ac102170 2023-10-06T21:13:38.812 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.806+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff598022470 con 0x7ff5ac102170 2023-10-06T21:13:38.812 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.807+0000 7ff5a9ffb700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 0x7ff59c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:38.813 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.807+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7ff5980cca30 con 0x7ff5ac102170 2023-10-06T21:13:38.813 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.808+0000 7ff5b0fb7700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 0x7ff59c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:38.813 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.809+0000 7ff5b0fb7700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 0x7ff59c0a38b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a0005f00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:38.815 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:38.812+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff5980978d0 con 0x7ff5ac102170 2023-10-06T21:13:39.023 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:39.020+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5ac10c670 con 0x7ff59c0a13f0 2023-10-06T21:13:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:38 smithi031 ceph-mon[98539]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:38 smithi146 ceph-mon[102809]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:38 smithi142 ceph-mon[102951]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:40.187 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:40.187 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:40.187 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:13:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:13:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:41.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:40 smithi146 ceph-mon[102809]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:41.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:40 smithi146 ceph-mon[102809]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:40 smithi031 ceph-mon[98539]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:40 smithi031 ceph-mon[98539]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:40 smithi142 ceph-mon[102951]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:13:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:40 smithi142 ceph-mon[102951]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/2481019204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]': finished 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:13:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/2216350797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/2481019204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]': finished 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:13:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/2216350797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/2481019204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]: dispatch 2023-10-06T21:13:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7"}]': finished 2023-10-06T21:13:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:13:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:13:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/2216350797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:13:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:44 smithi031 ceph-mon[98539]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:44 smithi146 ceph-mon[102809]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:44 smithi142 ceph-mon[102951]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:46 smithi142 ceph-mon[102951]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:46 smithi031 ceph-mon[98539]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:46 smithi146 ceph-mon[102809]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:48 smithi146 ceph-mon[102809]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:48 smithi142 ceph-mon[102951]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:48 smithi031 ceph-mon[98539]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:49 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:13:50.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:49 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:49 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:13:50.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:49 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:49 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:13:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:49 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:50 smithi142 ceph-mon[102951]: Deploying daemon osd.2 on smithi146 2023-10-06T21:13:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:50 smithi142 ceph-mon[102951]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:51.192 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:50 smithi146 ceph-mon[102809]: Deploying daemon osd.2 on smithi146 2023-10-06T21:13:51.192 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:50 smithi146 ceph-mon[102809]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:50 smithi031 ceph-mon[98539]: Deploying daemon osd.2 on smithi146 2023-10-06T21:13:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:50 smithi031 ceph-mon[98539]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:53.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:52 smithi146 ceph-mon[102809]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:53.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:52 smithi142 ceph-mon[102951]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:52 smithi031 ceph-mon[98539]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:54 smithi031 ceph-mon[98539]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:55.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:54 smithi146 ceph-mon[102809]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:54 smithi142 ceph-mon[102951]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:57.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:56 smithi146 ceph-mon[102809]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:57.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:56 smithi142 ceph-mon[102951]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:56 smithi031 ceph-mon[98539]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:13:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:13:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:13:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:13:59.302 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 2 on host 'smithi146' 2023-10-06T21:13:59.302 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.298+0000 7ff5a9ffb700 1 -- 172.21.15.146:0/2609158265 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5ac10c670 con 0x7ff59c0a13f0 2023-10-06T21:13:59.304 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 msgr2=0x7ff59c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:59.304 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 0x7ff59c0a38b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a0005f00 comp rx=0 tx=0).stop 2023-10-06T21:13:59.304 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 msgr2=0x7ff5ac1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff598005f50 tx=0x7ff598005740 comp rx=0 tx=0).stop 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 shutdown_connections 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff59c0a13f0 0x7ff59c0a38b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac102b10 0x7ff5ac19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5ac100f70 0x7ff5ac196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 --2- 172.21.15.146:0/2609158265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac102170 0x7ff5ac1970c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.301+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 >> 172.21.15.146:0/2609158265 conn(0x7ff5ac0fc4d0 msgr2=0x7ff5ac105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.302+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 shutdown_connections 2023-10-06T21:13:59.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:13:59.302+0000 7ff5b321b700 1 -- 172.21.15.146:0/2609158265 wait complete. 2023-10-06T21:13:59.725 DEBUG:teuthology.orchestra.run.smithi146:osd.2> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-06T21:13:59.729 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2023-10-06T21:13:59.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:13:59.803 INFO:journalctl@ceph.osd.2.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:14:00.473 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.502 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:00 smithi146 ceph-mon[102809]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:00 smithi031 ceph-mon[98539]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:14:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:00 smithi142 ceph-mon[102951]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:01.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:01 smithi031 ceph-mon[98539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:14:01.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:01 smithi031 ceph-mon[98539]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:14:01.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:01.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:01 smithi031 ceph-mon[98539]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:01.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:01 smithi031 ceph-mon[98539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-mon[102809]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:14:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-mon[102809]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:14:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:01.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-mon[102809]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:01.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-mon[102809]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:01.550 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2[108686]: 2023-10-06T21:14:01.268+0000 7fbdb0cfc700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T21:14:01.550 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2[108686]: 2023-10-06T21:14:01.275+0000 7fbdaa2ec700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:14:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:01 smithi142 ceph-mon[102951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:14:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:01 smithi142 ceph-mon[102951]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:14:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:01 smithi142 ceph-mon[102951]: from='osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:01 smithi142 ceph-mon[102951]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:14:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:14:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:14:02.446 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-10-06T21:14:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-10-06T21:14:02.446 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0295356 s, 355 MB/s 2023-10-06T21:14:02.447 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-10-06T21:14:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:02 smithi146 ceph-mon[102809]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:14:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:02 smithi146 ceph-mon[102809]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:14:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:02 smithi146 ceph-mon[102809]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:02.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:02 smithi031 ceph-mon[98539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:14:02.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:02 smithi031 ceph-mon[98539]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:14:02.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.609 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.609 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:02 smithi031 ceph-mon[98539]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:02 smithi142 ceph-mon[102951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:14:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:02 smithi142 ceph-mon[102951]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:14:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:02.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:02 smithi142 ceph-mon[102951]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:14:02.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] boot 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:14:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:14:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] boot 2023-10-06T21:14:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: osd.2 [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] boot 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:03.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:04.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:04 smithi031 ceph-mon[98539]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:14:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:04 smithi142 ceph-mon[102951]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:14:04.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:04 smithi146 ceph-mon[102809]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:14:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 -- 172.21.15.31:0/2198548249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 msgr2=0x7f5f9c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/2198548249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c073340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009990 tx=0x7f5f8c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 -- 172.21.15.31:0/2198548249 shutdown_connections 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/2198548249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c10b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/2198548249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c073340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/2198548249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 0x7f5f9c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 -- 172.21.15.31:0/2198548249 >> 172.21.15.31:0/2198548249 conn(0x7f5f9c0fc4d0 msgr2=0x7f5f9c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 -- 172.21.15.31:0/2198548249 shutdown_connections 2023-10-06T21:14:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.181+0000 7f5fa0b44700 1 -- 172.21.15.31:0/2198548249 wait complete. 2023-10-06T21:14:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 Processor -- start 2023-10-06T21:14:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 -- start start 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 0x7f5f9c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1a0870 con 0x7f5f9c074ee0 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1a09b0 con 0x7f5f9c108e40 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.183+0000 7f5fa0b44700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1a0b70 con 0x7f5f9c073910 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35492/0 (socket says 172.21.15.31:35492) 2023-10-06T21:14:05.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 -- 172.21.15.31:0/3822854126 learned_addr learned my addr 172.21.15.31:0/3822854126 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f9ad9d700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f9a59c700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 0x7f5f9c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 -- 172.21.15.31:0/3822854126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 msgr2=0x7f5f9c198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 0x7f5f9c198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 -- 172.21.15.31:0/3822854126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 msgr2=0x7f5f9c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 -- 172.21.15.31:0/3822854126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0095f0 con 0x7f5f9c074ee0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f9ad9d700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.184+0000 7f5f99d9b700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d980 tx=0x7f5f9000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.185+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000de60 con 0x7f5f9c074ee0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.185+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c1a0e50 con 0x7f5f9c074ee0 2023-10-06T21:14:05.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.185+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f900044b0 con 0x7f5f9c074ee0 2023-10-06T21:14:05.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.185+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90003b40 con 0x7f5f9c074ee0 2023-10-06T21:14:05.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.185+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c1a1270 con 0x7f5f9c074ee0 2023-10-06T21:14:05.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.186+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c1074f0 con 0x7f5f9c074ee0 2023-10-06T21:14:05.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.188+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5f90003ca0 con 0x7f5f9c074ee0 2023-10-06T21:14:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.190+0000 7f5f8b7fe700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 0x7f5f840a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.191+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f5f900cb580 con 0x7f5f9c074ee0 2023-10-06T21:14:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.191+0000 7f5f9a59c700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 0x7f5f840a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.191+0000 7f5f9a59c700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 0x7f5f840a3930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c000c00 tx=0x7f5f8c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:05.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.195+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5f90096000 con 0x7f5f9c074ee0 2023-10-06T21:14:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:05.417+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f9c02d2b0 con 0x7f5f840a1470 2023-10-06T21:14:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi146 2023-10-06T21:14:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 17267M 2023-10-06T21:14:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: pgmap v88: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:14:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:05 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi146 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 17267M 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: pgmap v88: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:14:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:05 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi146 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 17267M 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: pgmap v88: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:14:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:05 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:14:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:14:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:06 smithi142 ceph-mon[102951]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:14:06.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:06 smithi031 ceph-mon[98539]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:14:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:06 smithi146 ceph-mon[102809]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:07.010 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:06 smithi031 sudo[119973]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:07.010 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:06 smithi031 sudo[119973]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:07.010 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:06 smithi031 sudo[119973]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:07.010 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:06 smithi031 sudo[119973]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:07.593 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:07 smithi031 sudo[120006]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:07.593 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:07 smithi031 sudo[120006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:07.593 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:07 smithi031 sudo[120006]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:07.594 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:14:07 smithi031 sudo[120006]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:08.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:07 smithi031 ceph-mon[98539]: osdmap e22: 3 total, 3 up, 3 in 2023-10-06T21:14:08.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:08.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:07 smithi146 ceph-mon[102809]: osdmap e22: 3 total, 3 up, 3 in 2023-10-06T21:14:08.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:08.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:07 smithi142 ceph-mon[102951]: osdmap e22: 3 total, 3 up, 3 in 2023-10-06T21:14:08.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:08.151 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:07 smithi142 sudo[113309]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:08.152 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:07 smithi142 sudo[113309]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:08.152 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:07 smithi142 sudo[113309]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:08.152 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:07 smithi142 sudo[113309]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:08.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:08 smithi142 sudo[113312]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:08.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:08 smithi142 sudo[113312]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:08.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:08 smithi142 sudo[113312]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:08.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:14:08 smithi142 sudo[113312]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:08.721+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5f900963d0 con 0x7f5f9c074ee0 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: pgmap v93: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]: dispatch 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]': finished 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:14:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:09.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: pgmap v93: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:09.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:09.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]: dispatch 2023-10-06T21:14:09.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]': finished 2023-10-06T21:14:09.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:14:09.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:09.049 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:08 smithi146 sudo[113950]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:09.049 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:08 smithi146 sudo[113950]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:09.049 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:08 smithi146 sudo[113950]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:09.050 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:08 smithi146 sudo[113950]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: pgmap v93: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]: dispatch 2023-10-06T21:14:09.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3170178314' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53868b14-df36-4751-89c7-54721dec7417"}]': finished 2023-10-06T21:14:09.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:14:09.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:09.799 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:09 smithi146 sudo[113953]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:09.799 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:09 smithi146 sudo[113953]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:09.799 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:09 smithi146 sudo[113953]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:09.799 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:14:09 smithi146 sudo[113953]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:09 smithi031 sudo[120343]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:09 smithi031 sudo[120343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:09 smithi031 sudo[120343]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 sudo[120343]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:10.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:10 smithi142 ceph-mon[102951]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:14:10.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:10 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/913285225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:10.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:10.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:10 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 sudo[120534]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 sudo[120534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 sudo[120534]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 sudo[120534]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 ceph-mon[98539]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/913285225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:10 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:11.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:10 smithi146 ceph-mon[102809]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:14:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:10 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/913285225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:10 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:11.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113956]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113956]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113956]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: pgmap v95: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:12.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:12.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113959]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113959]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 sudo[113959]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: pgmap v95: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:14:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113315]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:12.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:12.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:12 smithi031 ceph-mon[98539]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:12.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113326]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:14:12.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113326]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 sudo[113326]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: pgmap v95: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:12.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:12.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:12.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:12.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:12.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:12 smithi142 ceph-mon[102951]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:14:13.051 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:12 smithi146 ceph-mon[102809]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:13.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:13 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:13.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:13 smithi142 ceph-mon[102951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:14.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:13 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:14.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:13 smithi031 ceph-mon[98539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:13 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:14:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:13 smithi146 ceph-mon[102809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:14:14.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:14 smithi031 ceph-mon[98539]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:14 smithi146 ceph-mon[102809]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:14 smithi142 ceph-mon[102951]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:17.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:16 smithi031 ceph-mon[98539]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:17.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:14:17.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:16 smithi142 ceph-mon[102951]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:14:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:16 smithi146 ceph-mon[102809]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:14:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:18.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:17 smithi031 ceph-mon[98539]: Deploying daemon osd.3 on smithi031 2023-10-06T21:14:18.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:17 smithi142 ceph-mon[102951]: Deploying daemon osd.3 on smithi031 2023-10-06T21:14:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:17 smithi146 ceph-mon[102809]: Deploying daemon osd.3 on smithi031 2023-10-06T21:14:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:18 smithi142 ceph-mon[102951]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-06T21:14:19.181 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:18 smithi031 ceph-mon[98539]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-06T21:14:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:18 smithi146 ceph-mon[102809]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-06T21:14:21.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:20 smithi031 ceph-mon[98539]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:14:21.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:20 smithi142 ceph-mon[102951]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:14:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:20 smithi146 ceph-mon[102809]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:14:23.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:22 smithi031 ceph-mon[98539]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:14:23.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:23.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:22 smithi142 ceph-mon[102951]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:14:23.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:22 smithi146 ceph-mon[102809]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:14:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:24.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:24 smithi031 ceph-mon[98539]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:25.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:24 smithi146 ceph-mon[102809]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:24 smithi142 ceph-mon[102951]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:25.681 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2023-10-06T21:14:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.678+0000 7f5f8b7fe700 1 -- 172.21.15.31:0/3822854126 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f9c02d2b0 con 0x7f5f840a1470 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 msgr2=0x7f5f840a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 0x7f5f840a3930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c000c00 tx=0x7f5f8c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 msgr2=0x7f5f9c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d980 tx=0x7f5f9000dd40 comp rx=0 tx=0).stop 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 shutdown_connections 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f840a1470 0x7f5f840a3930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f9c108e40 0x7f5f9c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c073910 0x7f5f9c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 --2- 172.21.15.31:0/3822854126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f9c074ee0 0x7f5f9c199260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 >> 172.21.15.31:0/3822854126 conn(0x7f5f9c0fc4d0 msgr2=0x7f5f9c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 shutdown_connections 2023-10-06T21:14:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:14:25.681+0000 7f5fa0b44700 1 -- 172.21.15.31:0/3822854126 wait complete. 2023-10-06T21:14:26.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:26.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-mon[98539]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:26.123 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-06T21:14:26.127 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2023-10-06T21:14:26.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:14:26.237 INFO:journalctl@ceph.osd.3.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:14:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:26.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:26 smithi142 ceph-mon[102951]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:26.510 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:14:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:14:26.239+0000 7ff1e9bce640 -1 osd.3 0 log_to_monitors true 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:26.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:26 smithi146 ceph-mon[102809]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:14:26.877 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:14:27.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:27 smithi031 ceph-mon[98539]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:14:27.382 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:27 smithi142 ceph-mon[102951]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:14:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:27 smithi146 ceph-mon[102809]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:14:28.076 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-mon[98539]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:14:28.076 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-mon[98539]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:14:28.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-mon[98539]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:14:28.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:28.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-mon[98539]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:28 smithi142 ceph-mon[102951]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:14:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:28 smithi142 ceph-mon[102951]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:14:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:28 smithi142 ceph-mon[102951]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:14:28.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:28.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:28 smithi142 ceph-mon[102951]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:28.510 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:14:28.072+0000 7ff1df245700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T21:14:28.510 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:14:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:14:28.082+0000 7ff1d9036700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:14:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:28 smithi146 ceph-mon[102809]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:14:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:28 smithi146 ceph-mon[102809]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:14:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:28 smithi146 ceph-mon[102809]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:14:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:28 smithi146 ceph-mon[102809]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:14:28.831 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:14:28.831 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:14:28.831 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-10-06T21:14:28.832 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-10-06T21:14:28.832 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0462653 s, 227 MB/s 2023-10-06T21:14:28.832 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 2023-10-06T21:14:28.832 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: osdmap e25: 4 total, 3 up, 4 in 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] boot 2023-10-06T21:14:29.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:14:29.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:29 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: osdmap e25: 4 total, 3 up, 4 in 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] boot 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:14:29.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:29 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: from='osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: osdmap e25: 4 total, 3 up, 4 in 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: osd.3 [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] boot 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:14:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:29 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:14:29.971 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:14:30.362 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi031 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 8636M 2023-10-06T21:14:30.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:30.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:30.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:14:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi031 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 8636M 2023-10-06T21:14:30.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:30.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:30.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi031 2023-10-06T21:14:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 8636M 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:30.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 -- 172.21.15.142:0/895284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 msgr2=0x7fec30073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 --2- 172.21.15.142:0/895284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30073d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fec24009990 tx=0x7fec24009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 -- 172.21.15.142:0/895284277 shutdown_connections 2023-10-06T21:14:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 --2- 172.21.15.142:0/895284277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec3010b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 --2- 172.21.15.142:0/895284277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 0x7fec30073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 --2- 172.21.15.142:0/895284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30073d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 -- 172.21.15.142:0/895284277 >> 172.21.15.142:0/895284277 conn(0x7fec300fc4d0 msgr2=0x7fec300fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 -- 172.21.15.142:0/895284277 shutdown_connections 2023-10-06T21:14:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.463+0000 7fec3657a700 1 -- 172.21.15.142:0/895284277 wait complete. 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.464+0000 7fec3657a700 1 Processor -- start 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 -- start start 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 0x7fec301992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec301a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:31.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301a08e0 con 0x7fec30074ee0 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301a0a20 con 0x7fec30073910 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec3657a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301a0be0 con 0x7fec30108e40 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec2ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec2ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33576/0 (socket says 172.21.15.142:33576) 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec2ffff700 1 -- 172.21.15.142:0/2196186773 learned_addr learned my addr 172.21.15.142:0/2196186773 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec34b17700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec301a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.465+0000 7fec2f7fe700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 0x7fec301992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 -- 172.21.15.142:0/2196186773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 msgr2=0x7fec301a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:31.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec301a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 -- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 msgr2=0x7fec301992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 0x7fec301992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 -- 172.21.15.142:0/2196186773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240095f0 con 0x7fec30073910 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec34b17700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec301a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2ffff700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e970 tx=0x7fec2000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20009790 con 0x7fec30073910 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec2000eec0 con 0x7fec30073910 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200054b0 con 0x7fec30073910 2023-10-06T21:14:31.472 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec301a0ec0 con 0x7fec30073910 2023-10-06T21:14:31.473 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.466+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec301a1280 con 0x7fec30073910 2023-10-06T21:14:31.473 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.467+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec301074f0 con 0x7fec30073910 2023-10-06T21:14:31.474 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.471+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fec20005610 con 0x7fec30073910 2023-10-06T21:14:31.475 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.472+0000 7fec2d7fa700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 0x7fec180a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:31.476 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.473+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fec20014070 con 0x7fec30073910 2023-10-06T21:14:31.476 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.473+0000 7fec2f7fe700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 0x7fec180a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:31.476 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.474+0000 7fec2f7fe700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 0x7fec180a3a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec24005910 tx=0x7fec2400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:31.478 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.475+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec200966d0 con 0x7fec30073910 2023-10-06T21:14:31.685 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:31.680+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fec3002d350 con 0x7fec180a1580 2023-10-06T21:14:32.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:31 smithi031 ceph-mon[98539]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:14:32.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:32.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:32.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:31 smithi146 ceph-mon[102809]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:14:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:31 smithi142 ceph-mon[102951]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:14:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:32 smithi031 ceph-mon[98539]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:32 smithi031 ceph-mon[98539]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:14:33.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:32 smithi031 ceph-mon[98539]: pgmap v112: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 156 KiB/s, 0 objects/s recovering 2023-10-06T21:14:33.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:32 smithi142 ceph-mon[102951]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:33.028 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:32 smithi142 ceph-mon[102951]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:14:33.028 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:32 smithi142 ceph-mon[102951]: pgmap v112: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 156 KiB/s, 0 objects/s recovering 2023-10-06T21:14:33.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:32 smithi146 ceph-mon[102809]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:14:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:32 smithi146 ceph-mon[102809]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:14:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:32 smithi146 ceph-mon[102809]: pgmap v112: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 156 KiB/s, 0 objects/s recovering 2023-10-06T21:14:33.843 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:33 smithi142 ceph-mon[102951]: osdmap e30: 4 total, 4 up, 4 in 2023-10-06T21:14:34.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:33 smithi031 ceph-mon[98539]: osdmap e30: 4 total, 4 up, 4 in 2023-10-06T21:14:34.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:33 smithi146 ceph-mon[102809]: osdmap e30: 4 total, 4 up, 4 in 2023-10-06T21:14:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:34 smithi031 ceph-mon[98539]: pgmap v114: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-10-06T21:14:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:34 smithi146 ceph-mon[102809]: pgmap v114: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-10-06T21:14:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:34 smithi142 ceph-mon[102951]: pgmap v114: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-10-06T21:14:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/1271379500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]': finished 2023-10-06T21:14:36.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:14:36.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:36.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:35 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/1368944588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/1271379500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]': finished 2023-10-06T21:14:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:14:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:36.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:35 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/1368944588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:36.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/1271379500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]: dispatch 2023-10-06T21:14:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95502ff0-ad06-45c6-8fe0-de7706d59da0"}]': finished 2023-10-06T21:14:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:14:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:35 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/1368944588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:14:37.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:36 smithi031 ceph-mon[98539]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-06T21:14:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:36 smithi146 ceph-mon[102809]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-06T21:14:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:36 smithi142 ceph-mon[102951]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-06T21:14:39.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:38 smithi142 ceph-mon[102951]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:38 smithi031 ceph-mon[98539]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:39.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:38 smithi146 ceph-mon[102809]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:40.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:40 smithi142 ceph-mon[102951]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:40 smithi031 ceph-mon[98539]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:40 smithi146 ceph-mon[102809]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:42 smithi142 ceph-mon[102951]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:14:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:42 smithi031 ceph-mon[98539]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:14:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:42 smithi146 ceph-mon[102809]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:14:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:44.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:43 smithi142 ceph-mon[102951]: Deploying daemon osd.4 on smithi142 2023-10-06T21:14:44.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:43 smithi031 ceph-mon[98539]: Deploying daemon osd.4 on smithi142 2023-10-06T21:14:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:43 smithi146 ceph-mon[102809]: Deploying daemon osd.4 on smithi142 2023-10-06T21:14:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:44 smithi142 ceph-mon[102951]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:44 smithi031 ceph-mon[98539]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:44 smithi146 ceph-mon[102809]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:46 smithi142 ceph-mon[102951]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:46 smithi031 ceph-mon[98539]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:46 smithi146 ceph-mon[102809]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:48 smithi142 ceph-mon[102951]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:48 smithi031 ceph-mon[98539]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:48 smithi146 ceph-mon[102809]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:50 smithi142 ceph-mon[102951]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:50 smithi031 ceph-mon[98539]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:50 smithi146 ceph-mon[102809]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:51.859 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.856+0000 7fec2d7fa700 1 -- 172.21.15.142:0/2196186773 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec3002d350 con 0x7fec180a1580 2023-10-06T21:14:51.859 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 msgr2=0x7fec180a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 0x7fec180a3a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec24005910 tx=0x7fec2400b260 comp rx=0 tx=0).stop 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 msgr2=0x7fec30198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e970 tx=0x7fec2000ec80 comp rx=0 tx=0).stop 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 shutdown_connections 2023-10-06T21:14:51.862 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec180a1580 0x7fec180a3a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec30108e40 0x7fec301a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec30073910 0x7fec30198d90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 --2- 172.21.15.142:0/2196186773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec30074ee0 0x7fec301992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.859+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 >> 172.21.15.142:0/2196186773 conn(0x7fec300fc4d0 msgr2=0x7fec300fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.860+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 shutdown_connections 2023-10-06T21:14:51.863 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:14:51.860+0000 7fec3657a700 1 -- 172.21.15.142:0/2196186773 wait complete. 2023-10-06T21:14:52.292 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.292 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.292 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:52.292 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:52.292 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.293 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:52.293 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.293 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.293 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:52.294 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-06T21:14:52.298 INFO:tasks.cephadm:Deploying osd.5 on smithi146 with /dev/vg_nvme/lv_3... 2023-10-06T21:14:52.298 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:14:52.367 INFO:journalctl@ceph.osd.4.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:14:52.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:52.651 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:14:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:14:52.535+0000 7f530ff5c640 -1 osd.4 0 log_to_monitors true 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:52.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:52.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:53.052 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:14:53.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:53 smithi142 ceph-mon[102951]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:53.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:53 smithi142 ceph-mon[102951]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:53.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:53 smithi146 ceph-mon[102809]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:53 smithi146 ceph-mon[102809]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:53 smithi031 ceph-mon[98539]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:53 smithi031 ceph-mon[98539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:14:54.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:54 smithi146 ceph-mon[102809]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:14:54.651 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:14:54.289+0000 7f53055d3700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T21:14:54.651 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:14:54.297+0000 7f53003c6700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:54 smithi142 ceph-mon[102951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: from='osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:54 smithi031 ceph-mon[98539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.02966 s, 354 MB/s 2023-10-06T21:14:55.141 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2023-10-06T21:14:55.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:55 smithi146 ceph-mon[102809]: osdmap e33: 5 total, 4 up, 5 in 2023-10-06T21:14:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:55 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:55 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:55.625 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2023-10-06T21:14:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:55 smithi142 ceph-mon[102951]: osdmap e33: 5 total, 4 up, 5 in 2023-10-06T21:14:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:55 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:55 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:55.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:55 smithi031 ceph-mon[98539]: osdmap e33: 5 total, 4 up, 5 in 2023-10-06T21:14:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:55 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:55 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.356 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:14:56.612 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] boot 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi142 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.613 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 8633M 2023-10-06T21:14:56.614 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.614 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:56.614 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:56.614 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.614 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:56 smithi146 ceph-mon[102809]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:14:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] boot 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi142 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 8633M 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:56.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:56.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:56 smithi142 ceph-mon[102951]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: osd.4 [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] boot 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:14:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi142 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 8633M 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:14:56.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:56 smithi031 ceph-mon[98539]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:14:57.592 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:57 smithi146 ceph-mon[102809]: osdmap e35: 5 total, 5 up, 5 in 2023-10-06T21:14:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:57 smithi142 ceph-mon[102951]: osdmap e35: 5 total, 5 up, 5 in 2023-10-06T21:14:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:57 smithi031 ceph-mon[98539]: osdmap e35: 5 total, 5 up, 5 in 2023-10-06T21:14:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 -- 172.21.15.146:0/751202283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 msgr2=0x7f10101093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 --2- 172.21.15.146:0/751202283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10101093c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f100800b0a0 tx=0x7f100800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 -- 172.21.15.146:0/751202283 shutdown_connections 2023-10-06T21:14:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 --2- 172.21.15.146:0/751202283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10101093c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 --2- 172.21.15.146:0/751202283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 0x7f1010101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 --2- 172.21.15.146:0/751202283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 0x7f10101025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 -- 172.21.15.146:0/751202283 >> 172.21.15.146:0/751202283 conn(0x7f10100fc4d0 msgr2=0x7f10100fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 -- 172.21.15.146:0/751202283 shutdown_connections 2023-10-06T21:14:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.829+0000 7f1017106700 1 -- 172.21.15.146:0/751202283 wait complete. 2023-10-06T21:14:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.831+0000 7f1017106700 1 Processor -- start 2023-10-06T21:14:57.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.831+0000 7f1017106700 1 -- start start 2023-10-06T21:14:57.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.831+0000 7f1017106700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 0x7f1010074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.831+0000 7f1017106700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 0x7f10100732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.831+0000 7f1017106700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f1017106700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010073f60 con 0x7f1010100f70 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f1017106700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100740d0 con 0x7f1010102170 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f1017106700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10101a0ca0 con 0x7f1010102b10 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37716/0 (socket says 172.21.15.146:37716) 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 -- 172.21.15.146:0/3420690654 learned_addr learned my addr 172.21.15.146:0/3420690654 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f1014ea2700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 0x7f1010074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:57.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f100ffff700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 0x7f10100732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 -- 172.21.15.146:0/3420690654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 msgr2=0x7f10100732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 0x7f10100732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 -- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 msgr2=0x7f1010074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 0x7f1010074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 -- 172.21.15.146:0/3420690654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009d00 con 0x7f1010102b10 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.832+0000 7f10156a3700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f100800d010 tx=0x7f100800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.833+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800e040 con 0x7f1010102b10 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.833+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1008003e10 con 0x7f1010102b10 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.833+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10101a0f20 con 0x7f1010102b10 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.833+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080052a0 con 0x7f1010102b10 2023-10-06T21:14:57.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.833+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10101a12d0 con 0x7f1010102b10 2023-10-06T21:14:57.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.834+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101010c630 con 0x7f1010102b10 2023-10-06T21:14:57.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.836+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1008005400 con 0x7f1010102b10 2023-10-06T21:14:57.840 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.837+0000 7f100dffb700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 0x7f10000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:57.840 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.837+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f10080ccf70 con 0x7f1010102b10 2023-10-06T21:14:57.840 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.838+0000 7f1014ea2700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 0x7f10000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:57.841 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.838+0000 7f1014ea2700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 0x7f10000a3940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1010101fd0 tx=0x7f1004009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:57.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:57.842+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1008097630 con 0x7f1010102b10 2023-10-06T21:14:58.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:14:58.048+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f10100612f0 con 0x7f10000a1480 2023-10-06T21:14:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:58 smithi142 ceph-mon[102951]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:14:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:58.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:58 smithi031 ceph-mon[98539]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:14:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:58 smithi146 ceph-mon[102809]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:14:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:14:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:14:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:59.984 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:14:59 smithi146 ceph-mon[102809]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:00.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:14:59 smithi031 ceph-mon[98539]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:14:59 smithi142 ceph-mon[102951]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:01.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:00 smithi031 ceph-mon[98539]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:01.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:00 smithi146 ceph-mon[102809]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:00 smithi142 ceph-mon[102951]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/2403170209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:01 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:01 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]': finished 2023-10-06T21:15:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:01 smithi031 ceph-mon[98539]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:15:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:02.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/2403170209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:01 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:01 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]': finished 2023-10-06T21:15:02.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:01 smithi146 ceph-mon[102809]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:15:02.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/2403170209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:01 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]: dispatch 2023-10-06T21:15:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:01 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d0247c-93bc-4b4c-8855-087be5ba9a66"}]': finished 2023-10-06T21:15:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:01 smithi142 ceph-mon[102951]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:15:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:03.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/3608921157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:03.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:02 smithi031 ceph-mon[98539]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:03.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:02 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/3608921157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:03.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:02 smithi146 ceph-mon[102809]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:03.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:02 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/3608921157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:03.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:02 smithi142 ceph-mon[102951]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:05.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:04 smithi031 ceph-mon[98539]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:05.018 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:04 smithi146 ceph-mon[102809]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:04 smithi142 ceph-mon[102951]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:07.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:06 smithi146 ceph-mon[102809]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:06 smithi142 ceph-mon[102951]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:06 smithi031 ceph-mon[98539]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:08 smithi142 ceph-mon[102951]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:15:09.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:08 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:09.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:08 smithi146 ceph-mon[102809]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:09.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:15:09.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:08 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:08 smithi031 ceph-mon[98539]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:15:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:08 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:10.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:09 smithi142 ceph-mon[102951]: Deploying daemon osd.5 on smithi146 2023-10-06T21:15:10.340 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:09 smithi031 ceph-mon[98539]: Deploying daemon osd.5 on smithi146 2023-10-06T21:15:10.341 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:09 smithi146 ceph-mon[102809]: Deploying daemon osd.5 on smithi146 2023-10-06T21:15:11.026 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:10 smithi146 ceph-mon[102809]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:11.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:10 smithi142 ceph-mon[102951]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:10 smithi031 ceph-mon[98539]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:13.089 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:12 smithi146 ceph-mon[102809]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:13.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:12 smithi142 ceph-mon[102951]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:12 smithi031 ceph-mon[98539]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:14 smithi031 ceph-mon[98539]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:14 smithi146 ceph-mon[102809]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:14 smithi142 ceph-mon[102951]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:17.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:16 smithi146 ceph-mon[102809]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:16 smithi142 ceph-mon[102951]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:16 smithi031 ceph-mon[98539]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:17.726 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.724+0000 7f100dffb700 1 -- 172.21.15.146:0/3420690654 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f10100612f0 con 0x7f10000a1480 2023-10-06T21:15:17.727 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 5 on host 'smithi146' 2023-10-06T21:15:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 msgr2=0x7f10000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 0x7f10000a3940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1010101fd0 tx=0x7f1004009260 comp rx=0 tx=0).stop 2023-10-06T21:15:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 msgr2=0x7f10100737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f100800d010 tx=0x7f100800bce0 comp rx=0 tx=0).stop 2023-10-06T21:15:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 shutdown_connections 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f10000a1480 0x7f10000a3940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1010102b10 0x7f10100737e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1010100f70 0x7f1010074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 --2- 172.21.15.146:0/3420690654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102170 0x7f10100732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 >> 172.21.15.146:0/3420690654 conn(0x7f10100fc4d0 msgr2=0x7f10100fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 shutdown_connections 2023-10-06T21:15:17.730 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:15:17.727+0000 7f1017106700 1 -- 172.21.15.146:0/3420690654 wait complete. 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.247 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.248 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:18.287 DEBUG:teuthology.orchestra.run.smithi146:osd.5> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-06T21:15:18.291 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2023-10-06T21:15:18.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:15:18.362 INFO:journalctl@ceph.osd.5.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:15:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:15:19.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:18 smithi146 ceph-mon[102809]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:19.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:18 smithi146 ceph-mon[102809]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:19.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:18 smithi146 ceph-mon[102809]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:18 smithi031 ceph-mon[98539]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:18 smithi031 ceph-mon[98539]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:18 smithi031 ceph-mon[98539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:18 smithi142 ceph-mon[102951]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:18 smithi142 ceph-mon[102951]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:18 smithi142 ceph-mon[102951]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:15:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-mon[102809]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:15:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-mon[102809]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:15:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-mon[102809]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:20.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-mon[102809]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:20.049 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:15:20.006+0000 7fe262314700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T21:15:20.050 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:15:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:15:20.014+0000 7fe25b904700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:15:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:19 smithi031 ceph-mon[98539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:15:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:19 smithi031 ceph-mon[98539]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:15:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:19 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:19 smithi031 ceph-mon[98539]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:19 smithi031 ceph-mon[98539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:20 smithi142 ceph-mon[102951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:15:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:20 smithi142 ceph-mon[102951]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:15:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:20 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:20.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:20 smithi142 ceph-mon[102951]: from='osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:20.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:20 smithi142 ceph-mon[102951]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:15:21.043 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:15:21.043 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:15:21.043 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-10-06T21:15:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-10-06T21:15:21.044 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0317765 s, 330 MB/s 2023-10-06T21:15:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-10-06T21:15:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:21 smithi031 ceph-mon[98539]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:21 smithi031 ceph-mon[98539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:15:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:21 smithi031 ceph-mon[98539]: osdmap e38: 6 total, 5 up, 6 in 2023-10-06T21:15:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:21 smithi146 ceph-mon[102809]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:21 smithi146 ceph-mon[102809]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:15:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:21 smithi146 ceph-mon[102809]: osdmap e38: 6 total, 5 up, 6 in 2023-10-06T21:15:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:21 smithi142 ceph-mon[102951]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:15:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:21 smithi142 ceph-mon[102951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:15:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:21 smithi142 ceph-mon[102951]: osdmap e38: 6 total, 5 up, 6 in 2023-10-06T21:15:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:21.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2023-10-06T21:15:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] boot 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:22.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:22.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:22 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:15:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:15:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] boot 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:22 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: osd.5 [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] boot 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:22.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:22 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:23.141 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:23 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi146 2023-10-06T21:15:23.141 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:23 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 8633M 2023-10-06T21:15:23.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:23 smithi031 ceph-mon[98539]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:23.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:23 smithi031 ceph-mon[98539]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:15:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:23 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi146 2023-10-06T21:15:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:23 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 8633M 2023-10-06T21:15:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:23 smithi146 ceph-mon[102809]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:23 smithi146 ceph-mon[102809]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:15:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:23 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi146 2023-10-06T21:15:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:23 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 8633M 2023-10-06T21:15:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:23 smithi142 ceph-mon[102951]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:23 smithi142 ceph-mon[102951]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 -- 172.21.15.31:0/1277847890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4102170 msgr2=0x7f28b41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/1277847890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4102170 0x7f28b41025d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28a40099c0 tx=0x7f28a4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 -- 172.21.15.31:0/1277847890 shutdown_connections 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/1277847890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 0x7f28b41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/1277847890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4100f70 0x7f28b4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/1277847890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4102170 0x7f28b41025d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 -- 172.21.15.31:0/1277847890 >> 172.21.15.31:0/1277847890 conn(0x7f28b40fc4d0 msgr2=0x7f28b40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 -- 172.21.15.31:0/1277847890 shutdown_connections 2023-10-06T21:15:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.767+0000 7f28bc0f4700 1 -- 172.21.15.31:0/1277847890 wait complete. 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.768+0000 7f28bc0f4700 1 Processor -- start 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.768+0000 7f28bc0f4700 1 -- start start 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4102170 0x7f28b40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 0x7f28b40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4073f60 con 0x7f28b4100f70 2023-10-06T21:15:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b40740d0 con 0x7f28b4102170 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28bc0f4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b41a0c40 con 0x7f28b4102b10 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38090/0 (socket says 172.21.15.31:38090) 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 -- 172.21.15.31:0/2803783749 learned_addr learned my addr 172.21.15.31:0/2803783749 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28ba691700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 0x7f28b40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b968f700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4102170 0x7f28b40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 -- 172.21.15.31:0/2803783749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 msgr2=0x7f28b40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 0x7f28b40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 -- 172.21.15.31:0/2803783749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4102170 msgr2=0x7f28b40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4102170 0x7f28b40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.769+0000 7f28b9e90700 1 -- 172.21.15.31:0/2803783749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b4100f70 2023-10-06T21:15:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28b9e90700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28b000d980 tx=0x7f28b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0004020 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28b000f690 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00039e0 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b41a0f20 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.770+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b41a12d0 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.771+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b41086a0 con 0x7f28b4100f70 2023-10-06T21:15:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.773+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f28b000f800 con 0x7f28b4100f70 2023-10-06T21:15:23.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.775+0000 7f28aaffd700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 0x7f28a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:23.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.775+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f28b00cbdd0 con 0x7f28b4100f70 2023-10-06T21:15:23.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.775+0000 7f28b968f700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 0x7f28a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:23.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.775+0000 7f28b968f700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 0x7f28a00a3940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b580 tx=0x7f28a4005ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:23.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.778+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f28b0096250 con 0x7f28b4100f70 2023-10-06T21:15:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:23.997+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b402d350 con 0x7f28a00a1480 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:15:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:15:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:24.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:24.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:24.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:24.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:26.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:25 smithi031 ceph-mon[98539]: osdmap e43: 6 total, 6 up, 6 in 2023-10-06T21:15:26.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:25 smithi146 ceph-mon[102809]: osdmap e43: 6 total, 6 up, 6 in 2023-10-06T21:15:26.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:25 smithi142 ceph-mon[102951]: osdmap e43: 6 total, 6 up, 6 in 2023-10-06T21:15:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:26 smithi031 ceph-mon[98539]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:26 smithi146 ceph-mon[102809]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:26 smithi142 ceph-mon[102951]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:27 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]: dispatch 2023-10-06T21:15:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:27 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]': finished 2023-10-06T21:15:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:27 smithi031 ceph-mon[98539]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:15:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:27 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:28.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:27 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]: dispatch 2023-10-06T21:15:28.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:27 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]': finished 2023-10-06T21:15:28.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:27 smithi146 ceph-mon[102809]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:15:28.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:27 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:27 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]: dispatch 2023-10-06T21:15:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:27 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2137339669' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e"}]': finished 2023-10-06T21:15:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:27 smithi142 ceph-mon[102951]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:15:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:27 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:29.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:28 smithi031 ceph-mon[98539]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-06T21:15:29.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:28 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1362944569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:28 smithi146 ceph-mon[102809]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-06T21:15:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:28 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1362944569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:29.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:28 smithi142 ceph-mon[102951]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-06T21:15:29.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:28 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1362944569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:31.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:30 smithi031 ceph-mon[98539]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-06T21:15:31.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:30 smithi142 ceph-mon[102951]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-06T21:15:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:30 smithi146 ceph-mon[102809]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-06T21:15:32.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:32 smithi031 ceph-mon[98539]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:15:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:32 smithi142 ceph-mon[102951]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:15:33.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:32 smithi146 ceph-mon[102809]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:15:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:34 smithi031 ceph-mon[98539]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-06T21:15:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:15:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:34 smithi146 ceph-mon[102809]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-06T21:15:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:15:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:34 smithi142 ceph-mon[102951]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-06T21:15:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:15:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:35.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:35 smithi031 ceph-mon[98539]: Deploying daemon osd.6 on smithi031 2023-10-06T21:15:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:35 smithi146 ceph-mon[102809]: Deploying daemon osd.6 on smithi031 2023-10-06T21:15:36.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:35 smithi142 ceph-mon[102951]: Deploying daemon osd.6 on smithi031 2023-10-06T21:15:37.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:36 smithi031 ceph-mon[98539]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:15:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:36 smithi146 ceph-mon[102809]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:15:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:36 smithi142 ceph-mon[102951]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:15:39.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:38 smithi142 ceph-mon[102951]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:38 smithi031 ceph-mon[98539]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:39.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:38 smithi146 ceph-mon[102809]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:40.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:40 smithi031 ceph-mon[98539]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:41.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:40 smithi142 ceph-mon[102951]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:41.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:40 smithi146 ceph-mon[102809]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:42.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.275 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:43.199 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:43 smithi031 ceph-mon[98539]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:43 smithi146 ceph-mon[102809]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:43 smithi142 ceph-mon[102951]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:44.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.307+0000 7f28aaffd700 1 -- 172.21.15.31:0/2803783749 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28b402d350 con 0x7f28a00a1480 2023-10-06T21:15:44.311 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.309+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 msgr2=0x7f28a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.309+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 0x7f28a00a3940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b580 tx=0x7f28a4005ea0 comp rx=0 tx=0).stop 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.309+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 msgr2=0x7f28b4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.309+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28b000d980 tx=0x7f28b000dc90 comp rx=0 tx=0).stop 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 shutdown_connections 2023-10-06T21:15:44.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28a00a1480 0x7f28a00a3940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b10 0x7f28b40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28b4100f70 0x7f28b4074c50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 --2- 172.21.15.31:0/2803783749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b4102170 0x7f28b40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 >> 172.21.15.31:0/2803783749 conn(0x7f28b40fc4d0 msgr2=0x7f28b41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 shutdown_connections 2023-10-06T21:15:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:15:44.310+0000 7f28bc0f4700 1 -- 172.21.15.31:0/2803783749 wait complete. 2023-10-06T21:15:44.804 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-06T21:15:44.809 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2023-10-06T21:15:44.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:15:44.884 INFO:journalctl@ceph.osd.6.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:15:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.261 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:15:44 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:15:44.986+0000 7f9cf8df3640 -1 osd.6 0 log_to_monitors true 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:45.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:45.544 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:15:46.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:45 smithi031 ceph-mon[98539]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:15:46.110 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:45 smithi142 ceph-mon[102951]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:15:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:45 smithi146 ceph-mon[102809]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:15:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:46 smithi142 ceph-mon[102951]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:46 smithi142 ceph-mon[102951]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:15:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:46 smithi142 ceph-mon[102951]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:15:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:46 smithi142 ceph-mon[102951]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:15:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:47.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:15:46.906+0000 7f9cee46a700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T21:15:47.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:15:46.922+0000 7f9ce8a5c700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:15:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-mon[98539]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-mon[98539]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:15:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-mon[98539]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:15:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-mon[98539]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:15:47.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:46 smithi146 ceph-mon[102809]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:46 smithi146 ceph-mon[102809]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:15:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:46 smithi146 ceph-mon[102809]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:15:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:46 smithi146 ceph-mon[102809]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:15:47.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:47.506 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:15:47.506 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:15:47.506 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-10-06T21:15:47.506 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-10-06T21:15:47.507 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0450291 s, 233 MB/s 2023-10-06T21:15:47.507 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-10-06T21:15:48.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2023-10-06T21:15:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:47 smithi031 ceph-mon[98539]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:15:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:47 smithi031 ceph-mon[98539]: osdmap e46: 7 total, 6 up, 7 in 2023-10-06T21:15:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:47 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:47 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:47 smithi146 ceph-mon[102809]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:15:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:47 smithi146 ceph-mon[102809]: osdmap e46: 7 total, 6 up, 7 in 2023-10-06T21:15:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:47 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:47 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:47 smithi142 ceph-mon[102951]: from='osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:15:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:47 smithi142 ceph-mon[102951]: osdmap e46: 7 total, 6 up, 7 in 2023-10-06T21:15:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:47 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:47 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:48.742 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] boot 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:15:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:15:49.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:15:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] boot 2023-10-06T21:15:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:15:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:49.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: osd.6 [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] boot 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:49.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:15:50.228 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 -- 172.21.15.142:0/3952196743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0100f70 msgr2=0x7f04d0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:50.228 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 --2- 172.21.15.142:0/3952196743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0100f70 0x7f04d0101390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f04c00099c0 tx=0x7f04c0009cd0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 -- 172.21.15.142:0/3952196743 shutdown_connections 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 --2- 172.21.15.142:0/3952196743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0102b10 0x7f04d01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 --2- 172.21.15.142:0/3952196743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0100f70 0x7f04d0101390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 --2- 172.21.15.142:0/3952196743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 0x7f04d01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 -- 172.21.15.142:0/3952196743 >> 172.21.15.142:0/3952196743 conn(0x7f04d00fc4d0 msgr2=0x7f04d00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 -- 172.21.15.142:0/3952196743 shutdown_connections 2023-10-06T21:15:50.229 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.226+0000 7f04d61b0700 1 -- 172.21.15.142:0/3952196743 wait complete. 2023-10-06T21:15:50.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 Processor -- start 2023-10-06T21:15:50.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 -- start start 2023-10-06T21:15:50.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0100f70 0x7f04d0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:50.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 0x7f04d0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:50.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01a08e0 con 0x7f04d0102170 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01a0a20 con 0x7f04d0102b10 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.227+0000 7f04d61b0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01a0bb0 con 0x7f04d0100f70 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37874/0 (socket says 172.21.15.142:37874) 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 -- 172.21.15.142:0/2374816200 learned_addr learned my addr 172.21.15.142:0/2374816200 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04ceffd700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 0x7f04d0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cf7fe700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0100f70 0x7f04d0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 -- 172.21.15.142:0/2374816200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0100f70 msgr2=0x7f04d0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0100f70 0x7f04d0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 -- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 msgr2=0x7f04d0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 0x7f04d0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 -- 172.21.15.142:0/2374816200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00095f0 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.228+0000 7f04cffff700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007d70 tx=0x7f04c800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.229+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800ad40 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.229+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04c80128b0 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.229+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d01a0e90 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.229+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80115e0 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.229+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d01a1300 con 0x7f04d0102b10 2023-10-06T21:15:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.230+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d010c670 con 0x7f04d0102b10 2023-10-06T21:15:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.232+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f04c8012a20 con 0x7f04d0102b10 2023-10-06T21:15:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.233+0000 7f04ccff9700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 0x7f04b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.233+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f04c80cc740 con 0x7f04d0102b10 2023-10-06T21:15:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.234+0000 7f04cf7fe700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 0x7f04b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.234+0000 7f04cf7fe700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 0x7f04b80a3940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009990 tx=0x7f04c001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:50.238 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.236+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f04c809a4a0 con 0x7f04d0102b10 2023-10-06T21:15:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:49 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi031 2023-10-06T21:15:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:49 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 5757M 2023-10-06T21:15:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:49 smithi031 ceph-mon[98539]: osdmap e48: 7 total, 7 up, 7 in 2023-10-06T21:15:50.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:49 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi031 2023-10-06T21:15:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:49 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 5757M 2023-10-06T21:15:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:49 smithi146 ceph-mon[102809]: osdmap e48: 7 total, 7 up, 7 in 2023-10-06T21:15:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:49 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi031 2023-10-06T21:15:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:49 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 5757M 2023-10-06T21:15:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:49 smithi142 ceph-mon[102951]: osdmap e48: 7 total, 7 up, 7 in 2023-10-06T21:15:50.445 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:15:50.443+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f04d0108b10 con 0x7f04b80a1480 2023-10-06T21:15:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:50 smithi031 ceph-mon[98539]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:50 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:50 smithi146 ceph-mon[102809]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:50 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:50 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:50 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:50 smithi142 ceph-mon[102951]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:50 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:15:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:50 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:15:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:50 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:52.113 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:51 smithi142 ceph-mon[102951]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:51 smithi031 ceph-mon[98539]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:51 smithi146 ceph-mon[102809]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:15:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:52 smithi031 ceph-mon[98539]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:52 smithi146 ceph-mon[102809]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:52 smithi142 ceph-mon[102951]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:53 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/3483525709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:53 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:53 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]': finished 2023-10-06T21:15:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:53 smithi031 ceph-mon[98539]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:15:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:53 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:15:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:53 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/3483525709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:53 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:53 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]': finished 2023-10-06T21:15:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:53 smithi146 ceph-mon[102809]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:15:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:53 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:15:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:53 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/3483525709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:53 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]: dispatch 2023-10-06T21:15:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:53 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5705f243-b05f-4005-90f0-b62dc8b0ffbb"}]': finished 2023-10-06T21:15:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:53 smithi142 ceph-mon[102951]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:15:54.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:53 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:15:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:54 smithi031 ceph-mon[98539]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:54 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/983006540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:54 smithi146 ceph-mon[102809]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:54 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/983006540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:55.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:54 smithi142 ceph-mon[102951]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:55.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:54 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/983006540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:15:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:56 smithi031 ceph-mon[98539]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:56 smithi146 ceph-mon[102809]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:57.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:56 smithi142 ceph-mon[102951]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:59.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:15:58 smithi142 ceph-mon[102951]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:15:58 smithi031 ceph-mon[98539]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:15:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:15:58 smithi146 ceph-mon[102809]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:00 smithi142 ceph-mon[102951]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:16:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:00 smithi031 ceph-mon[98539]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:16:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:00 smithi146 ceph-mon[102809]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:16:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:02.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:01 smithi142 ceph-mon[102951]: Deploying daemon osd.7 on smithi142 2023-10-06T21:16:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:01 smithi031 ceph-mon[98539]: Deploying daemon osd.7 on smithi142 2023-10-06T21:16:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:01 smithi146 ceph-mon[102809]: Deploying daemon osd.7 on smithi142 2023-10-06T21:16:03.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:02 smithi142 ceph-mon[102951]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:03.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:02 smithi031 ceph-mon[98539]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:02 smithi146 ceph-mon[102809]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:05.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:04 smithi142 ceph-mon[102951]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:05.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:04 smithi031 ceph-mon[98539]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:05.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:04 smithi146 ceph-mon[102809]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:06 smithi142 ceph-mon[102951]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:06 smithi031 ceph-mon[98539]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:07.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:06 smithi146 ceph-mon[102809]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:08.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:08 smithi142 ceph-mon[102951]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:08 smithi031 ceph-mon[98539]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:09.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:08 smithi146 ceph-mon[102809]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:10.123 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.120+0000 7f04ccff9700 1 -- 172.21.15.142:0/2374816200 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f04d0108b10 con 0x7f04b80a1480 2023-10-06T21:16:10.123 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.122+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 msgr2=0x7f04b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.122+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 0x7f04b80a3940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009990 tx=0x7f04c001a040 comp rx=0 tx=0).stop 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.122+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 msgr2=0x7f04d01a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.122+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007d70 tx=0x7f04c800a340 comp rx=0 tx=0).stop 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 shutdown_connections 2023-10-06T21:16:10.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f04b80a1480 0x7f04b80a3940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d0102b10 0x7f04d01a0310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d0100f70 0x7f04d0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 --2- 172.21.15.142:0/2374816200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d0102170 0x7f04d0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 >> 172.21.15.142:0/2374816200 conn(0x7f04d00fc4d0 msgr2=0x7f04d00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 shutdown_connections 2023-10-06T21:16:10.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:16:10.123+0000 7f04d61b0700 1 -- 172.21.15.142:0/2374816200 wait complete. 2023-10-06T21:16:10.655 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-06T21:16:10.659 INFO:tasks.cephadm:Deploying osd.8 on smithi146 with /dev/vg_nvme/lv_2... 2023-10-06T21:16:10.659 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:16:10.734 INFO:journalctl@ceph.osd.7.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:16:10.853 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:10.853 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:10.853 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:10.854 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:11.151 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:16:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:16:10.849+0000 7f9339dae640 -1 osd.7 0 log_to_monitors true 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:11.395 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:16:11.924 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:11 smithi146 ceph-mon[102809]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:11.925 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:11 smithi146 ceph-mon[102809]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:12.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:11 smithi142 ceph-mon[102951]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:12.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:11 smithi142 ceph-mon[102951]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:11 smithi031 ceph-mon[98539]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:11 smithi031 ceph-mon[98539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:16:13.151 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:16:12.851+0000 7f932f425700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T21:16:13.151 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:16:12.863+0000 7f9329216700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:12 smithi142 ceph-mon[102951]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.248 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:16:13.249 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:16:13.249 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2023-10-06T21:16:13.249 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2023-10-06T21:16:13.249 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0385177 s, 272 MB/s 2023-10-06T21:16:13.249 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:12 smithi031 ceph-mon[98539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: from='osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.286 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:12 smithi146 ceph-mon[102809]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:16:13.767 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2023-10-06T21:16:14.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: osdmap e51: 8 total, 7 up, 8 in 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] boot 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:16:14.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:13 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:16:14.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: osdmap e51: 8 total, 7 up, 8 in 2023-10-06T21:16:14.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] boot 2023-10-06T21:16:14.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:16:14.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:13 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: osdmap e51: 8 total, 7 up, 8 in 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: osd.7 [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] boot 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:16:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:13 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:16:14.507 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi142 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:14 smithi146 ceph-mon[102809]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:16:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:16:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:16:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi142 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:14 smithi142 ceph-mon[102951]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:16:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:16:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:16:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:16:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi142 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:15.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:15.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:15.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:14 smithi031 ceph-mon[98539]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:16:15.986 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 -- 172.21.15.146:0/729805215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248073910 msgr2=0x7f9248073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:15.986 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 --2- 172.21.15.146:0/729805215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248073910 0x7f9248073d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f923c009990 tx=0x7f923c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.986 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 -- 172.21.15.146:0/729805215 shutdown_connections 2023-10-06T21:16:15.986 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 --2- 172.21.15.146:0/729805215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248108e40 0x7f924810b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 --2- 172.21.15.146:0/729805215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 0x7f9248073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 --2- 172.21.15.146:0/729805215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248073910 0x7f9248073d90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 -- 172.21.15.146:0/729805215 >> 172.21.15.146:0/729805215 conn(0x7f92480fc4d0 msgr2=0x7f92480fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:15.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 -- 172.21.15.146:0/729805215 shutdown_connections 2023-10-06T21:16:15.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.983+0000 7f924e59b700 1 -- 172.21.15.146:0/729805215 wait complete. 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 Processor -- start 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 -- start start 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248073910 0x7f9248198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 0x7f92481992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a08e0 con 0x7f9248073910 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a0a20 con 0x7f9248074ee0 2023-10-06T21:16:15.989 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.984+0000 7f924e59b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a0be0 con 0x7f9248108e40 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59116/0 (socket says 172.21.15.146:59116) 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 -- 172.21.15.146:0/2484458843 learned_addr learned my addr 172.21.15.146:0/2484458843 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f92477fe700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 0x7f92481992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f9247fff700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248073910 0x7f9248198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 -- 172.21.15.146:0/2484458843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 msgr2=0x7f92481992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 0x7f92481992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 -- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248073910 msgr2=0x7f9248198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248073910 0x7f9248198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 -- 172.21.15.146:0/2484458843 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0095f0 con 0x7f9248108e40 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f924cb38700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9240007d70 tx=0x7f924000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:15.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.985+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924000ad40 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.986+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92400128b0 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.986+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92481a0ec0 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.986+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92400116b0 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.986+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92481a12d0 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.987+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92481074f0 con 0x7f9248108e40 2023-10-06T21:16:15.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.989+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9240012a20 con 0x7f9248108e40 2023-10-06T21:16:15.994 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.991+0000 7f92457fa700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 0x7f92300a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:15.994 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.992+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f92400cc8a0 con 0x7f9248108e40 2023-10-06T21:16:15.995 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.992+0000 7f9247fff700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 0x7f92300a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:15.995 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.992+0000 7f9247fff700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 0x7f92300a39c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9238005950 tx=0x7f92380058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:15.997 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:15.995+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f92400969a0 con 0x7f9248108e40 2023-10-06T21:16:16.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:15 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 5755M 2023-10-06T21:16:16.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:15 smithi146 ceph-mon[102809]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T21:16:16.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:15 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 5755M 2023-10-06T21:16:16.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:15 smithi142 ceph-mon[102951]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T21:16:16.204 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:16.201+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f924810e890 con 0x7f92300a1500 2023-10-06T21:16:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:15 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 5755M 2023-10-06T21:16:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:15 smithi031 ceph-mon[98539]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T21:16:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:16 smithi142 ceph-mon[102951]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:16 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:17.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:16 smithi142 ceph-mon[102951]: osdmap e55: 8 total, 8 up, 8 in 2023-10-06T21:16:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:16 smithi031 ceph-mon[98539]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:16 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:16 smithi031 ceph-mon[98539]: osdmap e55: 8 total, 8 up, 8 in 2023-10-06T21:16:17.273 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:16 smithi146 ceph-mon[102809]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:17.273 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:17.273 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:17.273 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:16 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:17.273 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:16 smithi146 ceph-mon[102809]: osdmap e55: 8 total, 8 up, 8 in 2023-10-06T21:16:18.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:17 smithi142 ceph-mon[102951]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:18.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:17 smithi142 ceph-mon[102951]: osdmap e56: 8 total, 8 up, 8 in 2023-10-06T21:16:18.214 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:17 smithi146 ceph-mon[102809]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:18.214 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:17 smithi146 ceph-mon[102809]: osdmap e56: 8 total, 8 up, 8 in 2023-10-06T21:16:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:17 smithi031 ceph-mon[98539]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:17 smithi031 ceph-mon[98539]: osdmap e56: 8 total, 8 up, 8 in 2023-10-06T21:16:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:18 smithi142 ceph-mon[102951]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:18 smithi031 ceph-mon[98539]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:18 smithi146 ceph-mon[102809]: pgmap v192: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:19 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/2777184064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:19 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:19 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]': finished 2023-10-06T21:16:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:19 smithi142 ceph-mon[102951]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:16:20.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:19 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:19 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/2777184064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:19 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:19 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]': finished 2023-10-06T21:16:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:19 smithi031 ceph-mon[98539]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:16:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:19 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:19 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/2777184064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:19 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]: dispatch 2023-10-06T21:16:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:19 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cc710f2-d725-47da-9841-e1c75d64bd0e"}]': finished 2023-10-06T21:16:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:19 smithi146 ceph-mon[102809]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:16:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:19 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:21.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:20 smithi142 ceph-mon[102951]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:21.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:20 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/1808128105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:20 smithi031 ceph-mon[98539]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:20 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/1808128105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:21.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:20 smithi146 ceph-mon[102809]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:20 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/1808128105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:23.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:22 smithi142 ceph-mon[102951]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:23.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:22 smithi146 ceph-mon[102809]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:23.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:22 smithi031 ceph-mon[98539]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:24.880 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:24 smithi146 ceph-mon[102809]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:24 smithi031 ceph-mon[98539]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:24 smithi142 ceph-mon[102951]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:26 smithi142 ceph-mon[102951]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:16:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:27.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:26 smithi031 ceph-mon[98539]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:27.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:16:27.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:26 smithi146 ceph-mon[102809]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:16:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:27.939 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:27 smithi146 ceph-mon[102809]: Deploying daemon osd.8 on smithi146 2023-10-06T21:16:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:27 smithi142 ceph-mon[102951]: Deploying daemon osd.8 on smithi146 2023-10-06T21:16:28.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:27 smithi031 ceph-mon[98539]: Deploying daemon osd.8 on smithi146 2023-10-06T21:16:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:28 smithi146 ceph-mon[102809]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:29.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:28 smithi142 ceph-mon[102951]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:29.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:28 smithi031 ceph-mon[98539]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:30 smithi146 ceph-mon[102809]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:31.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:30 smithi142 ceph-mon[102951]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:30 smithi031 ceph-mon[98539]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:32 smithi142 ceph-mon[102951]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:32 smithi031 ceph-mon[98539]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:32 smithi146 ceph-mon[102809]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:34 smithi031 ceph-mon[98539]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:35.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:34 smithi146 ceph-mon[102809]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:34 smithi142 ceph-mon[102951]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:35.929 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.926+0000 7f92457fa700 1 -- 172.21.15.146:0/2484458843 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f924810e890 con 0x7f92300a1500 2023-10-06T21:16:35.929 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 8 on host 'smithi146' 2023-10-06T21:16:35.931 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 msgr2=0x7f92300a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:35.931 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 0x7f92300a39c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9238005950 tx=0x7f92380058e0 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 msgr2=0x7f92481a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9240007d70 tx=0x7f924000a340 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 shutdown_connections 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f92300a1500 0x7f92300a39c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9248108e40 0x7f92481a0310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9248073910 0x7f9248198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 --2- 172.21.15.146:0/2484458843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9248074ee0 0x7f92481992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.929+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 >> 172.21.15.146:0/2484458843 conn(0x7f92480fc4d0 msgr2=0x7f92480fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.930+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 shutdown_connections 2023-10-06T21:16:35.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:16:35.930+0000 7f924e59b700 1 -- 172.21.15.146:0/2484458843 wait complete. 2023-10-06T21:16:36.323 DEBUG:teuthology.orchestra.run.smithi146:osd.8> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-06T21:16:36.328 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2023-10-06T21:16:36.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:16:36.398 INFO:journalctl@ceph.osd.8.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:16:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:36.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:36 smithi031 ceph-mon[98539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:36.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-mon[102809]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:36.800 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:16:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:16:36.519+0000 7fb43724f640 -1 osd.8 0 log_to_monitors true 2023-10-06T21:16:36.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:36.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:36 smithi142 ceph-mon[102951]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:16:37.079 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:16:38.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:37 smithi031 ceph-mon[98539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:16:38.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:37 smithi031 ceph-mon[98539]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:37 smithi031 ceph-mon[98539]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:16:38.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:38.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:37 smithi031 ceph-mon[98539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:16:37.930+0000 7fb42c8c6700 -1 osd.8 0 waiting for initial osdmap 2023-10-06T21:16:38.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:16:37.946+0000 7fb4246b3700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:16:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-mon[102809]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:16:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-mon[102809]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-mon[102809]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:16:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:37 smithi146 ceph-mon[102809]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:37 smithi142 ceph-mon[102951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:16:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:37 smithi142 ceph-mon[102951]: from='osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:37 smithi142 ceph-mon[102951]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:16:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:37 smithi142 ceph-mon[102951]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:16:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:16:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:16:38.997 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-10-06T21:16:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-10-06T21:16:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0388424 s, 270 MB/s 2023-10-06T21:16:38.997 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 2023-10-06T21:16:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: osdmap e59: 9 total, 8 up, 9 in 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] boot 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:16:39.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:38 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:16:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: osdmap e59: 9 total, 8 up, 9 in 2023-10-06T21:16:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] boot 2023-10-06T21:16:39.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:16:39.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:38 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:16:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:16:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: osdmap e59: 9 total, 8 up, 9 in 2023-10-06T21:16:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: osd.8 [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] boot 2023-10-06T21:16:39.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:16:39.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:38 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:16:39.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2023-10-06T21:16:40.228 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:40.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:40.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:39 smithi031 ceph-mon[98539]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:39 smithi146 ceph-mon[102809]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:16:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:16:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:16:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:16:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:39 smithi142 ceph-mon[102951]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:16:41.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:40 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi146 2023-10-06T21:16:41.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:40 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 5755M 2023-10-06T21:16:41.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:40 smithi031 ceph-mon[98539]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:41.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:40 smithi031 ceph-mon[98539]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:16:41.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:40 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi146 2023-10-06T21:16:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:40 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 5755M 2023-10-06T21:16:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:40 smithi146 ceph-mon[102809]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:40 smithi146 ceph-mon[102809]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:16:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:40 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi146 2023-10-06T21:16:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:40 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 5755M 2023-10-06T21:16:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:40 smithi142 ceph-mon[102951]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:40 smithi142 ceph-mon[102951]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:16:41.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 -- 172.21.15.31:0/1759712338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102b10 msgr2=0x7f7a5c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:41.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 --2- 172.21.15.31:0/1759712338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102b10 0x7f7a5c1093c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400b0d0 tx=0x7f7a5400b3e0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 -- 172.21.15.31:0/1759712338 shutdown_connections 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 --2- 172.21.15.31:0/1759712338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102b10 0x7f7a5c1093c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 --2- 172.21.15.31:0/1759712338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100f70 0x7f7a5c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 --2- 172.21.15.31:0/1759712338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102170 0x7f7a5c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.819+0000 7f7a6372b700 1 -- 172.21.15.31:0/1759712338 >> 172.21.15.31:0/1759712338 conn(0x7f7a5c0fc4d0 msgr2=0x7f7a5c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.820+0000 7f7a6372b700 1 -- 172.21.15.31:0/1759712338 shutdown_connections 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.820+0000 7f7a6372b700 1 -- 172.21.15.31:0/1759712338 wait complete. 2023-10-06T21:16:41.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.820+0000 7f7a6372b700 1 Processor -- start 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 -- start start 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c102170 0x7f7a5c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 0x7f7a5c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c073f60 con 0x7f7a5c100f70 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0740d0 con 0x7f7a5c102b10 2023-10-06T21:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a6372b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a0d10 con 0x7f7a5c102170 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a614c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a614c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33816/0 (socket says 172.21.15.31:33816) 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a614c7700 1 -- 172.21.15.31:0/745259722 learned_addr learned my addr 172.21.15.31:0/745259722 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a61cc8700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 0x7f7a5c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.821+0000 7f7a60cc6700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c102170 0x7f7a5c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 -- 172.21.15.31:0/745259722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c102170 msgr2=0x7f7a5c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c102170 0x7f7a5c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 -- 172.21.15.31:0/745259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 msgr2=0x7f7a5c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 0x7f7a5c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 -- 172.21.15.31:0/745259722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54009d00 con 0x7f7a5c100f70 2023-10-06T21:16:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a61cc8700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 0x7f7a5c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:41.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a614c7700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d980 tx=0x7f7a5800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800bb60 con 0x7f7a5c100f70 2023-10-06T21:16:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a58007620 con 0x7f7a5c100f70 2023-10-06T21:16:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c1a0ff0 con 0x7f7a5c100f70 2023-10-06T21:16:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800f3c0 con 0x7f7a5c100f70 2023-10-06T21:16:41.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.822+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c1a1410 con 0x7f7a5c100f70 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.823+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c1085b0 con 0x7f7a5c100f70 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.825+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7a58010460 con 0x7f7a5c100f70 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.826+0000 7f7a527fc700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 0x7f7a480a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.827+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f7a580cc690 con 0x7f7a5c100f70 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.827+0000 7f7a60cc6700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 0x7f7a480a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:41.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.827+0000 7f7a60cc6700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 0x7f7a480a39c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005950 tx=0x7f7a4c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:41.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:41.831+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7a580965e0 con 0x7f7a5c100f70 2023-10-06T21:16:42.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:16:42.054+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a5c02d2e0 con 0x7f7a480a1500 2023-10-06T21:16:43.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:42 smithi031 ceph-mon[98539]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:43.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:42 smithi031 ceph-mon[98539]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:16:43.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:43.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:43.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:42 smithi146 ceph-mon[102809]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:42 smithi146 ceph-mon[102809]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:16:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:42 smithi142 ceph-mon[102951]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:42 smithi142 ceph-mon[102951]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:16:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:16:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:16:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:44.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:43 smithi031 ceph-mon[98539]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:44.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:43 smithi031 ceph-mon[98539]: osdmap e64: 9 total, 9 up, 9 in 2023-10-06T21:16:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:43 smithi146 ceph-mon[102809]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:43 smithi146 ceph-mon[102809]: osdmap e64: 9 total, 9 up, 9 in 2023-10-06T21:16:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:43 smithi142 ceph-mon[102951]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:16:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:43 smithi142 ceph-mon[102951]: osdmap e64: 9 total, 9 up, 9 in 2023-10-06T21:16:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:44 smithi031 ceph-mon[98539]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:44 smithi146 ceph-mon[102809]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:45.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:44 smithi142 ceph-mon[102951]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]: dispatch 2023-10-06T21:16:46.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]': finished 2023-10-06T21:16:46.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:45 smithi031 ceph-mon[98539]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:16:46.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:16:46.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1979712207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]: dispatch 2023-10-06T21:16:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]': finished 2023-10-06T21:16:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:45 smithi146 ceph-mon[102809]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:16:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:16:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1979712207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]: dispatch 2023-10-06T21:16:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2685904683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50342566-6d06-4bdd-9336-075df7afc340"}]': finished 2023-10-06T21:16:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:45 smithi142 ceph-mon[102951]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:16:46.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:16:46.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1979712207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:16:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:46 smithi031 ceph-mon[98539]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-10-06T21:16:47.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:46 smithi146 ceph-mon[102809]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-10-06T21:16:47.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:46 smithi142 ceph-mon[102951]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-10-06T21:16:49.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:48 smithi031 ceph-mon[98539]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:16:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:48 smithi146 ceph-mon[102809]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:16:49.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:48 smithi142 ceph-mon[102951]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:16:51.223 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:50 smithi031 ceph-mon[98539]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-06T21:16:51.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:50 smithi146 ceph-mon[102809]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-06T21:16:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:50 smithi142 ceph-mon[102951]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-06T21:16:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:52 smithi031 ceph-mon[98539]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-06T21:16:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:16:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:52 smithi146 ceph-mon[102809]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-06T21:16:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:16:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:52 smithi142 ceph-mon[102951]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-06T21:16:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:16:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:53 smithi031 ceph-mon[98539]: Deploying daemon osd.9 on smithi031 2023-10-06T21:16:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:53 smithi146 ceph-mon[102809]: Deploying daemon osd.9 on smithi031 2023-10-06T21:16:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:53 smithi142 ceph-mon[102951]: Deploying daemon osd.9 on smithi031 2023-10-06T21:16:55.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:54 smithi031 ceph-mon[98539]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:16:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:54 smithi146 ceph-mon[102809]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:16:55.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:54 smithi142 ceph-mon[102951]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:16:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:56 smithi031 ceph-mon[98539]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:57.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:56 smithi146 ceph-mon[102809]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:57.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:56 smithi142 ceph-mon[102951]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:16:58 smithi031 ceph-mon[98539]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:59.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:16:58 smithi146 ceph-mon[102809]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:16:59.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:16:58 smithi142 ceph-mon[102951]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:00 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:00.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:00 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:00 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:01 smithi146 ceph-mon[102809]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:01.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:01 smithi142 ceph-mon[102951]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:01.448 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:01 smithi031 ceph-mon[98539]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:02.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.615+0000 7f7a527fc700 1 -- 172.21.15.31:0/745259722 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7a5c02d2e0 con 0x7f7a480a1500 2023-10-06T21:17:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 msgr2=0x7f7a480a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 0x7f7a480a39c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005950 tx=0x7f7a4c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 msgr2=0x7f7a5c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d980 tx=0x7f7a5800dd40 comp rx=0 tx=0).stop 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 shutdown_connections 2023-10-06T21:17:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a480a1500 0x7f7a480a39c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102b10 0x7f7a5c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c100f70 0x7f7a5c074c50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 --2- 172.21.15.31:0/745259722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c102170 0x7f7a5c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 >> 172.21.15.31:0/745259722 conn(0x7f7a5c0fc4d0 msgr2=0x7f7a5c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 shutdown_connections 2023-10-06T21:17:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:02.618+0000 7f7a6372b700 1 -- 172.21.15.31:0/745259722 wait complete. 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:03 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:03.167 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-06T21:17:03.172 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2023-10-06T21:17:03.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:17:03.244 INFO:journalctl@ceph.osd.9.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:03 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:03 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:03.901 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:17:04.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:04 smithi031 ceph-mon[98539]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:17:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:04 smithi146 ceph-mon[102809]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:17:04.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:04 smithi142 ceph-mon[102951]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:17:05.143 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:05 smithi142 ceph-mon[102951]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:05.143 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:05 smithi142 ceph-mon[102951]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:17:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:05 smithi142 ceph-mon[102951]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:17:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:05 smithi142 ceph-mon[102951]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:17:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:05 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-mon[98539]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-mon[98539]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:17:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-mon[98539]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:17:05.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-mon[98539]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:17:05.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:05.261 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:17:05.028+0000 7f24643f2700 -1 osd.9 0 waiting for initial osdmap 2023-10-06T21:17:05.261 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:17:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:17:05.045+0000 7f24601e7700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:17:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:05 smithi146 ceph-mon[102809]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:05 smithi146 ceph-mon[102809]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:17:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:05 smithi146 ceph-mon[102809]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:17:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:05 smithi146 ceph-mon[102809]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-10-06T21:17:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:05 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:05.860 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:17:05.861 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:17:05.861 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-10-06T21:17:05.861 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-10-06T21:17:05.861 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0302688 s, 346 MB/s 2023-10-06T21:17:05.861 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 2023-10-06T21:17:05.862 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-10-06T21:17:06.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:06 smithi142 ceph-mon[102951]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:17:06.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:06 smithi142 ceph-mon[102951]: osdmap e67: 10 total, 9 up, 10 in 2023-10-06T21:17:06.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:06.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:06 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:06.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2023-10-06T21:17:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:06 smithi146 ceph-mon[102809]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:17:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:06 smithi146 ceph-mon[102809]: osdmap e67: 10 total, 9 up, 10 in 2023-10-06T21:17:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:06 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:06.320 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:06 smithi031 ceph-mon[98539]: from='osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-10-06T21:17:06.320 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:06 smithi031 ceph-mon[98539]: osdmap e67: 10 total, 9 up, 10 in 2023-10-06T21:17:06.320 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:06.321 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:06 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:17:07.296 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:17:07.296 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:17:07.296 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:07.296 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] boot 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:07.298 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:07.298 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:07 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] boot 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:07.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:07 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:17:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:17:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:17:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: osd.9 [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] boot 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:07.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:07.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:07.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:07 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:08.101 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:08 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi031 2023-10-06T21:17:08.101 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:08 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 4318M 2023-10-06T21:17:08.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:08 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi031 2023-10-06T21:17:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:08 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 4318M 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 -- 172.21.15.142:0/4145432605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b10 msgr2=0x7fafcc1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 --2- 172.21.15.142:0/4145432605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b10 0x7fafcc1093c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b0d0 tx=0x7fafc400b3e0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 -- 172.21.15.142:0/4145432605 shutdown_connections 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 --2- 172.21.15.142:0/4145432605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b10 0x7fafcc1093c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 --2- 172.21.15.142:0/4145432605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 0x7fafcc101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 --2- 172.21.15.142:0/4145432605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102170 0x7fafcc1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.344 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.340+0000 7fafd24e9700 1 -- 172.21.15.142:0/4145432605 >> 172.21.15.142:0/4145432605 conn(0x7fafcc0fc4d0 msgr2=0x7fafcc0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:08.345 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.341+0000 7fafd24e9700 1 -- 172.21.15.142:0/4145432605 shutdown_connections 2023-10-06T21:17:08.345 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.341+0000 7fafd24e9700 1 -- 172.21.15.142:0/4145432605 wait complete. 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 Processor -- start 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 -- start start 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 0x7fafcc198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102b10 0x7fafcc1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a09d0 con 0x7fafcc100f70 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a0b10 con 0x7fafcc102170 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafd24e9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a0ca0 con 0x7fafcc102b10 2023-10-06T21:17:08.347 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58662/0 (socket says 172.21.15.142:58662) 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafcb7fe700 1 -- 172.21.15.142:0/249966283 learned_addr learned my addr 172.21.15.142:0/249966283 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.342+0000 7fafcbfff700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 0x7fafcc198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafd0a86700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102b10 0x7fafcc1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 -- 172.21.15.142:0/249966283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102b10 msgr2=0x7fafcc1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102b10 0x7fafcc1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 -- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 msgr2=0x7fafcc198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 0x7fafcc198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 -- 172.21.15.142:0/249966283 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc4009d00 con 0x7fafcc102170 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafcb7fe700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fafc000d950 tx=0x7fafc000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:08.348 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0009940 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc0010460 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc1a0f80 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.343+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000f620 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.344+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1a13f0 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.345+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc193420 con 0x7fafcc102170 2023-10-06T21:17:08.349 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.347+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fafc00105d0 con 0x7fafcc102170 2023-10-06T21:17:08.351 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.348+0000 7fafc97fa700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 0x7fafb40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:08.351 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.348+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fafc00cc360 con 0x7fafcc102170 2023-10-06T21:17:08.351 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.348+0000 7fafcbfff700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 0x7fafb40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:08.352 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.349+0000 7fafcbfff700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 0x7fafb40a3940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fafcc101fd0 tx=0x7fafbc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:08.353 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.350+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fafc0096120 con 0x7fafcc102170 2023-10-06T21:17:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:08 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi031 2023-10-06T21:17:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:08 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 4318M 2023-10-06T21:17:08.564 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:08.560+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fafcc02d2f0 con 0x7fafb40a1480 2023-10-06T21:17:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: osdmap e69: 10 total, 10 up, 10 in 2023-10-06T21:17:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:09.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:09 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: osdmap e69: 10 total, 10 up, 10 in 2023-10-06T21:17:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:09.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:09.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:09.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:09 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: osdmap e69: 10 total, 10 up, 10 in 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:09 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:10.200 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:10 smithi142 ceph-mon[102951]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:10 smithi031 ceph-mon[98539]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:10 smithi146 ceph-mon[102809]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:11 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/1680890092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:11 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:11 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]': finished 2023-10-06T21:17:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:11 smithi146 ceph-mon[102809]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:17:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:11 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:11.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:11 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/1680890092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:11.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:11 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:11.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:11 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]': finished 2023-10-06T21:17:11.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:11 smithi142 ceph-mon[102951]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:17:11.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:11 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:11 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/1680890092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:11 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]: dispatch 2023-10-06T21:17:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:11 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d033e7e5-3d15-45be-8b09-0fb8f7187f9e"}]': finished 2023-10-06T21:17:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:11 smithi031 ceph-mon[98539]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:17:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:11 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:12 smithi146 ceph-mon[102809]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:12 smithi146 ceph-mon[102809]: from='client.? 172.21.15.142:0/2701313073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:12.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:12 smithi142 ceph-mon[102951]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:12.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:12 smithi142 ceph-mon[102951]: from='client.? 172.21.15.142:0/2701313073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:13.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:12 smithi031 ceph-mon[98539]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:13.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:12 smithi031 ceph-mon[98539]: from='client.? 172.21.15.142:0/2701313073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:14 smithi031 ceph-mon[98539]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:15.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:14 smithi146 ceph-mon[102809]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:14 smithi142 ceph-mon[102951]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:16 smithi142 ceph-mon[102951]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:16 smithi031 ceph-mon[98539]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:16 smithi146 ceph-mon[102809]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:18 smithi142 ceph-mon[102951]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:19.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:18 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:17:19.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:18 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:18 smithi031 ceph-mon[98539]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:17:19.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:18 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:18 smithi146 ceph-mon[102809]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:17:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:18 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:19.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:19 smithi142 ceph-mon[102951]: Deploying daemon osd.10 on smithi142 2023-10-06T21:17:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:19 smithi031 ceph-mon[98539]: Deploying daemon osd.10 on smithi142 2023-10-06T21:17:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:19 smithi146 ceph-mon[102809]: Deploying daemon osd.10 on smithi142 2023-10-06T21:17:21.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:20 smithi142 ceph-mon[102951]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:20 smithi031 ceph-mon[98539]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:20 smithi146 ceph-mon[102809]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:23.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:22 smithi142 ceph-mon[102951]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:23.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:22 smithi031 ceph-mon[98539]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:23.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:22 smithi146 ceph-mon[102809]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:24 smithi031 ceph-mon[98539]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:25.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:24 smithi146 ceph-mon[102809]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:25.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:24 smithi142 ceph-mon[102951]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:25.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:25.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:26.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:26 smithi142 ceph-mon[102951]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:27.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:26 smithi031 ceph-mon[98539]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:27.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:26 smithi146 ceph-mon[102809]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:28.188 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.185+0000 7fafc97fa700 1 -- 172.21.15.142:0/249966283 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fafcc02d2f0 con 0x7fafb40a1480 2023-10-06T21:17:28.188 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2023-10-06T21:17:28.190 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 msgr2=0x7fafb40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:28.190 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 0x7fafb40a3940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fafcc101fd0 tx=0x7fafbc009260 comp rx=0 tx=0).stop 2023-10-06T21:17:28.190 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 msgr2=0x7fafcc199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:28.190 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fafc000d950 tx=0x7fafc000dc60 comp rx=0 tx=0).stop 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 shutdown_connections 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafb40a1480 0x7fafb40a3940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc102b10 0x7fafcc1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafcc100f70 0x7fafcc198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 --2- 172.21.15.142:0/249966283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102170 0x7fafcc199380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 >> 172.21.15.142:0/249966283 conn(0x7fafcc0fc4d0 msgr2=0x7fafcc107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 shutdown_connections 2023-10-06T21:17:28.191 INFO:teuthology.orchestra.run.smithi142.stderr:2023-10-06T21:17:28.188+0000 7fafd24e9700 1 -- 172.21.15.142:0/249966283 wait complete. 2023-10-06T21:17:28.626 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-06T21:17:28.631 INFO:tasks.cephadm:Deploying osd.11 on smithi146 with /dev/vg_nvme/lv_1... 2023-10-06T21:17:28.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:17:28.708 INFO:journalctl@ceph.osd.10.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:17:28.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:28.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:28.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:28.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:28.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:28.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:28.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:28.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:28.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:28.904 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:17:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:17:28.756+0000 7f5b2896d640 -1 osd.10 0 log_to_monitors true 2023-10-06T21:17:29.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:28 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:29.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:28 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:29.375 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:17:29.920 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:29 smithi146 ceph-mon[102809]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:29.920 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:29 smithi146 ceph-mon[102809]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:30.117 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:29 smithi142 ceph-mon[102951]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:30.117 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:29 smithi142 ceph-mon[102951]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:29 smithi031 ceph-mon[98539]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:29 smithi031 ceph-mon[98539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:30 smithi146 ceph-mon[102809]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:31.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:17:30.787+0000 7f5b1dfe4700 -1 osd.10 0 waiting for initial osdmap 2023-10-06T21:17:31.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:17:30.808+0000 7f5b195d8700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:30 smithi142 ceph-mon[102951]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: from='osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-10-06T21:17:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:30 smithi031 ceph-mon[98539]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:31.318 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:17:31.318 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:17:31.319 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2023-10-06T21:17:31.319 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2023-10-06T21:17:31.319 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0296117 s, 354 MB/s 2023-10-06T21:17:31.319 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2023-10-06T21:17:31.850 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: osdmap e72: 11 total, 10 up, 11 in 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] boot 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:17:32.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:31 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:17:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:17:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: osdmap e72: 11 total, 10 up, 11 in 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] boot 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:17:32.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:31 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: osdmap e72: 11 total, 10 up, 11 in 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: osd.10 [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] boot 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:17:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:31 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:17:32.586 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:17:33.161 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:33.161 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi142 2023-10-06T21:17:33.161 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 4316M 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:33.162 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.163 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:33 smithi146 ceph-mon[102809]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:17:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi142 2023-10-06T21:17:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 4316M 2023-10-06T21:17:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:33.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:33.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:33 smithi031 ceph-mon[98539]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:17:33.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi142 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 4316M 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:33.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:33.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:33.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:33 smithi142 ceph-mon[102951]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:17:34.049 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 -- 172.21.15.146:0/1480242103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102170 msgr2=0x7fd41c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 --2- 172.21.15.146:0/1480242103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102170 0x7fd41c1025d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009990 tx=0x7fd40c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 -- 172.21.15.146:0/1480242103 shutdown_connections 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 --2- 172.21.15.146:0/1480242103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102b10 0x7fd41c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 --2- 172.21.15.146:0/1480242103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 --2- 172.21.15.146:0/1480242103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102170 0x7fd41c1025d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.050 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 -- 172.21.15.146:0/1480242103 >> 172.21.15.146:0/1480242103 conn(0x7fd41c0fc4d0 msgr2=0x7fd41c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:34.051 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 -- 172.21.15.146:0/1480242103 shutdown_connections 2023-10-06T21:17:34.051 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.047+0000 7fd424498700 1 -- 172.21.15.146:0/1480242103 wait complete. 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.048+0000 7fd424498700 1 Processor -- start 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.048+0000 7fd424498700 1 -- start start 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 0x7fd41c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19e7a0 con 0x7fd41c100f70 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19e8e0 con 0x7fd41c102170 2023-10-06T21:17:34.053 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd424498700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19eaa0 con 0x7fd41c102b10 2023-10-06T21:17:34.054 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:34.054 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:48910/0 (socket says 172.21.15.146:48910) 2023-10-06T21:17:34.054 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 -- 172.21.15.146:0/3549270601 learned_addr learned my addr 172.21.15.146:0/3549270601 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422234700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd421a33700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 0x7fd41c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 -- 172.21.15.146:0/3549270601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 msgr2=0x7fd41c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 0x7fd41c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 -- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 msgr2=0x7fd41c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:34.056 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422a35700 1 -- 172.21.15.146:0/3549270601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0095f0 con 0x7fd41c102b10 2023-10-06T21:17:34.056 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd421a33700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 0x7fd41c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:17:34.057 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.049+0000 7fd422234700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:17:34.057 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd422a35700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ef30 tx=0x7fd41400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:34.057 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414018070 con 0x7fd41c102b10 2023-10-06T21:17:34.057 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd41400f040 con 0x7fd41c102b10 2023-10-06T21:17:34.058 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c19ed20 con 0x7fd41c102b10 2023-10-06T21:17:34.058 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414003d00 con 0x7fd41c102b10 2023-10-06T21:17:34.058 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.050+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c19f170 con 0x7fd41c102b10 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.051+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c108610 con 0x7fd41c102b10 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.053+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd41400caa0 con 0x7fd41c102b10 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.055+0000 7fd4137fe700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.055+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fd4140cce50 con 0x7fd41c102b10 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.055+0000 7fd422234700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.056+0000 7fd422234700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 0x7fd4080a3940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd41c198130 tx=0x7fd418008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.057+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd414096ad0 con 0x7fd41c102b10 2023-10-06T21:17:34.266 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:34.264+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd41c10c710 con 0x7fd4080a1480 2023-10-06T21:17:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:34 smithi142 ceph-mon[102951]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:34 smithi142 ceph-mon[102951]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:17:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:35.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:34 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:34 smithi031 ceph-mon[98539]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:34 smithi031 ceph-mon[98539]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:17:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:34 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:34 smithi146 ceph-mon[102809]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:34 smithi146 ceph-mon[102809]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:17:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:17:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:17:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:34 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:36.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:35 smithi146 ceph-mon[102809]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:36.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:35 smithi146 ceph-mon[102809]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:17:36.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:35 smithi142 ceph-mon[102951]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:36.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:35 smithi142 ceph-mon[102951]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:17:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:35 smithi031 ceph-mon[98539]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:17:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:35 smithi031 ceph-mon[98539]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:17:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:36 smithi142 ceph-mon[102951]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:36 smithi142 ceph-mon[102951]: osdmap e77: 11 total, 11 up, 11 in 2023-10-06T21:17:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:36 smithi031 ceph-mon[98539]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:36 smithi031 ceph-mon[98539]: osdmap e77: 11 total, 11 up, 11 in 2023-10-06T21:17:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:36 smithi146 ceph-mon[102809]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:36 smithi146 ceph-mon[102809]: osdmap e77: 11 total, 11 up, 11 in 2023-10-06T21:17:38.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:37 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/4256878399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:37 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:37 smithi142 ceph-mon[102951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]': finished 2023-10-06T21:17:38.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:37 smithi142 ceph-mon[102951]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:17:38.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:37 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:37 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/4256878399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:37 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:37 smithi031 ceph-mon[98539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]': finished 2023-10-06T21:17:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:37 smithi031 ceph-mon[98539]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:17:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:37 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:38.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:37 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/4256878399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:37 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]: dispatch 2023-10-06T21:17:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:37 smithi146 ceph-mon[102809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4"}]': finished 2023-10-06T21:17:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:37 smithi146 ceph-mon[102809]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:17:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:37 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:39.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:38 smithi142 ceph-mon[102951]: pgmap v254: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:39.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:38 smithi142 ceph-mon[102951]: from='client.? 172.21.15.146:0/928303119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:38 smithi031 ceph-mon[98539]: pgmap v254: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:38 smithi031 ceph-mon[98539]: from='client.? 172.21.15.146:0/928303119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:38 smithi146 ceph-mon[102809]: pgmap v254: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:17:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:38 smithi146 ceph-mon[102809]: from='client.? 172.21.15.146:0/928303119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:17:41.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:40 smithi142 ceph-mon[102951]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:17:41.220 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:40 smithi146 ceph-mon[102809]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:17:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:40 smithi031 ceph-mon[98539]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:17:43.011 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:42 smithi146 ceph-mon[102809]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:42 smithi142 ceph-mon[102951]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:42 smithi031 ceph-mon[98539]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:44 smithi031 ceph-mon[98539]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:17:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:44 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:44 smithi146 ceph-mon[102809]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:17:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:44 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:44 smithi142 ceph-mon[102951]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:17:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:44 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:45.918 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:45 smithi146 ceph-mon[102809]: Deploying daemon osd.11 on smithi146 2023-10-06T21:17:46.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:45 smithi031 ceph-mon[98539]: Deploying daemon osd.11 on smithi146 2023-10-06T21:17:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:45 smithi142 ceph-mon[102951]: Deploying daemon osd.11 on smithi146 2023-10-06T21:17:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:46 smithi146 ceph-mon[102809]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:46 smithi142 ceph-mon[102951]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:47.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:46 smithi031 ceph-mon[98539]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:49.127 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:48 smithi146 ceph-mon[102809]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:48 smithi142 ceph-mon[102951]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:48 smithi031 ceph-mon[98539]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:50.911 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:50 smithi146 ceph-mon[102809]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:50 smithi142 ceph-mon[102951]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:50 smithi031 ceph-mon[98539]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:51 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:51 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:51 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:51 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:51 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:51 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:51 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:51 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:51 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:52 smithi031 ceph-mon[98539]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:52 smithi146 ceph-mon[102809]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:52 smithi142 ceph-mon[102951]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:54.443 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 11 on host 'smithi146' 2023-10-06T21:17:54.443 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.439+0000 7fd4137fe700 1 -- 172.21.15.146:0/3549270601 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd41c10c710 con 0x7fd4080a1480 2023-10-06T21:17:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 msgr2=0x7fd4080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:54.445 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 0x7fd4080a3940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd41c198130 tx=0x7fd418008040 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 msgr2=0x7fd41c19e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ef30 tx=0x7fd41400c450 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 shutdown_connections 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd41c102b10 0x7fd41c19e1d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41c100f70 0x7fd41c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 --2- 172.21.15.146:0/3549270601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd41c102170 0x7fd41c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 >> 172.21.15.146:0/3549270601 conn(0x7fd41c0fc4d0 msgr2=0x7fd41c105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 shutdown_connections 2023-10-06T21:17:54.446 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:17:54.442+0000 7fd424498700 1 -- 172.21.15.146:0/3549270601 wait complete. 2023-10-06T21:17:54.908 DEBUG:teuthology.orchestra.run.smithi146:osd.11> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-06T21:17:54.914 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-06T21:17:54.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T21:17:54.988 INFO:journalctl@ceph.osd.11.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:17:55.042 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:55.042 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.042 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.042 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:55.042 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:55.043 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.043 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.043 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.043 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:55.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:55.299 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:17:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:17:55.037+0000 7fa2bfa54640 -1 osd.11 0 log_to_monitors true 2023-10-06T21:17:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:17:56.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:55 smithi146 ceph-mon[102809]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:56.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:55 smithi146 ceph-mon[102809]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:56.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:55 smithi031 ceph-mon[98539]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:56.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:55 smithi031 ceph-mon[98539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:56.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:55 smithi142 ceph-mon[102951]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:56.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:55 smithi142 ceph-mon[102951]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:17:57.049 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:17:56.782+0000 7fa2b50cb700 -1 osd.11 0 waiting for initial osdmap 2023-10-06T21:17:57.049 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:17:56.799+0000 7fa2aeebc700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:17:57.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:17:57.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:17:57.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:57.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:56 smithi146 ceph-mon[102809]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:17:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:17:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:56 smithi142 ceph-mon[102951]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.155+0000 7f43c762d700 1 -- 172.21.15.31:0/1228217949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 msgr2=0x7f43c01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.155+0000 7f43c762d700 1 --2- 172.21.15.31:0/1228217949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c01025d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009990 tx=0x7f43b0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 -- 172.21.15.31:0/1228217949 shutdown_connections 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 --2- 172.21.15.31:0/1228217949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0102b10 0x7f43c01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 --2- 172.21.15.31:0/1228217949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0100f70 0x7f43c0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 --2- 172.21.15.31:0/1228217949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c01025d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 -- 172.21.15.31:0/1228217949 >> 172.21.15.31:0/1228217949 conn(0x7f43c00fc4d0 msgr2=0x7f43c00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 -- 172.21.15.31:0/1228217949 shutdown_connections 2023-10-06T21:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.156+0000 7f43c762d700 1 -- 172.21.15.31:0/1228217949 wait complete. 2023-10-06T21:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.157+0000 7f43c762d700 1 Processor -- start 2023-10-06T21:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 -- start start 2023-10-06T21:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0102b10 0x7f43c019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019e760 con 0x7f43c0100f70 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019e8a0 con 0x7f43c0102b10 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c762d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019ea30 con 0x7f43c0102170 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c53c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c53c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34014/0 (socket says 172.21.15.31:34014) 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.158+0000 7f43c53c9700 1 -- 172.21.15.31:0/600049870 learned_addr learned my addr 172.21.15.31:0/600049870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c4bc8700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 -- 172.21.15.31:0/600049870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 msgr2=0x7f43c0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 -- 172.21.15.31:0/600049870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0102b10 msgr2=0x7f43c019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0102b10 0x7f43c019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 -- 172.21.15.31:0/600049870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009c40 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c53c9700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00e9a0 tx=0x7f43bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc009790 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b00095f0 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43bc0044b0 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.159+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003be0 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.160+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c019f0d0 con 0x7f43c0100f70 2023-10-06T21:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.161+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0108650 con 0x7f43c0100f70 2023-10-06T21:17:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.163+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f43bc003d40 con 0x7f43c0100f70 2023-10-06T21:17:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.164+0000 7f43b67fc700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 0x7f43ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.164+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f43bc014070 con 0x7f43c0100f70 2023-10-06T21:17:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.164+0000 7f43c4bc8700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 0x7f43ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.165+0000 7f43c4bc8700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 0x7f43ac0a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b2e0 tx=0x7f43b0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.168+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f43bc096600 con 0x7f43c0100f70 2023-10-06T21:17:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:17:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:17:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: from='osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2023-10-06T21:17:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:56 smithi031 ceph-mon[98539]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:17:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.377+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f43c010c870 con 0x7f43c0100f70 2023-10-06T21:17:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.377+0000 7f43b67fc700 1 -- 172.21.15.31:0/600049870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f43bc099c20 con 0x7f43c0100f70 2023-10-06T21:17:57.381 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 msgr2=0x7f43ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 0x7f43ac0a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b2e0 tx=0x7f43b0005fd0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 msgr2=0x7f43c0196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00e9a0 tx=0x7f43bc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 shutdown_connections 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f43ac0a1480 0x7f43ac0a3940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0102b10 0x7f43c019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0100f70 0x7f43c0196bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 --2- 172.21.15.31:0/600049870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43c0102170 0x7f43c0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 >> 172.21.15.31:0/600049870 conn(0x7f43c00fc4d0 msgr2=0x7f43c0105790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 shutdown_connections 2023-10-06T21:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:17:57.380+0000 7f43c762d700 1 -- 172.21.15.31:0/600049870 wait complete. 2023-10-06T21:17:57.842 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1696627051,"num_in_osds":12,"osd_in_since":1696627057,"num_remapped_pgs":0} 2023-10-06T21:17:58.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:57 smithi146 ceph-mon[102809]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:17:58.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:57 smithi146 ceph-mon[102809]: osdmap e80: 12 total, 11 up, 12 in 2023-10-06T21:17:58.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:57 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/600049870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:17:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:57 smithi142 ceph-mon[102951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:17:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:57 smithi142 ceph-mon[102951]: osdmap e80: 12 total, 11 up, 12 in 2023-10-06T21:17:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:57 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/600049870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:17:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:57 smithi031 ceph-mon[98539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2023-10-06T21:17:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:57 smithi031 ceph-mon[98539]: osdmap e80: 12 total, 11 up, 12 in 2023-10-06T21:17:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:57 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/600049870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:17:58.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: purged_snaps scrub starts 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: purged_snaps scrub ok 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] boot 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:59.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:59.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: purged_snaps scrub starts 2023-10-06T21:17:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: purged_snaps scrub ok 2023-10-06T21:17:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] boot 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: purged_snaps scrub starts 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: purged_snaps scrub ok 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: osd.11 [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] boot 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:17:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:17:59.591 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:18:00.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:59 smithi146 ceph-mon[102809]: Detected new or changed devices on smithi146 2023-10-06T21:18:00.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:59 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 4316M 2023-10-06T21:18:00.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:17:59 smithi146 ceph-mon[102809]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:18:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:59 smithi142 ceph-mon[102951]: Detected new or changed devices on smithi146 2023-10-06T21:18:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:59 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 4316M 2023-10-06T21:18:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:17:59 smithi142 ceph-mon[102951]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:18:00.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:59 smithi031 ceph-mon[98539]: Detected new or changed devices on smithi146 2023-10-06T21:18:00.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:59 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 4316M 2023-10-06T21:18:00.191 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:17:59 smithi031 ceph-mon[98539]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.104+0000 7fbb2416f700 1 -- 172.21.15.31:0/1013725368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c073910 msgr2=0x7fbb1c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.104+0000 7fbb2416f700 1 --2- 172.21.15.31:0/1013725368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c073910 0x7fbb1c073d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009990 tx=0x7fbb0c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 -- 172.21.15.31:0/1013725368 shutdown_connections 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 --2- 172.21.15.31:0/1013725368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c108e40 0x7fbb1c10b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 --2- 172.21.15.31:0/1013725368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 0x7fbb1c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 --2- 172.21.15.31:0/1013725368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c073910 0x7fbb1c073d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 -- 172.21.15.31:0/1013725368 >> 172.21.15.31:0/1013725368 conn(0x7fbb1c0fc4d0 msgr2=0x7fbb1c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 -- 172.21.15.31:0/1013725368 shutdown_connections 2023-10-06T21:18:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.105+0000 7fbb2416f700 1 -- 172.21.15.31:0/1013725368 wait complete. 2023-10-06T21:18:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 Processor -- start 2023-10-06T21:18:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 -- start start 2023-10-06T21:18:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 0x7fbb1c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 0x7fbb1c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a0930 con 0x7fbb1c108e40 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a0a70 con 0x7fbb1c074ee0 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.106+0000 7fbb2416f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a0c00 con 0x7fbb1c073910 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34046/0 (socket says 172.21.15.31:34046) 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 -- 172.21.15.31:0/4280493441 learned_addr learned my addr 172.21.15.31:0/4280493441 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb21f0b700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 0x7fbb1c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2170a700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 0x7fbb1c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 -- 172.21.15.31:0/4280493441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 msgr2=0x7fbb1c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 0x7fbb1c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 -- 172.21.15.31:0/4280493441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 msgr2=0x7fbb1c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 0x7fbb1c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 -- 172.21.15.31:0/4280493441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c0095f0 con 0x7fbb1c108e40 2023-10-06T21:18:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb21f0b700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 0x7fbb1c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.107+0000 7fbb2270c700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f10 tx=0x7fbb1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.108+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.108+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb1400f040 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.108+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14005710 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.108+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1a0ee0 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.108+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1a1300 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.109+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c1074f0 con 0x7fbb1c108e40 2023-10-06T21:18:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.111+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb14004ad0 con 0x7fbb1c108e40 2023-10-06T21:18:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.112+0000 7fbb12ffd700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 0x7fbb080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.112+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb140cbfe0 con 0x7fbb1c108e40 2023-10-06T21:18:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.112+0000 7fbb21f0b700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 0x7fbb080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.113+0000 7fbb21f0b700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 0x7fbb080a39c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005950 tx=0x7fbb180058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.117+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb14095a80 con 0x7fbb1c108e40 2023-10-06T21:18:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:00 smithi142 ceph-mon[102951]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:00 smithi031 ceph-mon[98539]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:01.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:00 smithi146 ceph-mon[102809]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.326+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbb1c10e890 con 0x7fbb1c108e40 2023-10-06T21:18:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.327+0000 7fbb12ffd700 1 -- 172.21.15.31:0/4280493441 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7fbb140990a0 con 0x7fbb1c108e40 2023-10-06T21:18:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 msgr2=0x7fbb080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 0x7fbb080a39c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005950 tx=0x7fbb180058e0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 msgr2=0x7fbb1c1a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f10 tx=0x7fbb1400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 shutdown_connections 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbb080a1500 0x7fbb080a39c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb1c108e40 0x7fbb1c1a0360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb1c073910 0x7fbb1c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 --2- 172.21.15.31:0/4280493441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb1c074ee0 0x7fbb1c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 >> 172.21.15.31:0/4280493441 conn(0x7fbb1c0fc4d0 msgr2=0x7fbb1c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 shutdown_connections 2023-10-06T21:18:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:01.329+0000 7fbb2416f700 1 -- 172.21.15.31:0/4280493441 wait complete. 2023-10-06T21:18:01.778 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1696627077,"num_in_osds":12,"osd_in_since":1696627057,"num_remapped_pgs":0} 2023-10-06T21:18:01.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:18:02.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4280493441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:18:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4280493441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:18:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4280493441' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:18:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:18:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:02 smithi031 ceph-mon[98539]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:03.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:02 smithi142 ceph-mon[102951]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:02 smithi146 ceph-mon[102809]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.020+0000 7f77272e0700 1 -- 172.21.15.31:0/1962203044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 msgr2=0x7f772010d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.020+0000 7f77272e0700 1 --2- 172.21.15.31:0/1962203044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f772010d3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7710009990 tx=0x7f7710009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 -- 172.21.15.31:0/1962203044 shutdown_connections 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 --2- 172.21.15.31:0/1962203044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f772010d930 0x7f772010fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 --2- 172.21.15.31:0/1962203044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200730e0 0x7f77200734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 --2- 172.21.15.31:0/1962203044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f772010d3f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 -- 172.21.15.31:0/1962203044 >> 172.21.15.31:0/1962203044 conn(0x7f77200780a0 msgr2=0x7f77200784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 -- 172.21.15.31:0/1962203044 shutdown_connections 2023-10-06T21:18:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.021+0000 7f77272e0700 1 -- 172.21.15.31:0/1962203044 wait complete. 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 Processor -- start 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 -- start start 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77200730e0 0x7f7720199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 0x7f772019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772019e1c0 con 0x7f7720073a00 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772019e330 con 0x7f772010d930 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f77272e0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772019e4f0 con 0x7f77200730e0 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.022+0000 7f772487b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40360/0 (socket says 172.21.15.31:40360) 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 -- 172.21.15.31:0/2065957477 learned_addr learned my addr 172.21.15.31:0/2065957477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772507c700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77200730e0 0x7f7720199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 -- 172.21.15.31:0/2065957477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77200730e0 msgr2=0x7f7720199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77200730e0 0x7f7720199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772587d700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 0x7f772019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 -- 172.21.15.31:0/2065957477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 msgr2=0x7f772019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 0x7f772019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 -- 172.21.15.31:0/2065957477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100095f0 con 0x7f7720073a00 2023-10-06T21:18:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772587d700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 0x7f772019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.023+0000 7f772487b700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77100036c0 tx=0x7f7710004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.024+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771001d070 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.024+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7710004000 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.024+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772019e770 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.024+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710022d30 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.024+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77201a5e60 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.025+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772004ebb0 con 0x7f7720073a00 2023-10-06T21:18:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.027+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7710003b70 con 0x7f7720073a00 2023-10-06T21:18:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.028+0000 7f77167fc700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 0x7f770c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.028+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77100cdef0 con 0x7f7720073a00 2023-10-06T21:18:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.028+0000 7f772507c700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 0x7f770c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.029+0000 7f772507c700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 0x7f770c0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f771c006fd0 tx=0x7f771c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:04.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.032+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77100979e0 con 0x7f7720073a00 2023-10-06T21:18:04.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.243+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7720112fe0 con 0x7f7720073a00 2023-10-06T21:18:04.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.244+0000 7f77167fc700 1 -- 172.21.15.31:0/2065957477 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f771009b000 con 0x7f7720073a00 2023-10-06T21:18:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:04.247 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:17:58.784810+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 msgr2=0x7f770c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 0x7f770c0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f771c006fd0 tx=0x7f771c009190 comp rx=0 tx=0).stop 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 msgr2=0x7f7720199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77100036c0 tx=0x7f7710004730 comp rx=0 tx=0).stop 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 shutdown_connections 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f770c0a1480 0x7f770c0a3940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772010d930 0x7f772019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77200730e0 0x7f7720199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 --2- 172.21.15.31:0/2065957477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7720073a00 0x7f7720199740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 >> 172.21.15.31:0/2065957477 conn(0x7f77200780a0 msgr2=0x7f77200ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 shutdown_connections 2023-10-06T21:18:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:04.247+0000 7f77272e0700 1 -- 172.21.15.31:0/2065957477 wait complete. 2023-10-06T21:18:04.697 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:18:04.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-06T21:18:05.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:04 smithi031 ceph-mon[98539]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:05.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:04 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2065957477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:05.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:04 smithi146 ceph-mon[102809]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:05.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:04 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2065957477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:04 smithi142 ceph-mon[102951]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:04 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2065957477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 -- 172.21.15.31:0/2910758410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940101940 msgr2=0x7f794010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 --2- 172.21.15.31:0/2910758410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940101940 0x7f794010ddb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f793800b0d0 tx=0x7f793800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 -- 172.21.15.31:0/2910758410 shutdown_connections 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 --2- 172.21.15.31:0/2910758410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940101940 0x7f794010ddb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 --2- 172.21.15.31:0/2910758410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940106f90 0x7f7940107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 --2- 172.21.15.31:0/2910758410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f7940101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 -- 172.21.15.31:0/2910758410 >> 172.21.15.31:0/2910758410 conn(0x7f79400780a0 msgr2=0x7f79400784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 -- 172.21.15.31:0/2910758410 shutdown_connections 2023-10-06T21:18:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.940+0000 7f7948015700 1 -- 172.21.15.31:0/2910758410 wait complete. 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.941+0000 7f7948015700 1 Processor -- start 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 -- start start 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940101940 0x7f7940199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 0x7f794019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019e1a0 con 0x7f7940106f90 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019e310 con 0x7f7940101940 2023-10-06T21:18:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7948015700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019e4d0 con 0x7f7940100f80 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7945db1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7945db1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43222/0 (socket says 172.21.15.31:43222) 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f7945db1700 1 -- 172.21.15.31:0/4194329053 learned_addr learned my addr 172.21.15.31:0/4194329053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f79465b2700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 0x7f794019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.942+0000 7f79455b0700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940101940 0x7f7940199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 -- 172.21.15.31:0/4194329053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940101940 msgr2=0x7f7940199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940101940 0x7f7940199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 -- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 msgr2=0x7f794019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 0x7f794019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 -- 172.21.15.31:0/4194329053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009d00 con 0x7f7940100f80 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f79465b2700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 0x7f794019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7945db1700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f793c00d950 tx=0x7f793c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00f890 con 0x7f7940100f80 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f793c00fe80 con 0x7f7940100f80 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794019e7b0 con 0x7f7940100f80 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.943+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00e3b0 con 0x7f7940100f80 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.944+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401a6010 con 0x7f7940100f80 2023-10-06T21:18:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.945+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794004ebb0 con 0x7f7940100f80 2023-10-06T21:18:06.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.946+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f793c010460 con 0x7f7940100f80 2023-10-06T21:18:06.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.948+0000 7f7936ffd700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 0x7f792c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:06.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.948+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f793c0cca00 con 0x7f7940100f80 2023-10-06T21:18:06.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.948+0000 7f79455b0700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 0x7f792c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:06.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.949+0000 7f79455b0700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 0x7f792c0a3940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f794019a800 tx=0x7f793000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:06.950+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f793c096460 con 0x7f7940100f80 2023-10-06T21:18:07.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:06 smithi031 ceph-mon[98539]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:06 smithi142 ceph-mon[102951]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.158+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f794019a540 con 0x7f7940100f80 2023-10-06T21:18:07.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.159+0000 7f7936ffd700 1 -- 172.21.15.31:0/4194329053 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f793c099a80 con 0x7f7940100f80 2023-10-06T21:18:07.161 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 msgr2=0x7f792c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 0x7f792c0a3940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f794019a800 tx=0x7f793000b410 comp rx=0 tx=0).stop 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 msgr2=0x7f79401991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f793c00d950 tx=0x7f793c00dc60 comp rx=0 tx=0).stop 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 shutdown_connections 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f792c0a1480 0x7f792c0a3940 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7940106f90 0x7f794019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7940100f80 0x7f79401991e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:07.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 --2- 172.21.15.31:0/4194329053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7940101940 0x7f7940199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:07.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.161+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 >> 172.21.15.31:0/4194329053 conn(0x7f79400780a0 msgr2=0x7f794010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:07.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.162+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 shutdown_connections 2023-10-06T21:18:07.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:07.162+0000 7f7948015700 1 -- 172.21.15.31:0/4194329053 wait complete. 2023-10-06T21:18:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:06 smithi146 ceph-mon[102809]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.a on smithi031 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.d on smithi031 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.g on smithi031 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.j on smithi031 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.b on smithi142 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.e on smithi142 2023-10-06T21:18:07.618 INFO:tasks.cephadm:Adding mds.h on smithi142 2023-10-06T21:18:07.619 INFO:tasks.cephadm:Adding mds.k on smithi142 2023-10-06T21:18:07.619 INFO:tasks.cephadm:Adding mds.c on smithi146 2023-10-06T21:18:07.619 INFO:tasks.cephadm:Adding mds.f on smithi146 2023-10-06T21:18:07.619 INFO:tasks.cephadm:Adding mds.i on smithi146 2023-10-06T21:18:07.619 INFO:tasks.cephadm:Adding mds.l on smithi146 2023-10-06T21:18:07.619 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l' 2023-10-06T21:18:08.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:07 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4194329053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:08.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:07 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4194329053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:07 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4194329053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:08.359 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:18:08.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:08 smithi146 ceph-mon[102809]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:08 smithi142 ceph-mon[102951]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:08 smithi031 ceph-mon[98539]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:09.760 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 -- 172.21.15.146:0/22384728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c102170 msgr2=0x7f0a3c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:09.760 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 --2- 172.21.15.146:0/22384728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c102170 0x7f0a3c1025d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009990 tx=0x7f0a30009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.760 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 -- 172.21.15.146:0/22384728 shutdown_connections 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 --2- 172.21.15.146:0/22384728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102b10 0x7f0a3c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 --2- 172.21.15.146:0/22384728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c100f70 0x7f0a3c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 --2- 172.21.15.146:0/22384728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c102170 0x7f0a3c1025d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 -- 172.21.15.146:0/22384728 >> 172.21.15.146:0/22384728 conn(0x7f0a3c0fc4d0 msgr2=0x7f0a3c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 -- 172.21.15.146:0/22384728 shutdown_connections 2023-10-06T21:18:09.761 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.757+0000 7f0a419ad700 1 -- 172.21.15.146:0/22384728 wait complete. 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.758+0000 7f0a419ad700 1 Processor -- start 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.758+0000 7f0a419ad700 1 -- start start 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 0x7f0a3c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c102b10 0x7f0a3c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:09.763 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c073f60 con 0x7f0a3c102b10 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0740d0 con 0x7f0a3c102170 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a419ad700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c1a0bb0 con 0x7f0a3c100f70 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58006/0 (socket says 172.21.15.146:58006) 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 -- 172.21.15.146:0/3843607022 learned_addr learned my addr 172.21.15.146:0/3843607022 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3a7fc700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 0x7f0a3c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:09.764 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3b7fe700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c102b10 0x7f0a3c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 -- 172.21.15.146:0/3843607022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 msgr2=0x7f0a3c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 0x7f0a3c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 -- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c102b10 msgr2=0x7f0a3c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c102b10 0x7f0a3c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3affd700 1 -- 172.21.15.146:0/3843607022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300095f0 con 0x7f0a3c100f70 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.759+0000 7f0a3a7fc700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 0x7f0a3c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a3affd700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e970 tx=0x7f0a2c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009790 con 0x7f0a3c100f70 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a2c0044b0 con 0x7f0a3c100f70 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c003dd0 con 0x7f0a3c100f70 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c1a0e90 con 0x7f0a3c100f70 2023-10-06T21:18:09.765 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.760+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1a12a0 con 0x7f0a3c100f70 2023-10-06T21:18:09.766 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.761+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c108610 con 0x7f0a3c100f70 2023-10-06T21:18:09.766 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.763+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0a2c003f30 con 0x7f0a3c100f70 2023-10-06T21:18:09.767 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.764+0000 7f0a409ab700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 0x7f0a240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:09.767 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.764+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a2c014070 con 0x7f0a3c100f70 2023-10-06T21:18:09.767 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.765+0000 7f0a3a7fc700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 0x7f0a240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:09.768 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.765+0000 7f0a3a7fc700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 0x7f0a240a39c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b2e0 tx=0x7f0a30005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:09.771 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.768+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a2c0965d0 con 0x7f0a3c100f70 2023-10-06T21:18:09.978 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.976+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a3c02d390 con 0x7f0a240a1500 2023-10-06T21:18:09.987 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.984+0000 7f0a409ab700 1 -- 172.21.15.146:0/3843607022 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f0a3c02d390 con 0x7f0a240a1500 2023-10-06T21:18:09.987 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mds.all update... 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 msgr2=0x7f0a240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 0x7f0a240a39c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b2e0 tx=0x7f0a30005fb0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 msgr2=0x7f0a3c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e970 tx=0x7f0a2c00ed30 comp rx=0 tx=0).stop 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 shutdown_connections 2023-10-06T21:18:09.990 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0a240a1500 0x7f0a240a39c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a3c102b10 0x7f0a3c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a3c100f70 0x7f0a3c074c50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 --2- 172.21.15.146:0/3843607022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a3c102170 0x7f0a3c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 >> 172.21.15.146:0/3843607022 conn(0x7f0a3c0fc4d0 msgr2=0x7f0a3c105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 shutdown_connections 2023-10-06T21:18:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:2023-10-06T21:18:09.987+0000 7f0a419ad700 1 -- 172.21.15.146:0/3843607022 wait complete. 2023-10-06T21:18:10.437 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-06T21:18:10.441 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-06T21:18:10.487 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-06T21:18:10.523 INFO:journalctl@ceph.mds.a.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:18:10.527 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-06T21:18:10.561 INFO:journalctl@ceph.mds.d.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:18:10.565 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-06T21:18:10.570 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-06T21:18:10.599 INFO:journalctl@ceph.mds.g.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:18:10.617 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-06T21:18:10.634 INFO:journalctl@ceph.mds.j.smithi031.stdout:-- Logs begin at Fri 2023-10-06 20:52:03 UTC. -- 2023-10-06T21:18:10.649 INFO:journalctl@ceph.mds.b.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:18:10.652 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-06T21:18:10.689 INFO:journalctl@ceph.mds.e.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:18:10.693 DEBUG:teuthology.orchestra.run.smithi146:mds.c> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-06T21:18:10.697 DEBUG:teuthology.orchestra.run.smithi146:mds.f> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-06T21:18:10.722 INFO:journalctl@ceph.mds.h.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:18:10.743 DEBUG:teuthology.orchestra.run.smithi146:mds.i> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-06T21:18:10.763 INFO:journalctl@ceph.mds.k.smithi142.stdout:-- Logs begin at Fri 2023-10-06 20:52:02 UTC. -- 2023-10-06T21:18:10.773 INFO:journalctl@ceph.mds.c.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:18:10.776 DEBUG:teuthology.orchestra.run.smithi146:mds.l> sudo journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-06T21:18:10.817 INFO:journalctl@ceph.mds.f.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:18:10.821 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-06T21:18:10.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:18:10.849 INFO:journalctl@ceph.mds.i.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:18:10.891 INFO:journalctl@ceph.mds.l.smithi146.stdout:-- Logs begin at Fri 2023-10-06 20:52:10 UTC. -- 2023-10-06T21:18:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 -- 172.21.15.31:0/1148083067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08100ea0 msgr2=0x7f1e081012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/1148083067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08100ea0 0x7f1e081012c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009990 tx=0x7f1e04009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 -- 172.21.15.31:0/1148083067 shutdown_connections 2023-10-06T21:18:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/1148083067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 0x7f1e08109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/1148083067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08100ea0 0x7f1e081012c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/1148083067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08102080 0x7f1e08102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 -- 172.21.15.31:0/1148083067 >> 172.21.15.31:0/1148083067 conn(0x7f1e080fc420 msgr2=0x7f1e080fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:11.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 -- 172.21.15.31:0/1148083067 shutdown_connections 2023-10-06T21:18:11.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.035+0000 7f1e0fbff700 1 -- 172.21.15.31:0/1148083067 wait complete. 2023-10-06T21:18:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 Processor -- start 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 -- start start 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08100ea0 0x7f1e08196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 0x7f1e0819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819e760 con 0x7f1e08102080 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819e8a0 con 0x7f1e08102a40 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.036+0000 7f1e0fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819ea30 con 0x7f1e08100ea0 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40384/0 (socket says 172.21.15.31:40384) 2023-10-06T21:18:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 -- 172.21.15.31:0/3149368657 learned_addr learned my addr 172.21.15.31:0/3149368657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d99b700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08100ea0 0x7f1e08196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0e19c700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 0x7f1e0819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 -- 172.21.15.31:0/3149368657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08100ea0 msgr2=0x7f1e08196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08100ea0 0x7f1e08196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 -- 172.21.15.31:0/3149368657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 msgr2=0x7f1e0819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 0x7f1e0819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 -- 172.21.15.31:0/3149368657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e040095f0 con 0x7f1e08102080 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1e0d19a700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e970 tx=0x7f1df800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.037+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8009790 con 0x7f1e08102080 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.038+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1df800c7e0 con 0x7f1e08102080 2023-10-06T21:18:11.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.038+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0819ed10 con 0x7f1e08102080 2023-10-06T21:18:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.038+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003bd0 con 0x7f1e08102080 2023-10-06T21:18:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.038+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0819f1d0 con 0x7f1e08102080 2023-10-06T21:18:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.039+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e08108610 con 0x7f1e08102080 2023-10-06T21:18:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.041+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1df800c950 con 0x7f1e08102080 2023-10-06T21:18:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.042+0000 7f1dfeffd700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 0x7f1df40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.042+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1df8014070 con 0x7f1e08102080 2023-10-06T21:18:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.042+0000 7f1e0d99b700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 0x7f1df40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.043+0000 7f1e0d99b700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 0x7f1df40a3990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005910 tx=0x7f1e0400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.046+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1df8096540 con 0x7f1e08102080 2023-10-06T21:18:11.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:11.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:11.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:11.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:10 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:11.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:10 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.259+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e0810c610 con 0x7f1e08102080 2023-10-06T21:18:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.260+0000 7f1dfeffd700 1 -- 172.21.15.31:0/3149368657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f1df8099b60 con 0x7f1e08102080 2023-10-06T21:18:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:11.262 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:17:58.784810+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 msgr2=0x7f1df40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 0x7f1df40a3990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005910 tx=0x7f1e0400b260 comp rx=0 tx=0).stop 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 msgr2=0x7f1e08197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e970 tx=0x7f1df800ec80 comp rx=0 tx=0).stop 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 shutdown_connections 2023-10-06T21:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1df40a14d0 0x7f1df40a3990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e08102a40 0x7f1e0819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e08100ea0 0x7f1e08196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 --2- 172.21.15.31:0/3149368657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e08102080 0x7f1e08197120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 >> 172.21.15.31:0/3149368657 conn(0x7f1e080fc420 msgr2=0x7f1e08105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 shutdown_connections 2023-10-06T21:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.262+0000 7f1e0fbff700 1 -- 172.21.15.31:0/3149368657 wait complete. 2023-10-06T21:18:11.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:18:11.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:18:11.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:11.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:18:11.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2023-10-06T21:18:11.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:11.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:10 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 -- 172.21.15.31:0/2642428735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140108da0 msgr2=0x7f714010b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 --2- 172.21.15.31:0/2642428735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140108da0 0x7f714010b230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f713800b0a0 tx=0x7f713800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 -- 172.21.15.31:0/2642428735 shutdown_connections 2023-10-06T21:18:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 --2- 172.21.15.31:0/2642428735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140108da0 0x7f714010b230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 --2- 172.21.15.31:0/2642428735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140074ef0 0x7f7140073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 --2- 172.21.15.31:0/2642428735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140073910 0x7f7140073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 -- 172.21.15.31:0/2642428735 >> 172.21.15.31:0/2642428735 conn(0x7f71400fc430 msgr2=0x7f71400fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 -- 172.21.15.31:0/2642428735 shutdown_connections 2023-10-06T21:18:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.490+0000 7f7147fff700 1 -- 172.21.15.31:0/2642428735 wait complete. 2023-10-06T21:18:11.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 Processor -- start 2023-10-06T21:18:11.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 -- start start 2023-10-06T21:18:11.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 0x7f7140198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140108da0 0x7f71401a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401a0990 con 0x7f7140074ef0 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401a0ad0 con 0x7f7140073910 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f7147fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401a0c60 con 0x7f7140108da0 2023-10-06T21:18:11.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.491+0000 7f714559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40406/0 (socket says 172.21.15.31:40406) 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 -- 172.21.15.31:0/2428881010 learned_addr learned my addr 172.21.15.31:0/2428881010 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714659c700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140108da0 0x7f71401a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7145d9b700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 0x7f7140198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 -- 172.21.15.31:0/2428881010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140108da0 msgr2=0x7f71401a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140108da0 0x7f71401a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 -- 172.21.15.31:0/2428881010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 msgr2=0x7f7140198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 0x7f7140198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 -- 172.21.15.31:0/2428881010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138009d00 con 0x7f7140074ef0 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7145d9b700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 0x7f7140198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f714559a700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f713000d980 tx=0x7f713000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130004020 con 0x7f7140074ef0 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7130009730 con 0x7f7140074ef0 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130003a00 con 0x7f7140074ef0 2023-10-06T21:18:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.492+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71401a0f40 con 0x7f7140074ef0 2023-10-06T21:18:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.493+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71401a1370 con 0x7f7140074ef0 2023-10-06T21:18:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.494+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7140107450 con 0x7f7140074ef0 2023-10-06T21:18:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.496+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f71300098a0 con 0x7f7140074ef0 2023-10-06T21:18:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.498+0000 7f7136ffd700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 0x7f712c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.498+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71300cc540 con 0x7f7140074ef0 2023-10-06T21:18:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.498+0000 7f7145d9b700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 0x7f712c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.499+0000 7f7145d9b700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 0x7f712c0a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f713c005950 tx=0x7f713c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.502+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7130095fa0 con 0x7f7140074ef0 2023-10-06T21:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.713+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f714002d090 con 0x7f7140074ef0 2023-10-06T21:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.714+0000 7f7136ffd700 1 -- 172.21.15.31:0/2428881010 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71300995c0 con 0x7f7140074ef0 2023-10-06T21:18:11.716 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 msgr2=0x7f712c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 0x7f712c0a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f713c005950 tx=0x7f713c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 msgr2=0x7f7140199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f713000d980 tx=0x7f713000dc90 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 shutdown_connections 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f712c0a1500 0x7f712c0a39c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7140108da0 0x7f71401a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7140073910 0x7f7140198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 --2- 172.21.15.31:0/2428881010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7140074ef0 0x7f7140199350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.716+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 >> 172.21.15.31:0/2428881010 conn(0x7f71400fc430 msgr2=0x7f71400fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.717+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 shutdown_connections 2023-10-06T21:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.717+0000 7f7147fff700 1 -- 172.21.15.31:0/2428881010 wait complete. 2023-10-06T21:18:11.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:11.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 -- 172.21.15.31:0/2761112461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102080 msgr2=0x7fc3b0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/2761112461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102080 0x7fc3b0102500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009990 tx=0x7fc3a4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 -- 172.21.15.31:0/2761112461 shutdown_connections 2023-10-06T21:18:11.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/2761112461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102a40 0x7fc3b0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/2761112461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/2761112461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102080 0x7fc3b0102500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 -- 172.21.15.31:0/2761112461 >> 172.21.15.31:0/2761112461 conn(0x7fc3b00fc420 msgr2=0x7fc3b00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:11.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 -- 172.21.15.31:0/2761112461 shutdown_connections 2023-10-06T21:18:11.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.940+0000 7fc3b4fff700 1 -- 172.21.15.31:0/2761112461 wait complete. 2023-10-06T21:18:11.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.941+0000 7fc3b4fff700 1 Processor -- start 2023-10-06T21:18:11.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.941+0000 7fc3b4fff700 1 -- start start 2023-10-06T21:18:11.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.941+0000 7fc3b4fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.941+0000 7fc3b4fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102080 0x7fc3b01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3b4fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 0x7fc3b01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3b4fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01a08f0 con 0x7fc3b0102a40 2023-10-06T21:18:11.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3b4fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01a0a30 con 0x7fc3b0102080 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3b4fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01a0bc0 con 0x7fc3b0100ea0 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43244/0 (socket says 172.21.15.31:43244) 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3ae59c700 1 -- 172.21.15.31:0/3052409344 learned_addr learned my addr 172.21.15.31:0/3052409344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 0x7fc3b01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3add9b700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102080 0x7fc3b01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 -- 172.21.15.31:0/3052409344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 msgr2=0x7fc3b0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 -- 172.21.15.31:0/3052409344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102080 msgr2=0x7fc3b01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102080 0x7fc3b01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 -- 172.21.15.31:0/3052409344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40095f0 con 0x7fc3b0102a40 2023-10-06T21:18:11.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3ae59c700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:11.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.942+0000 7fc3aed9d700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 0x7fc3b01a0320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800ab40 tx=0x7fc3a800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.943+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800f070 con 0x7fc3b0102a40 2023-10-06T21:18:11.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.943+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3a8007a70 con 0x7fc3b0102a40 2023-10-06T21:18:11.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.943+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8005280 con 0x7fc3b0102a40 2023-10-06T21:18:11.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.943+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b01a0ea0 con 0x7fc3b0102a40 2023-10-06T21:18:11.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.943+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b01a12d0 con 0x7fc3b0102a40 2023-10-06T21:18:11.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.944+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b0193400 con 0x7fc3b0102a40 2023-10-06T21:18:11.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.946+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc3a8004ad0 con 0x7fc3b0102a40 2023-10-06T21:18:11.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.947+0000 7fc39f7fe700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 0x7fc3980a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:11.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.947+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3a80cbc80 con 0x7fc3b0102a40 2023-10-06T21:18:11.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.947+0000 7fc3ae59c700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 0x7fc3980a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:11.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.948+0000 7fc3ae59c700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 0x7fc3980a3a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005950 tx=0x7fc3a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:11.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:11.951+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc3a8095770 con 0x7fc3b0102a40 2023-10-06T21:18:12.025 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:11 smithi146 ceph-mon[102809]: Deploying daemon mds.l on smithi146 2023-10-06T21:18:12.026 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:11 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3149368657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:12.026 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:11 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2428881010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:12.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.211+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3b010c610 con 0x7fc3b0102a40 2023-10-06T21:18:12.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.212+0000 7fc39f7fe700 1 -- 172.21.15.31:0/3052409344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fc3a8098d90 con 0x7fc3b0102a40 2023-10-06T21:18:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:12.214 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 msgr2=0x7fc3980a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 0x7fc3980a3a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005950 tx=0x7fc3a0009260 comp rx=0 tx=0).stop 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 msgr2=0x7fc3b01a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 0x7fc3b01a0320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800ab40 tx=0x7fc3a800ae50 comp rx=0 tx=0).stop 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 shutdown_connections 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc3980a1550 0x7fc3980a3a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:12.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b0102a40 0x7fc3b01a0320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b0100ea0 0x7fc3b0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 --2- 172.21.15.31:0/3052409344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b0102080 0x7fc3b01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 >> 172.21.15.31:0/3052409344 conn(0x7fc3b00fc420 msgr2=0x7fc3b0107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 shutdown_connections 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:12.214+0000 7fc3b4fff700 1 -- 172.21.15.31:0/3052409344 wait complete. 2023-10-06T21:18:12.217 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2023-10-06T21:18:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:11 smithi031 ceph-mon[98539]: Deploying daemon mds.l on smithi146 2023-10-06T21:18:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:11 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3149368657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:11 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2428881010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:11 smithi142 ceph-mon[102951]: Deploying daemon mds.l on smithi146 2023-10-06T21:18:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:11 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3149368657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:11 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2428881010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:12.973 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:12 smithi146 systemd[1]: Starting Ceph mds.l for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:12 smithi031 ceph-mon[98539]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:12 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3052409344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:12 smithi146 ceph-mon[102809]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:12 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3052409344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:12 smithi142 ceph-mon[102951]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:12 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3052409344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:13.728 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:13 smithi146 podman[139115]: 2023-10-06T21:18:13.729 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:13 smithi146 bash[139115]: 3d1747b0af8713daa0639d3906a3401345dfabbdbf4761fb4ef8ca889b0c8c0d 2023-10-06T21:18:13.729 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:13 smithi146 ceph-mds[139156]: -- 172.21.15.146:0/2825155281 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d28d44b1e0 con 0x55d28d420c00 2023-10-06T21:18:13.729 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-l[139131]: starting mds.l at 2023-10-06T21:18:14.049 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:18:13 smithi146 systemd[1]: Started Ceph mds.l for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:14.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.450+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3961115437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb8106c00 msgr2=0x7f9fb8109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.450+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3961115437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb8106c00 0x7f9fb8109070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009990 tx=0x7f9fb0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3961115437 shutdown_connections 2023-10-06T21:18:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3961115437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb8106c00 0x7f9fb8109070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3961115437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81019a0 0x7f9fb8103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3961115437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb81042d0 0x7f9fb81066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3961115437 >> 172.21.15.31:0/3961115437 conn(0x7f9fb80fb3b0 msgr2=0x7f9fb80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3961115437 shutdown_connections 2023-10-06T21:18:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.451+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3961115437 wait complete. 2023-10-06T21:18:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 Processor -- start 2023-10-06T21:18:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 -- start start 2023-10-06T21:18:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81042d0 0x7f9fb80ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb8106c00 0x7f9fb80ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8100630 con 0x7f9fb81019a0 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81007a0 con 0x7f9fb8106c00 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.452+0000 7f9fbfbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81a0a30 con 0x7f9fb81042d0 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59626/0 (socket says 172.21.15.31:59626) 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 -- 172.21.15.31:0/3729894745 learned_addr learned my addr 172.21.15.31:0/3729894745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbe19c700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb8106c00 0x7f9fb80ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:14.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd19a700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81042d0 0x7f9fb80ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 -- 172.21.15.31:0/3729894745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81042d0 msgr2=0x7f9fb80ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81042d0 0x7f9fb80ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 -- 172.21.15.31:0/3729894745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb8106c00 msgr2=0x7f9fb80ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb8106c00 0x7f9fb80ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 -- 172.21.15.31:0/3729894745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00095f0 con 0x7f9fb81019a0 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fbd99b700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.453+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4009940 con 0x7f9fb81019a0 2023-10-06T21:18:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.454+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fa4010460 con 0x7f9fb81019a0 2023-10-06T21:18:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.454+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb81a0d10 con 0x7f9fb81019a0 2023-10-06T21:18:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.454+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400f620 con 0x7f9fb81019a0 2023-10-06T21:18:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.454+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb81a11d0 con 0x7f9fb81019a0 2023-10-06T21:18:14.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.455+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb80fcf30 con 0x7f9fb81019a0 2023-10-06T21:18:14.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.457+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9fa40105d0 con 0x7f9fb81019a0 2023-10-06T21:18:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.458+0000 7f9fb6ffd700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 0x7f9fa80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.458+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9fa40cc760 con 0x7f9fb81019a0 2023-10-06T21:18:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.458+0000 7f9fbd19a700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 0x7f9fa80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.459+0000 7f9fbd19a700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 0x7f9fa80a3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009380 tx=0x7f9fac00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.462+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fa40961c0 con 0x7f9fb81019a0 2023-10-06T21:18:14.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.723+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb802d020 con 0x7f9fb81019a0 2023-10-06T21:18:14.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.723+0000 7f9fb6ffd700 1 -- 172.21.15.31:0/3729894745 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f9fa40997e0 con 0x7f9fb81019a0 2023-10-06T21:18:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:14.726 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 msgr2=0x7f9fa80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 0x7f9fa80a3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009380 tx=0x7f9fac00b040 comp rx=0 tx=0).stop 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 msgr2=0x7f9fb8101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400dc60 comp rx=0 tx=0).stop 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 shutdown_connections 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fa80a14d0 0x7f9fa80a3990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb8106c00 0x7f9fb80ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fb81019a0 0x7f9fb8101310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 --2- 172.21.15.31:0/3729894745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb81042d0 0x7f9fb80ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 >> 172.21.15.31:0/3729894745 conn(0x7f9fb80fb3b0 msgr2=0x7f9fb8102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 shutdown_connections 2023-10-06T21:18:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:14.726+0000 7f9fbfbff700 1 -- 172.21.15.31:0/3729894745 wait complete. 2023-10-06T21:18:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2023-10-06T21:18:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6832/1755294522,v1:172.21.15.146:6833/1755294522] up:boot 2023-10-06T21:18:15.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: fsmap 1 up:standby 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:15.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:15.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:15.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:14 smithi031 ceph-mon[98539]: Deploying daemon mds.k on smithi142 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6832/1755294522,v1:172.21.15.146:6833/1755294522] up:boot 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: fsmap 1 up:standby 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:15.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:15.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:14 smithi146 ceph-mon[102809]: Deploying daemon mds.k on smithi142 2023-10-06T21:18:15.094 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6832/1755294522,v1:172.21.15.146:6833/1755294522] up:boot 2023-10-06T21:18:15.094 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: fsmap 1 up:standby 2023-10-06T21:18:15.094 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:18:15.094 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:15.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:15.096 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:14 smithi142 ceph-mon[102951]: Deploying daemon mds.k on smithi142 2023-10-06T21:18:16.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:15 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3729894745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:16.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:15 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3729894745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:16.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:15 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3729894745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:16.402 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 systemd[1]: Starting Ceph mds.k for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:16.832 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:16 smithi142 ceph-mon[102951]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:16.832 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 podman[139555]: 2023-10-06T21:18:16.832 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 bash[139555]: 584618a3fa7eb3e47de52ce6fc5b170efd10386ed78160d83b1e054540986049 2023-10-06T21:18:16.832 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 ceph-mds[139595]: -- 172.21.15.142:0/1084895746 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5590029f71e0 con 0x5590029ccc00 2023-10-06T21:18:16.833 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-k[139571]: starting mds.k at 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.961+0000 7f44579ff700 1 -- 172.21.15.31:0/10847809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450074ef0 msgr2=0x7f4450073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.961+0000 7f44579ff700 1 --2- 172.21.15.31:0/10847809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450074ef0 0x7f4450073340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f444c009990 tx=0x7f444c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 -- 172.21.15.31:0/10847809 shutdown_connections 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 --2- 172.21.15.31:0/10847809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f445010d4c0 0x7f445010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 --2- 172.21.15.31:0/10847809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450074ef0 0x7f4450073340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 --2- 172.21.15.31:0/10847809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450073910 0x7f4450073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 -- 172.21.15.31:0/10847809 >> 172.21.15.31:0/10847809 conn(0x7f44500fc420 msgr2=0x7f44500fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 -- 172.21.15.31:0/10847809 shutdown_connections 2023-10-06T21:18:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.962+0000 7f44579ff700 1 -- 172.21.15.31:0/10847809 wait complete. 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 Processor -- start 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 -- start start 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4450074ef0 0x7f445019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010d4c0 0x7f44501a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501a4c70 con 0x7f4450073910 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501a4db0 con 0x7f445010d4c0 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.963+0000 7f44579ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501a4f40 con 0x7f4450074ef0 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59644/0 (socket says 172.21.15.31:59644) 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 -- 172.21.15.31:0/1462271554 learned_addr learned my addr 172.21.15.31:0/1462271554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f4454f9a700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4450074ef0 0x7f445019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f4455f9c700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010d4c0 0x7f44501a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 -- 172.21.15.31:0/1462271554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4450074ef0 msgr2=0x7f445019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4450074ef0 0x7f445019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 -- 172.21.15.31:0/1462271554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010d4c0 msgr2=0x7f44501a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010d4c0 0x7f44501a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 -- 172.21.15.31:0/1462271554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0095f0 con 0x7f4450073910 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.964+0000 7f445579b700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f444c0098e0 tx=0x7f444c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.965+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c01d070 con 0x7f4450073910 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.965+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f444c022470 con 0x7f4450073910 2023-10-06T21:18:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.965+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44501a51c0 con 0x7f4450073910 2023-10-06T21:18:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.965+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00f6c0 con 0x7f4450073910 2023-10-06T21:18:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.965+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44501a55e0 con 0x7f4450073910 2023-10-06T21:18:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.966+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450103010 con 0x7f4450073910 2023-10-06T21:18:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.968+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f444c00baa0 con 0x7f4450073910 2023-10-06T21:18:16.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.969+0000 7f44467fc700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 0x7f443c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:16.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.969+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f444c0cde70 con 0x7f4450073910 2023-10-06T21:18:16.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.969+0000 7f4454f9a700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 0x7f443c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:16.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.970+0000 7f4454f9a700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 0x7f443c0a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4450074c10 tx=0x7f4440009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:16.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:16.973+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f444c097960 con 0x7f4450073910 2023-10-06T21:18:17.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:16 smithi031 ceph-mon[98539]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:17.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:16 smithi146 ceph-mon[102809]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:17.151 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:18:16 smithi142 systemd[1]: Started Ceph mds.k for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:17.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.234+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4450112be0 con 0x7f4450073910 2023-10-06T21:18:17.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.234+0000 7f44467fc700 1 -- 172.21.15.31:0/1462271554 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f444c09af80 con 0x7f4450073910 2023-10-06T21:18:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 msgr2=0x7f443c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 0x7f443c0a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4450074c10 tx=0x7f4440009260 comp rx=0 tx=0).stop 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 msgr2=0x7f445019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f444c0098e0 tx=0x7f444c009570 comp rx=0 tx=0).stop 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 shutdown_connections 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f443c0a14d0 0x7f443c0a3990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010d4c0 0x7f44501a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4450073910 0x7f445019d0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 --2- 172.21.15.31:0/1462271554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4450074ef0 0x7f445019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 >> 172.21.15.31:0/1462271554 conn(0x7f44500fc420 msgr2=0x7f44500fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 shutdown_connections 2023-10-06T21:18:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:17.237+0000 7f44579ff700 1 -- 172.21.15.31:0/1462271554 wait complete. 2023-10-06T21:18:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2023-10-06T21:18:18.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: Deploying daemon mds.j on smithi031 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1462271554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:boot 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: fsmap 2 up:standby 2023-10-06T21:18:18.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:17 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: Deploying daemon mds.j on smithi031 2023-10-06T21:18:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1462271554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:18.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:boot 2023-10-06T21:18:18.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: fsmap 2 up:standby 2023-10-06T21:18:18.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:17 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:18:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: Deploying daemon mds.j on smithi031 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1462271554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:boot 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: fsmap 2 up:standby 2023-10-06T21:18:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:17 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:18:19.159 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:18 smithi031 ceph-mon[98539]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:19.159 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 systemd[1]: Starting Ceph mds.j for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:19.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:18 smithi146 ceph-mon[102809]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:18 smithi142 ceph-mon[102951]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 -- 172.21.15.31:0/2396347100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102080 msgr2=0x7f80d8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2396347100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102080 0x7f80d8102500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009990 tx=0x7f80c8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 -- 172.21.15.31:0/2396347100 shutdown_connections 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2396347100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102a40 0x7f80d8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2396347100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 0x7f80d81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2396347100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102080 0x7f80d8102500 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.487+0000 7f80dfdff700 1 -- 172.21.15.31:0/2396347100 >> 172.21.15.31:0/2396347100 conn(0x7f80d80fc420 msgr2=0x7f80d80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.488+0000 7f80dfdff700 1 -- 172.21.15.31:0/2396347100 shutdown_connections 2023-10-06T21:18:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.488+0000 7f80dfdff700 1 -- 172.21.15.31:0/2396347100 wait complete. 2023-10-06T21:18:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.489+0000 7f80dfdff700 1 Processor -- start 2023-10-06T21:18:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.489+0000 7f80dfdff700 1 -- start start 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 0x7f80d8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102080 0x7f80d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8073f80 con 0x7f80d8102a40 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80740f0 con 0x7f80d8100ea0 2023-10-06T21:18:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dfdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d81a0bb0 con 0x7f80d8102080 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80de39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80de39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59668/0 (socket says 172.21.15.31:59668) 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80de39c700 1 -- 172.21.15.31:0/2786137244 learned_addr learned my addr 172.21.15.31:0/2786137244 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80ddb9b700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 0x7f80d8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.490+0000 7f80dd39a700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102080 0x7f80d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 -- 172.21.15.31:0/2786137244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102080 msgr2=0x7f80d80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102080 0x7f80d80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 -- 172.21.15.31:0/2786137244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 msgr2=0x7f80d8074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:19.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 0x7f80d8074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 -- 172.21.15.31:0/2786137244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80095f0 con 0x7f80d8102a40 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80de39c700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f80d000ef30 tx=0x7f80d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0018070 con 0x7f80d8102a40 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d81a0e30 con 0x7f80d8102a40 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d000f040 con 0x7f80d8102a40 2023-10-06T21:18:19.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cbc0 con 0x7f80d8102a40 2023-10-06T21:18:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.491+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d81a1260 con 0x7f80d8102a40 2023-10-06T21:18:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.493+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d81085e0 con 0x7f80d8102a40 2023-10-06T21:18:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.494+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80d0007500 con 0x7f80d8102a40 2023-10-06T21:18:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.496+0000 7f80ceffd700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 0x7f80c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.496+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80d00cc3c0 con 0x7f80d8102a40 2023-10-06T21:18:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.496+0000 7f80ddb9b700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 0x7f80c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.497+0000 7f80ddb9b700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 0x7f80c40a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f80d8101bf0 tx=0x7f80d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.500+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f80d0095eb0 con 0x7f80d8102a40 2023-10-06T21:18:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.773+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80d810c610 con 0x7f80d8102a40 2023-10-06T21:18:19.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.774+0000 7f80ceffd700 1 -- 172.21.15.31:0/2786137244 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f80d00994d0 con 0x7f80d8102a40 2023-10-06T21:18:19.776 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:19.776 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 msgr2=0x7f80c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 0x7f80c40a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f80d8101bf0 tx=0x7f80d4009260 comp rx=0 tx=0).stop 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 msgr2=0x7f80d80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f80d000ef30 tx=0x7f80d000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 shutdown_connections 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f80c40a14d0 0x7f80c40a3990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d8102a40 0x7f80d80737e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80d8100ea0 0x7f80d8074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 --2- 172.21.15.31:0/2786137244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d8102080 0x7f80d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 >> 172.21.15.31:0/2786137244 conn(0x7f80d80fc420 msgr2=0x7f80d8105720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 shutdown_connections 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:19.777+0000 7f80dfdff700 1 -- 172.21.15.31:0/2786137244 wait complete. 2023-10-06T21:18:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2023-10-06T21:18:19.913 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 podman[145881]: 2023-10-06T21:18:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:19 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2786137244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:20.260 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 bash[145881]: 2cc9e2cfbfe7346fdd5746735f59b5039b7191dac050d714910afdb98956a901 2023-10-06T21:18:20.260 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 ceph-mds[145943]: -- 172.21.15.31:0/1133515391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631f12771e0 con 0x5631f124d000 2023-10-06T21:18:20.260 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-j[145917]: starting mds.j at 2023-10-06T21:18:20.260 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:18:19 smithi031 systemd[1]: Started Ceph mds.j for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:19 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2786137244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:19 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2786137244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.31:6834/3223954371,v1:172.21.15.31:6835/3223954371] up:boot 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: fsmap 3 up:standby 2023-10-06T21:18:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:21 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:21.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.31:6834/3223954371,v1:172.21.15.31:6835/3223954371] up:boot 2023-10-06T21:18:21.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: fsmap 3 up:standby 2023-10-06T21:18:21.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:21 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:18:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.31:6834/3223954371,v1:172.21.15.31:6835/3223954371] up:boot 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: fsmap 3 up:standby 2023-10-06T21:18:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:21 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:18:21.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 -- 172.21.15.31:0/2343480006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24102a40 msgr2=0x7f5b24109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 --2- 172.21.15.31:0/2343480006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24102a40 0x7f5b24109360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00b0a0 tx=0x7f5b1c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 -- 172.21.15.31:0/2343480006 shutdown_connections 2023-10-06T21:18:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 --2- 172.21.15.31:0/2343480006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24102a40 0x7f5b24109360 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 --2- 172.21.15.31:0/2343480006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24100ea0 0x7f5b241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 --2- 172.21.15.31:0/2343480006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102080 0x7f5b24102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 -- 172.21.15.31:0/2343480006 >> 172.21.15.31:0/2343480006 conn(0x7f5b240fc420 msgr2=0x7f5b240fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 -- 172.21.15.31:0/2343480006 shutdown_connections 2023-10-06T21:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.216+0000 7f5b293ff700 1 -- 172.21.15.31:0/2343480006 wait complete. 2023-10-06T21:18:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.217+0000 7f5b293ff700 1 Processor -- start 2023-10-06T21:18:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.217+0000 7f5b293ff700 1 -- start start 2023-10-06T21:18:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 0x7f5b240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 0x7f5b240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24073f80 con 0x7f5b24100ea0 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240740f0 con 0x7f5b24102a40 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b293ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241a0bf0 con 0x7f5b24102080 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59716/0 (socket says 172.21.15.31:59716) 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b22ffd700 1 -- 172.21.15.31:0/1555706815 learned_addr learned my addr 172.21.15.31:0/1555706815 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b237fe700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 0x7f5b240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.218+0000 7f5b227fc700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 0x7f5b240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 -- 172.21.15.31:0/1555706815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 msgr2=0x7f5b240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 0x7f5b240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 -- 172.21.15.31:0/1555706815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 msgr2=0x7f5b240737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 0x7f5b240737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 -- 172.21.15.31:0/1555706815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c009d00 con 0x7f5b24100ea0 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b227fc700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 0x7f5b240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b237fe700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 0x7f5b240737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b22ffd700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400d980 tx=0x7f5b1400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400de60 con 0x7f5b24100ea0 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b140044b0 con 0x7f5b24100ea0 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14003b40 con 0x7f5b24100ea0 2023-10-06T21:18:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b241a0ed0 con 0x7f5b24100ea0 2023-10-06T21:18:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.219+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b241a12f0 con 0x7f5b24100ea0 2023-10-06T21:18:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.220+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24108460 con 0x7f5b24100ea0 2023-10-06T21:18:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.222+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5b14003ca0 con 0x7f5b24100ea0 2023-10-06T21:18:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.224+0000 7f5b0bfff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 0x7f5b0c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.224+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b140cc5b0 con 0x7f5b24100ea0 2023-10-06T21:18:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.224+0000 7f5b227fc700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 0x7f5b0c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.225+0000 7f5b227fc700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 0x7f5b0c0a3a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b24100bc0 tx=0x7f5b18009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.228+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5b14096010 con 0x7f5b24100ea0 2023-10-06T21:18:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:22 smithi146 ceph-mon[102809]: Deploying daemon mds.d on smithi031 2023-10-06T21:18:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:22 smithi142 ceph-mon[102951]: Deploying daemon mds.d on smithi031 2023-10-06T21:18:22.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:22 smithi031 ceph-mon[98539]: Deploying daemon mds.d on smithi031 2023-10-06T21:18:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.488+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b2410c610 con 0x7f5b24100ea0 2023-10-06T21:18:22.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.489+0000 7f5b0bfff700 1 -- 172.21.15.31:0/1555706815 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f5b14009940 con 0x7f5b24100ea0 2023-10-06T21:18:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:22.491 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 msgr2=0x7f5b0c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 0x7f5b0c0a3a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b24100bc0 tx=0x7f5b18009260 comp rx=0 tx=0).stop 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 msgr2=0x7f5b24074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400d980 tx=0x7f5b1400dd40 comp rx=0 tx=0).stop 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 shutdown_connections 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5b0c0a1550 0x7f5b0c0a3a10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b24102a40 0x7f5b240737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24100ea0 0x7f5b24074c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 --2- 172.21.15.31:0/1555706815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b24102080 0x7f5b240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 >> 172.21.15.31:0/1555706815 conn(0x7f5b240fc420 msgr2=0x7f5b241055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 shutdown_connections 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:22.492+0000 7f5b293ff700 1 -- 172.21.15.31:0/1555706815 wait complete. 2023-10-06T21:18:22.495 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2023-10-06T21:18:22.760 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:22 smithi031 systemd[1]: Starting Ceph mds.d for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:23.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:23 smithi031 ceph-mon[98539]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:23.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:23 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1555706815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:23.184 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:22 smithi031 podman[146287]: 2023-10-06T21:18:23.185 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:23 smithi031 bash[146287]: ad1280ec686ef3f019b661fc84314ce06944654b82f480d9b6c26c6bb0fa10bf 2023-10-06T21:18:23.185 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:23 smithi031 ceph-mds[146327]: -- 172.21.15.31:0/65981747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0f64471e0 con 0x55d0f641c800 2023-10-06T21:18:23.185 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-d[146303]: starting mds.d at 2023-10-06T21:18:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:23 smithi146 ceph-mon[102809]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:23 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1555706815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:23 smithi142 ceph-mon[102951]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1555706815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:23.510 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:18:23 smithi031 systemd[1]: Started Ceph mds.d for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: Deploying daemon mds.e on smithi142 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.31:6836/2454240095,v1:172.21.15.31:6837/2454240095] up:boot 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: fsmap 4 up:standby 2023-10-06T21:18:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:24 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:18:24.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: Deploying daemon mds.e on smithi142 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:24.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.31:6836/2454240095,v1:172.21.15.31:6837/2454240095] up:boot 2023-10-06T21:18:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: fsmap 4 up:standby 2023-10-06T21:18:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:24 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: Deploying daemon mds.e on smithi142 2023-10-06T21:18:24.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:24.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.31:6836/2454240095,v1:172.21.15.31:6837/2454240095] up:boot 2023-10-06T21:18:24.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: fsmap 4 up:standby 2023-10-06T21:18:24.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:24 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:18:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 -- 172.21.15.31:0/3773203368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 msgr2=0x7f7a5c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 --2- 172.21.15.31:0/3773203368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c109360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400b0a0 tx=0x7f7a5400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 -- 172.21.15.31:0/3773203368 shutdown_connections 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 --2- 172.21.15.31:0/3773203368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c109360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 --2- 172.21.15.31:0/3773203368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 --2- 172.21.15.31:0/3773203368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 -- 172.21.15.31:0/3773203368 >> 172.21.15.31:0/3773203368 conn(0x7f7a5c0fc420 msgr2=0x7f7a5c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 -- 172.21.15.31:0/3773203368 shutdown_connections 2023-10-06T21:18:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.732+0000 7f7a62fff700 1 -- 172.21.15.31:0/3773203368 wait complete. 2023-10-06T21:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.733+0000 7f7a62fff700 1 Processor -- start 2023-10-06T21:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.733+0000 7f7a62fff700 1 -- start start 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a08f0 con 0x7f7a5c102a40 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a0a60 con 0x7f7a5c102080 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a62fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a0c20 con 0x7f7a5c100ea0 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2023-10-06T21:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 -- 172.21.15.31:0/1228669774 learned_addr learned my addr 172.21.15.31:0/1228669774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a60d9b700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 -- 172.21.15.31:0/1228669774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 msgr2=0x7f7a5c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 -- 172.21.15.31:0/1228669774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 msgr2=0x7f7a5c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a6159c700 1 -- 172.21.15.31:0/1228669774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54009d00 con 0x7f7a5c102a40 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a60d9b700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.734+0000 7f7a5bfff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a6159c700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d010 tx=0x7f7a5400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400e040 con 0x7f7a5c102a40 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a54003e10 con 0x7f7a5c102a40 2023-10-06T21:18:24.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54005260 con 0x7f7a5c102a40 2023-10-06T21:18:24.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c1a0ea0 con 0x7f7a5c102a40 2023-10-06T21:18:24.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.735+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c1a1210 con 0x7f7a5c102a40 2023-10-06T21:18:24.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.736+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c10c5c0 con 0x7f7a5c102a40 2023-10-06T21:18:24.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.738+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7a5401da10 con 0x7f7a5c102a40 2023-10-06T21:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.739+0000 7f7a59ffb700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 0x7f7a4c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.740+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a540ce0a0 con 0x7f7a5c102a40 2023-10-06T21:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.740+0000 7f7a60d9b700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 0x7f7a4c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.740+0000 7f7a60d9b700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 0x7f7a4c0a3a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a50005f20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:24.742+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7a54097b90 con 0x7f7a5c102a40 2023-10-06T21:18:25.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.001+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a5c1a1400 con 0x7f7a5c102a40 2023-10-06T21:18:25.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.001+0000 7f7a59ffb700 1 -- 172.21.15.31:0/1228669774 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f7a5409b1b0 con 0x7f7a5c102a40 2023-10-06T21:18:25.003 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:25.003 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:18:25.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.003+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 msgr2=0x7f7a4c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:25.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.003+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 0x7f7a4c0a3a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a50005f20 comp rx=0 tx=0).stop 2023-10-06T21:18:25.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 msgr2=0x7f7a5c1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d010 tx=0x7f7a5400bcc0 comp rx=0 tx=0).stop 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 shutdown_connections 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7a4c0a1550 0x7f7a4c0a3a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c102a40 0x7f7a5c1a0320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a5c100ea0 0x7f7a5c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 --2- 172.21.15.31:0/1228669774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c102080 0x7f7a5c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 >> 172.21.15.31:0/1228669774 conn(0x7f7a5c0fc420 msgr2=0x7f7a5c1079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 shutdown_connections 2023-10-06T21:18:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:25.004+0000 7f7a62fff700 1 -- 172.21.15.31:0/1228669774 wait complete. 2023-10-06T21:18:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2023-10-06T21:18:25.594 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:25 smithi142 systemd[1]: Starting Ceph mds.e for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:26.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:25 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1228669774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:26.081 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:25 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1228669774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:25 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1228669774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:26.333 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:26 smithi142 podman[139896]: 2023-10-06T21:18:26.651 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mds[139936]: -- 172.21.15.142:0/3182386188 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac6d0d51e0 con 0x55ac6d0ab000 2023-10-06T21:18:26.651 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-e[139912]: starting mds.e at 2023-10-06T21:18:26.651 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:26 smithi142 bash[139896]: 86d74ae6dc858ece50d9d17669076c73c9e603e9e7f30cba6687162afbaab2de 2023-10-06T21:18:26.652 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:18:26 smithi142 systemd[1]: Started Ceph mds.e for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:27.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:27.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:27.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:27.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:27.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.095 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.096 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:27.096 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:27.096 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:27.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 -- 172.21.15.31:0/3619006742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102a40 msgr2=0x7f28d4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:27.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/3619006742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102a40 0x7f28d4109360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00b0a0 tx=0x7f28cc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 -- 172.21.15.31:0/3619006742 shutdown_connections 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/3619006742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102a40 0x7f28d4109360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/3619006742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4100ea0 0x7f28d41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/3619006742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4102080 0x7f28d4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 -- 172.21.15.31:0/3619006742 >> 172.21.15.31:0/3619006742 conn(0x7f28d40fc420 msgr2=0x7f28d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 -- 172.21.15.31:0/3619006742 shutdown_connections 2023-10-06T21:18:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.241+0000 7f28dc1ff700 1 -- 172.21.15.31:0/3619006742 wait complete. 2023-10-06T21:18:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.242+0000 7f28dc1ff700 1 Processor -- start 2023-10-06T21:18:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.242+0000 7f28dc1ff700 1 -- start start 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4100ea0 0x7f28d4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 0x7f28d41a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a08f0 con 0x7f28d4102080 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a0a60 con 0x7f28d4100ea0 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28dc1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a0c20 con 0x7f28d4102a40 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d9f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4100ea0 0x7f28d4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56308/0 (socket says 172.21.15.31:56308) 2023-10-06T21:18:27.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 -- 172.21.15.31:0/2154460707 learned_addr learned my addr 172.21.15.31:0/2154460707 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28da79c700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 0x7f28d41a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 -- 172.21.15.31:0/2154460707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 msgr2=0x7f28d41a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 0x7f28d41a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 -- 172.21.15.31:0/2154460707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4100ea0 msgr2=0x7f28d4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4100ea0 0x7f28d4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28d979a700 1 -- 172.21.15.31:0/2154460707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009d00 con 0x7f28d4102080 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.243+0000 7f28da79c700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 0x7f28d41a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28d979a700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e970 tx=0x7f28c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4009790 con 0x7f28d4102080 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c400eec0 con 0x7f28d4102080 2023-10-06T21:18:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d41a0f00 con 0x7f28d4102080 2023-10-06T21:18:27.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4003c10 con 0x7f28d4102080 2023-10-06T21:18:27.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.244+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d41a1380 con 0x7f28d4102080 2023-10-06T21:18:27.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.245+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d410c5c0 con 0x7f28d4102080 2023-10-06T21:18:27.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.247+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f28c4003db0 con 0x7f28d4102080 2023-10-06T21:18:27.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.249+0000 7f28caffd700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 0x7f28c00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.249+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28c4014070 con 0x7f28d4102080 2023-10-06T21:18:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.249+0000 7f28d9f9b700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 0x7f28c00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.249+0000 7f28d9f9b700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 0x7f28c00a3a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28d419a280 tx=0x7f28d000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.252+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f28c4096560 con 0x7f28d4102080 2023-10-06T21:18:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.507+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28d41a1800 con 0x7f28d4102080 2023-10-06T21:18:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.508+0000 7f28caffd700 1 -- 172.21.15.31:0/2154460707 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f28c4099b80 con 0x7f28d4102080 2023-10-06T21:18:27.510 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-06T21:18:27.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.510+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 msgr2=0x7f28c00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.510+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 0x7f28c00a3a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28d419a280 tx=0x7f28d000b410 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.510+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 msgr2=0x7f28d41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.510+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e970 tx=0x7f28c400ec80 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 shutdown_connections 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f28c00a1550 0x7f28c00a3a10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4102a40 0x7f28d41a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4100ea0 0x7f28d4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 --2- 172.21.15.31:0/2154460707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d4102080 0x7f28d41992f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 >> 172.21.15.31:0/2154460707 conn(0x7f28d40fc420 msgr2=0x7f28d41079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 shutdown_connections 2023-10-06T21:18:27.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:27.511+0000 7f28dc1ff700 1 -- 172.21.15.31:0/2154460707 wait complete. 2023-10-06T21:18:27.514 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2023-10-06T21:18:27.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:27 smithi146 ceph-mon[102809]: Deploying daemon mds.f on smithi146 2023-10-06T21:18:27.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:27 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:boot 2023-10-06T21:18:27.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:27 smithi146 ceph-mon[102809]: fsmap 5 up:standby 2023-10-06T21:18:27.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:27 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:18:27.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:27 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2154460707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:27 smithi031 ceph-mon[98539]: Deploying daemon mds.f on smithi146 2023-10-06T21:18:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:27 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:boot 2023-10-06T21:18:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:27 smithi031 ceph-mon[98539]: fsmap 5 up:standby 2023-10-06T21:18:28.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:27 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:18:28.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:27 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2154460707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:27 smithi142 ceph-mon[102951]: Deploying daemon mds.f on smithi146 2023-10-06T21:18:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:27 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:boot 2023-10-06T21:18:28.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:27 smithi142 ceph-mon[102951]: fsmap 5 up:standby 2023-10-06T21:18:28.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:27 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:18:28.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:27 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2154460707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:28.725 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:28 smithi146 systemd[1]: Starting Ceph mds.f for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:29.048 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:28 smithi146 ceph-mon[102809]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:29.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:28 smithi142 ceph-mon[102951]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:29.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:28 smithi031 ceph-mon[98539]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:29.480 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:29 smithi146 podman[139471]: 2023-10-06T21:18:29.480 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:29 smithi146 bash[139471]: 2edef506065cf622221ee54f4aa2fc30310f942999e82d3d71483e929553c2ee 2023-10-06T21:18:29.480 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:29 smithi146 ceph-mds[139511]: -- 172.21.15.146:0/2693801658 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c14810b1e0 con 0x55c1480e1000 2023-10-06T21:18:29.480 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: starting mds.f at 2023-10-06T21:18:29.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:29.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 -- 172.21.15.31:0/545851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420073910 msgr2=0x7fa420073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 --2- 172.21.15.31:0/545851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420073910 0x7fa420073d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa408009990 tx=0x7fa408009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 -- 172.21.15.31:0/545851925 shutdown_connections 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 --2- 172.21.15.31:0/545851925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 0x7fa42010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 --2- 172.21.15.31:0/545851925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420074ef0 0x7fa420073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 --2- 172.21.15.31:0/545851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420073910 0x7fa420073d90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.746+0000 7fa424bff700 1 -- 172.21.15.31:0/545851925 >> 172.21.15.31:0/545851925 conn(0x7fa4200fc420 msgr2=0x7fa4200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.747+0000 7fa424bff700 1 -- 172.21.15.31:0/545851925 shutdown_connections 2023-10-06T21:18:29.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.747+0000 7fa424bff700 1 -- 172.21.15.31:0/545851925 wait complete. 2023-10-06T21:18:29.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.747+0000 7fa424bff700 1 Processor -- start 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 -- start start 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420073910 0x7fa420104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 0x7fa420100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420101540 con 0x7fa420074ef0 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201016b0 con 0x7fa420073910 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa424bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420101870 con 0x7fa420108db0 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa41e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa41e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56328/0 (socket says 172.21.15.31:56328) 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa41e59c700 1 -- 172.21.15.31:0/2560639108 learned_addr learned my addr 172.21.15.31:0/2560639108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:29.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa41ed9d700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 0x7fa420100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.748+0000 7fa41659c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420073910 0x7fa420104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 -- 172.21.15.31:0/2560639108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 msgr2=0x7fa420100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 0x7fa420100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 -- 172.21.15.31:0/2560639108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420073910 msgr2=0x7fa420104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420073910 0x7fa420104210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 -- 172.21.15.31:0/2560639108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080095f0 con 0x7fa420074ef0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa41e59c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa408005910 tx=0x7fa408005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40801d070 con 0x7fa420074ef0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa40800bdb0 con 0x7fa420074ef0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201a5240 con 0x7fa420074ef0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408022d40 con 0x7fa420074ef0 2023-10-06T21:18:29.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.749+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201a56c0 con 0x7fa420074ef0 2023-10-06T21:18:29.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.750+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420107460 con 0x7fa420074ef0 2023-10-06T21:18:29.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.752+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa408022470 con 0x7fa420074ef0 2023-10-06T21:18:29.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.755+0000 7fa417fff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 0x7fa40c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:29.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.755+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa4080cddb0 con 0x7fa420074ef0 2023-10-06T21:18:29.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.755+0000 7fa41659c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 0x7fa40c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:29.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.756+0000 7fa41659c700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 0x7fa40c0a3a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa410009c60 tx=0x7fa410009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:29.759+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa4080978a0 con 0x7fa420074ef0 2023-10-06T21:18:29.798 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:18:29 smithi146 systemd[1]: Started Ceph mds.f for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.020+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa42010e7e0 con 0x7fa420074ef0 2023-10-06T21:18:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.021+0000 7fa417fff700 1 -- 172.21.15.31:0/2560639108 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fa408097020 con 0x7fa420074ef0 2023-10-06T21:18:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:30.024 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-06T21:18:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 msgr2=0x7fa40c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 0x7fa40c0a3a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa410009c60 tx=0x7fa410009260 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 msgr2=0x7fa420104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa408005910 tx=0x7fa408005740 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 shutdown_connections 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa40c0a1550 0x7fa40c0a3a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa420108db0 0x7fa420100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa420073910 0x7fa420104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 --2- 172.21.15.31:0/2560639108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa420074ef0 0x7fa420104750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.023+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 >> 172.21.15.31:0/2560639108 conn(0x7fa4200fc420 msgr2=0x7fa4200fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.024+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 shutdown_connections 2023-10-06T21:18:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:30.024+0000 7fa424bff700 1 -- 172.21.15.31:0/2560639108 wait complete. 2023-10-06T21:18:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: Deploying daemon mds.c on smithi146 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:boot 2023-10-06T21:18:30.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: fsmap 6 up:standby 2023-10-06T21:18:30.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:18:30.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:30.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:30 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2560639108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:30.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: Deploying daemon mds.c on smithi146 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:boot 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: fsmap 6 up:standby 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:18:30.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:30.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:30 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2560639108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:31.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:31.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:31.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:31.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: Deploying daemon mds.c on smithi146 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:boot 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: fsmap 6 up:standby 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:31.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:30 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2560639108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:32.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:32.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 -- 172.21.15.31:0/1631456646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174101a20 msgr2=0x7f3174103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:32.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 --2- 172.21.15.31:0/1631456646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174101a20 0x7f3174103e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3170009990 tx=0x7f3170009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 -- 172.21.15.31:0/1631456646 shutdown_connections 2023-10-06T21:18:32.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 --2- 172.21.15.31:0/1631456646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174106c80 0x7f3174109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 --2- 172.21.15.31:0/1631456646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174101a20 0x7f3174103e10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 --2- 172.21.15.31:0/1631456646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 0x7f3174106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 -- 172.21.15.31:0/1631456646 >> 172.21.15.31:0/1631456646 conn(0x7f31740fb3b0 msgr2=0x7f31740fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 -- 172.21.15.31:0/1631456646 shutdown_connections 2023-10-06T21:18:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.256+0000 7f317b7ff700 1 -- 172.21.15.31:0/1631456646 wait complete. 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.257+0000 7f317b7ff700 1 Processor -- start 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.257+0000 7f317b7ff700 1 -- start start 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.257+0000 7f317b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317b7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 0x7f3174196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174106c80 0x7f317419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317419e620 con 0x7f3174106c80 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317b7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317419e760 con 0x7f3174104350 2023-10-06T21:18:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317419e8f0 con 0x7f3174101a20 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36578/0 (socket says 172.21.15.31:36578) 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 -- 172.21.15.31:0/2296913635 learned_addr learned my addr 172.21.15.31:0/2296913635 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f3179d9c700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174106c80 0x7f317419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f3178d9a700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 0x7f3174196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 -- 172.21.15.31:0/2296913635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 msgr2=0x7f3174196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 0x7f3174196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 -- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174106c80 msgr2=0x7f317419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174106c80 0x7f317419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.258+0000 7f317959b700 1 -- 172.21.15.31:0/2296913635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700095f0 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f317959b700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3170005910 tx=0x7f3170005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317001d070 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317419eb70 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3170022470 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000f6c0 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.259+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317419eff0 con 0x7f3174101a20 2023-10-06T21:18:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.260+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317410c600 con 0x7f3174101a20 2023-10-06T21:18:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.262+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f31700225e0 con 0x7f3174101a20 2023-10-06T21:18:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.263+0000 7f316a7fc700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 0x7f31600a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.264+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31700ce510 con 0x7f3174101a20 2023-10-06T21:18:32.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.264+0000 7f3178d9a700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 0x7f31600a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:32.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.264+0000 7f3178d9a700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 0x7f31600a3990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3164005950 tx=0x7f31640058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.268+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3170098000 con 0x7f3174101a20 2023-10-06T21:18:32.299 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:31 smithi146 systemd[1]: Starting Ceph mds.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:32.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.524+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f317410c840 con 0x7f3174101a20 2023-10-06T21:18:32.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.525+0000 7f316a7fc700 1 -- 172.21.15.31:0/2296913635 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f317009b620 con 0x7f3174101a20 2023-10-06T21:18:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:32.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-06T21:18:32.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 msgr2=0x7f31600a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 0x7f31600a3990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3164005950 tx=0x7f31640058e0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 msgr2=0x7f3174196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3170005910 tx=0x7f3170005720 comp rx=0 tx=0).stop 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 shutdown_connections 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f31600a14d0 0x7f31600a3990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3174106c80 0x7f317419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3174101a20 0x7f3174196aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 --2- 172.21.15.31:0/2296913635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174104350 0x7f3174196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.527+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 >> 172.21.15.31:0/2296913635 conn(0x7f31740fb3b0 msgr2=0x7f3174102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.528+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 shutdown_connections 2023-10-06T21:18:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:32.528+0000 7f317b7ff700 1 -- 172.21.15.31:0/2296913635 wait complete. 2023-10-06T21:18:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2023-10-06T21:18:32.661 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:32 smithi146 podman[139819]: 2023-10-06T21:18:32.661 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mds[139860]: -- 172.21.15.146:0/404157983 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c1b1551e0 con 0x556c1b12a800 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-c[139835]: starting mds.c at 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:32 smithi146 bash[139819]: 4e3072c9ec4c706862888a2ef3e3c3b87e1d7af92a6a0dcb44d50c37f0c49e0c 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:18:32 smithi146 systemd[1]: Started Ceph mds.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2296913635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:33.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:33.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:33.050 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:32 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:33.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:33.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2296913635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:33.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:33.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:33.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:32 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2296913635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:32 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:33.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:33 smithi142 ceph-mon[102951]: Deploying daemon mds.h on smithi142 2023-10-06T21:18:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:33 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6836/824494679,v1:172.21.15.146:6837/824494679] up:boot 2023-10-06T21:18:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:33 smithi142 ceph-mon[102951]: fsmap 7 up:standby 2023-10-06T21:18:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:33 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:18:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:33 smithi031 ceph-mon[98539]: Deploying daemon mds.h on smithi142 2023-10-06T21:18:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:33 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6836/824494679,v1:172.21.15.146:6837/824494679] up:boot 2023-10-06T21:18:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:33 smithi031 ceph-mon[98539]: fsmap 7 up:standby 2023-10-06T21:18:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:33 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:18:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:33 smithi146 ceph-mon[102809]: Deploying daemon mds.h on smithi142 2023-10-06T21:18:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:33 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6836/824494679,v1:172.21.15.146:6837/824494679] up:boot 2023-10-06T21:18:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:33 smithi146 ceph-mon[102809]: fsmap 7 up:standby 2023-10-06T21:18:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:33 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:18:34.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 -- 172.21.15.31:0/2240369200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 msgr2=0x7fc6c8109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 --2- 172.21.15.31:0/2240369200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c8109070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000b0a0 tx=0x7fc6c000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 -- 172.21.15.31:0/2240369200 shutdown_connections 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 --2- 172.21.15.31:0/2240369200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c8109070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 --2- 172.21.15.31:0/2240369200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 --2- 172.21.15.31:0/2240369200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 0x7fc6c81066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 -- 172.21.15.31:0/2240369200 >> 172.21.15.31:0/2240369200 conn(0x7fc6c80fb3b0 msgr2=0x7fc6c80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 -- 172.21.15.31:0/2240369200 shutdown_connections 2023-10-06T21:18:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.761+0000 7fc6cddff700 1 -- 172.21.15.31:0/2240369200 wait complete. 2023-10-06T21:18:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.762+0000 7fc6cddff700 1 Processor -- start 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.762+0000 7fc6cddff700 1 -- start start 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 0x7fc6c81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a08f0 con 0x7fc6c8106c00 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a0a60 con 0x7fc6c81019a0 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6cddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a0c20 con 0x7fc6c81042d0 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33986/0 (socket says 172.21.15.31:33986) 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6c77fe700 1 -- 172.21.15.31:0/385274774 learned_addr learned my addr 172.21.15.31:0/385274774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.763+0000 7fc6c6ffd700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 0x7fc6c81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 -- 172.21.15.31:0/385274774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 msgr2=0x7fc6c81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 0x7fc6c81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 -- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 msgr2=0x7fc6c81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 -- 172.21.15.31:0/385274774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8009520 con 0x7fc6c81019a0 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c7fff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.764+0000 7fc6c77fe700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800eb30 tx=0x7fc6b800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.766+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8009880 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.766+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6b80044b0 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.766+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0009d00 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.766+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8003ce0 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.766+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c81a1210 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.767+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c8108780 con 0x7fc6c81019a0 2023-10-06T21:18:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.769+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc6b8003e40 con 0x7fc6c81019a0 2023-10-06T21:18:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.770+0000 7fc6c4ff9700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 0x7fc6b00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.771+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6b8014070 con 0x7fc6c81019a0 2023-10-06T21:18:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.771+0000 7fc6c6ffd700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 0x7fc6b00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.771+0000 7fc6c6ffd700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 0x7fc6b00a3a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005950 tx=0x7fc6bc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:34.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:34.775+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc6b80967b0 con 0x7fc6c81019a0 2023-10-06T21:18:34.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:34 smithi142 ceph-mon[102951]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:35.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:34 smithi031 ceph-mon[98539]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.037+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c810c5b0 con 0x7fc6c81019a0 2023-10-06T21:18:35.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.037+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/385274774 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fc6b8099dd0 con 0x7fc6c81019a0 2023-10-06T21:18:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:35.041 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 msgr2=0x7fc6b00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 0x7fc6b00a3a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005950 tx=0x7fc6bc00b410 comp rx=0 tx=0).stop 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 msgr2=0x7fc6c8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800eb30 tx=0x7fc6b800ee40 comp rx=0 tx=0).stop 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 shutdown_connections 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc6b00a1550 0x7fc6b00a3a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c8106c00 0x7fc6c81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6c81019a0 0x7fc6c8198db0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 --2- 172.21.15.31:0/385274774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c81042d0 0x7fc6c81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 >> 172.21.15.31:0/385274774 conn(0x7fc6c80fb3b0 msgr2=0x7fc6c8102630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 shutdown_connections 2023-10-06T21:18:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:35.040+0000 7fc6cddff700 1 -- 172.21.15.31:0/385274774 wait complete. 2023-10-06T21:18:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T21:18:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:34 smithi146 ceph-mon[102809]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:35.401 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:34 smithi142 systemd[1]: Starting Ceph mds.h for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:35.664 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:35 smithi142 podman[140238]: 2023-10-06T21:18:36.151 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mds[140278]: -- 172.21.15.142:0/514312446 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55759dd571e0 con 0x55759dd2d000 2023-10-06T21:18:36.151 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: starting mds.h at 2023-10-06T21:18:36.151 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:35 smithi142 bash[140238]: f27725d1278e85f723257b0b751a5e86fd0c9595ecb848a0b044c631ce88fbaf 2023-10-06T21:18:36.151 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:18:35 smithi142 systemd[1]: Started Ceph mds.h for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/385274774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:36.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:35 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/385274774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:36.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:36.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:35 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/385274774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:35 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:36 smithi146 ceph-mon[102809]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:36 smithi146 ceph-mon[102809]: Deploying daemon mds.i on smithi146 2023-10-06T21:18:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:36 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6836/1999670279,v1:172.21.15.142:6837/1999670279] up:boot 2023-10-06T21:18:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:36 smithi146 ceph-mon[102809]: fsmap 8 up:standby 2023-10-06T21:18:37.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:36 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:18:37.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:37.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:36 smithi031 ceph-mon[98539]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:37.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:36 smithi031 ceph-mon[98539]: Deploying daemon mds.i on smithi146 2023-10-06T21:18:37.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:36 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6836/1999670279,v1:172.21.15.142:6837/1999670279] up:boot 2023-10-06T21:18:37.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:36 smithi031 ceph-mon[98539]: fsmap 8 up:standby 2023-10-06T21:18:37.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:36 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:18:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:36 smithi142 ceph-mon[102951]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:36 smithi142 ceph-mon[102951]: Deploying daemon mds.i on smithi146 2023-10-06T21:18:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:36 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6836/1999670279,v1:172.21.15.142:6837/1999670279] up:boot 2023-10-06T21:18:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:36 smithi142 ceph-mon[102951]: fsmap 8 up:standby 2023-10-06T21:18:37.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:36 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:18:37.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 -- 172.21.15.31:0/3409064542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 msgr2=0x7ff7b01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/3409064542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b01012c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009990 tx=0x7ff7a0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 -- 172.21.15.31:0/3409064542 shutdown_connections 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/3409064542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 0x7ff7b0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/3409064542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b01012c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/3409064542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 0x7ff7b0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 -- 172.21.15.31:0/3409064542 >> 172.21.15.31:0/3409064542 conn(0x7ff7b00fc420 msgr2=0x7ff7b00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 -- 172.21.15.31:0/3409064542 shutdown_connections 2023-10-06T21:18:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.274+0000 7ff7b57ff700 1 -- 172.21.15.31:0/3409064542 wait complete. 2023-10-06T21:18:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.275+0000 7ff7b57ff700 1 Processor -- start 2023-10-06T21:18:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.275+0000 7ff7b57ff700 1 -- start start 2023-10-06T21:18:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 0x7ff7b0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 0x7ff7b01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b01a0850 con 0x7ff7b0100ea0 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b01a09c0 con 0x7ff7b0102a40 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7b57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b01a0b80 con 0x7ff7b0102080 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41064/0 (socket says 172.21.15.31:41064) 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2879093367 learned_addr learned my addr 172.21.15.31:0/2879093367 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 0x7ff7b0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7af7fe700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 0x7ff7b01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2879093367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 msgr2=0x7ff7b0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 0x7ff7b0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2879093367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 msgr2=0x7ff7b01a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 0x7ff7b01a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.276+0000 7ff7aeffd700 1 -- 172.21.15.31:0/2879093367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00095f0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff7aeffd700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0004730 tx=0x7ff7a0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a001d070 con 0x7ff7b0100ea0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7a0004cc0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b01a0e00 con 0x7ff7b0100ea0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0005280 con 0x7ff7b0100ea0 2023-10-06T21:18:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.277+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b01a11c0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.278+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b0193400 con 0x7ff7b0100ea0 2023-10-06T21:18:37.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.280+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff7a0003b70 con 0x7ff7b0100ea0 2023-10-06T21:18:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.281+0000 7ff797fff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 0x7ff7980a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.282+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7a00cdab0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.282+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 0x7ff7980a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.282+0000 7ff7ae7fc700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 0x7ff7980a3990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0100bc0 tx=0x7ff7a400a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:37.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.286+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff7a00975a0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.544+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7b010c610 con 0x7ff7b0100ea0 2023-10-06T21:18:37.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.544+0000 7ff797fff700 1 -- 172.21.15.31:0/2879093367 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7ff7a009abc0 con 0x7ff7b0100ea0 2023-10-06T21:18:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:37.548 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 msgr2=0x7ff7980a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 0x7ff7980a3990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0100bc0 tx=0x7ff7a400a3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 msgr2=0x7ff7b0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0004730 tx=0x7ff7a0004760 comp rx=0 tx=0).stop 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 shutdown_connections 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ff7980a14d0 0x7ff7980a3990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b0102a40 0x7ff7b01a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b0100ea0 0x7ff7b0198d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 --2- 172.21.15.31:0/2879093367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b0102080 0x7ff7b0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 >> 172.21.15.31:0/2879093367 conn(0x7ff7b00fc420 msgr2=0x7ff7b0107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 shutdown_connections 2023-10-06T21:18:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:37.547+0000 7ff7b57ff700 1 -- 172.21.15.31:0/2879093367 wait complete. 2023-10-06T21:18:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2023-10-06T21:18:38.092 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:37 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2879093367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:38.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:37 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2879093367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:37 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2879093367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:38.549 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 systemd[1]: Starting Ceph mds.i for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:38.878 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 podman[140165]: 2023-10-06T21:18:38.878 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 bash[140165]: baf3c43e16a4d921fe4b05badfabe7ae9dc916a6fb596b53ff7a8bfc1b8b7432 2023-10-06T21:18:38.878 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 ceph-mds[140205]: -- 172.21.15.146:0/4104289150 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559723e2f1e0 con 0x559723e05000 2023-10-06T21:18:38.878 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-i[140181]: starting mds.i at 2023-10-06T21:18:39.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:38 smithi142 ceph-mon[102951]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:39.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:38 smithi031 ceph-mon[98539]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:38 smithi146 ceph-mon[102809]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:39.299 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:18:38 smithi146 systemd[1]: Started Ceph mds.i for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:39.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:39.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 -- 172.21.15.31:0/546276437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4101940 msgr2=0x7febb4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:39.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 --2- 172.21.15.31:0/546276437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4101940 0x7febb4103d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feba8009990 tx=0x7feba8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 -- 172.21.15.31:0/546276437 shutdown_connections 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 --2- 172.21.15.31:0/546276437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4106ba0 0x7febb4109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 --2- 172.21.15.31:0/546276437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4101940 0x7febb4103d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 --2- 172.21.15.31:0/546276437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4104270 0x7febb4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 -- 172.21.15.31:0/546276437 >> 172.21.15.31:0/546276437 conn(0x7febb40fb310 msgr2=0x7febb40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 -- 172.21.15.31:0/546276437 shutdown_connections 2023-10-06T21:18:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.789+0000 7febbb1ff700 1 -- 172.21.15.31:0/546276437 wait complete. 2023-10-06T21:18:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.790+0000 7febbb1ff700 1 Processor -- start 2023-10-06T21:18:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.790+0000 7febbb1ff700 1 -- start start 2023-10-06T21:18:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 0x7febb4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4106ba0 0x7febb419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb419e640 con 0x7febb4104270 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb419e780 con 0x7febb4106ba0 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febbb1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb419e910 con 0x7febb4101940 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41086/0 (socket says 172.21.15.31:41086) 2023-10-06T21:18:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 -- 172.21.15.31:0/2744223862 learned_addr learned my addr 172.21.15.31:0/2744223862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb8f9b700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 0x7febb4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb979c700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4106ba0 0x7febb419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 -- 172.21.15.31:0/2744223862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 msgr2=0x7febb4196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 0x7febb4196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 -- 172.21.15.31:0/2744223862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4106ba0 msgr2=0x7febb419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4106ba0 0x7febb419e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb3fff700 1 -- 172.21.15.31:0/2744223862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80095f0 con 0x7febb4104270 2023-10-06T21:18:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.791+0000 7febb8f9b700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 0x7febb4196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febb3fff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feba000d980 tx=0x7feba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0004220 con 0x7febb4104270 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba0007640 con 0x7febb4104270 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb419ebf0 con 0x7febb4104270 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0005660 con 0x7febb4104270 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.792+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb419f150 con 0x7febb4104270 2023-10-06T21:18:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.793+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb40fce90 con 0x7febb4104270 2023-10-06T21:18:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.795+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feba0004750 con 0x7febb4104270 2023-10-06T21:18:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.797+0000 7febb1ffb700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 0x7feba40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.797+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feba00cc630 con 0x7febb4104270 2023-10-06T21:18:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.798+0000 7febb8f9b700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 0x7feba40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.798+0000 7febb8f9b700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 0x7feba40a3a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:39.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:39.802+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feba0096090 con 0x7febb4104270 2023-10-06T21:18:40.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.075+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb410c5a0 con 0x7febb4104270 2023-10-06T21:18:40.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.076+0000 7febb1ffb700 1 -- 172.21.15.31:0/2744223862 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7feba00996b0 con 0x7febb4104270 2023-10-06T21:18:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:40.079 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T21:18:40.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.078+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 msgr2=0x7feba40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.078+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 0x7feba40a3a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba800b260 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.078+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 msgr2=0x7febb4197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.078+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feba000d980 tx=0x7feba000dc90 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 shutdown_connections 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feba40a1550 0x7feba40a3a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febb4106ba0 0x7febb419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febb4101940 0x7febb4196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 --2- 172.21.15.31:0/2744223862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb4104270 0x7febb4197000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 >> 172.21.15.31:0/2744223862 conn(0x7febb40fb310 msgr2=0x7febb41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:40.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 shutdown_connections 2023-10-06T21:18:40.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:40.079+0000 7febbb1ff700 1 -- 172.21.15.31:0/2744223862 wait complete. 2023-10-06T21:18:40.083 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:40.229 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: Deploying daemon mds.g on smithi031 2023-10-06T21:18:40.230 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:boot 2023-10-06T21:18:40.230 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: fsmap 9 up:standby 2023-10-06T21:18:40.230 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:39 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: Deploying daemon mds.g on smithi031 2023-10-06T21:18:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:boot 2023-10-06T21:18:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: fsmap 9 up:standby 2023-10-06T21:18:40.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:39 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:18:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: Deploying daemon mds.g on smithi031 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:boot 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: fsmap 9 up:standby 2023-10-06T21:18:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:39 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:18:41.170 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:40 smithi031 ceph-mon[98539]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:41.170 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2744223862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:40 smithi146 ceph-mon[102809]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:40 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2744223862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:40 smithi142 ceph-mon[102951]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:40 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2744223862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:41.510 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 systemd[1]: Starting Ceph mds.g for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:41.893 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 podman[146946]: 2023-10-06T21:18:42.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:42.170 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 bash[146946]: b488988d89df5be413d5deed807e621294a2e79d743f554719be5c7ebb442988 2023-10-06T21:18:42.170 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 ceph-mds[146986]: -- 172.21.15.31:0/15045183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557139bef1e0 con 0x557139bc5000 2023-10-06T21:18:42.170 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-g[146962]: starting mds.g at 2023-10-06T21:18:42.170 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:18:41 smithi031 systemd[1]: Started Ceph mds.g for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3751337444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4074ef0 msgr2=0x7f63b4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3751337444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4074ef0 0x7f63b4073340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009990 tx=0x7f63a8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3751337444 shutdown_connections 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3751337444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4108db0 0x7f63b410b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3751337444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4074ef0 0x7f63b4073340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3751337444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4073910 0x7f63b4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3751337444 >> 172.21.15.31:0/3751337444 conn(0x7f63b40fc420 msgr2=0x7f63b40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:42.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3751337444 shutdown_connections 2023-10-06T21:18:42.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.313+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3751337444 wait complete. 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 Processor -- start 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 -- start start 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4073910 0x7f63b4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4074ef0 0x7f63b4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41a0860 con 0x7f63b4108db0 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41a09d0 con 0x7f63b4074ef0 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.314+0000 7f63bb1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41a0b90 con 0x7f63b4073910 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41120/0 (socket says 172.21.15.31:41120) 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 -- 172.21.15.31:0/3750956346 learned_addr learned my addr 172.21.15.31:0/3750956346 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b3fff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4074ef0 0x7f63b4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b8f9b700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4073910 0x7f63b4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 -- 172.21.15.31:0/3750956346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4073910 msgr2=0x7f63b4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4073910 0x7f63b4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 -- 172.21.15.31:0/3750956346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4074ef0 msgr2=0x7f63b4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4074ef0 0x7f63b4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 -- 172.21.15.31:0/3750956346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80095f0 con 0x7f63b4108db0 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.315+0000 7f63b979c700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63ac007ee0 tx=0x7f63ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.316+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac017070 con 0x7f63b4108db0 2023-10-06T21:18:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.316+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63ac00f040 con 0x7f63b4108db0 2023-10-06T21:18:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.316+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b41a0e70 con 0x7f63b4108db0 2023-10-06T21:18:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.316+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac005570 con 0x7f63b4108db0 2023-10-06T21:18:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.316+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41a1290 con 0x7f63b4108db0 2023-10-06T21:18:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.317+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4107460 con 0x7f63b4108db0 2023-10-06T21:18:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.319+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f63ac00db20 con 0x7f63b4108db0 2023-10-06T21:18:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.321+0000 7f63b1ffb700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 0x7f63a40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.321+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63ac0cbfc0 con 0x7f63b4108db0 2023-10-06T21:18:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.321+0000 7f63b8f9b700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 0x7f63a40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.322+0000 7f63b8f9b700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 0x7f63a40a3a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:42.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.325+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f63ac095a20 con 0x7f63b4108db0 2023-10-06T21:18:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.614+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63b410e7e0 con 0x7f63b4108db0 2023-10-06T21:18:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.614+0000 7f63b1ffb700 1 -- 172.21.15.31:0/3750956346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f63ac099040 con 0x7f63b4108db0 2023-10-06T21:18:42.617 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:42.618 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T21:18:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 msgr2=0x7f63a40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 0x7f63a40a3a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a800b260 comp rx=0 tx=0).stop 2023-10-06T21:18:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 msgr2=0x7f63b41a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63ac007ee0 tx=0x7f63ac00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 shutdown_connections 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f63a40a1550 0x7f63a40a3a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63b4108db0 0x7f63b41a0290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63b4073910 0x7f63b4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 --2- 172.21.15.31:0/3750956346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63b4074ef0 0x7f63b4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 >> 172.21.15.31:0/3750956346 conn(0x7f63b40fc420 msgr2=0x7f63b40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.617+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 shutdown_connections 2023-10-06T21:18:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:42.618+0000 7f63bb1ff700 1 -- 172.21.15.31:0/3750956346 wait complete. 2023-10-06T21:18:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2023-10-06T21:18:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.31:6838/1272583602,v1:172.21.15.31:6839/1272583602] up:boot 2023-10-06T21:18:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: fsmap 10 up:standby 2023-10-06T21:18:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: Deploying daemon mds.a on smithi031 2023-10-06T21:18:43.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3750956346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.31:6838/1272583602,v1:172.21.15.31:6839/1272583602] up:boot 2023-10-06T21:18:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: fsmap 10 up:standby 2023-10-06T21:18:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: Deploying daemon mds.a on smithi031 2023-10-06T21:18:43.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3750956346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.31:6838/1272583602,v1:172.21.15.31:6839/1272583602] up:boot 2023-10-06T21:18:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: fsmap 10 up:standby 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: Deploying daemon mds.a on smithi031 2023-10-06T21:18:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3750956346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:44.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:44.713 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:44 smithi031 systemd[1]: Starting Ceph mds.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 -- 172.21.15.31:0/2726829346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 msgr2=0x7fec3c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 --2- 172.21.15.31:0/2726829346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c102500 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec30009990 tx=0x7fec30009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 -- 172.21.15.31:0/2726829346 shutdown_connections 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 --2- 172.21.15.31:0/2726829346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c102a40 0x7fec3c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 --2- 172.21.15.31:0/2726829346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c100ea0 0x7fec3c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 --2- 172.21.15.31:0/2726829346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c102500 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 -- 172.21.15.31:0/2726829346 >> 172.21.15.31:0/2726829346 conn(0x7fec3c0fc420 msgr2=0x7fec3c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 -- 172.21.15.31:0/2726829346 shutdown_connections 2023-10-06T21:18:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.866+0000 7fec423ff700 1 -- 172.21.15.31:0/2726829346 wait complete. 2023-10-06T21:18:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.867+0000 7fec423ff700 1 Processor -- start 2023-10-06T21:18:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.867+0000 7fec423ff700 1 -- start start 2023-10-06T21:18:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c100ea0 0x7fec3c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c102a40 0x7fec3c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c19e6c0 con 0x7fec3c102080 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c19e800 con 0x7fec3c102a40 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec423ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c19e990 con 0x7fec3c100ea0 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48276/0 (socket says 172.21.15.31:48276) 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 -- 172.21.15.31:0/523005031 learned_addr learned my addr 172.21.15.31:0/523005031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec4099c700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c102a40 0x7fec3c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3bfff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c100ea0 0x7fec3c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 -- 172.21.15.31:0/523005031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c100ea0 msgr2=0x7fec3c196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c100ea0 0x7fec3c196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 -- 172.21.15.31:0/523005031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c102a40 msgr2=0x7fec3c19e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c102a40 0x7fec3c19e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.868+0000 7fec3b7fe700 1 -- 172.21.15.31:0/523005031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300095f0 con 0x7fec3c102080 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec3b7fe700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec300095c0 tx=0x7fec30005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3001d070 con 0x7fec3c102080 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c19ec10 con 0x7fec3c102080 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec30004000 con 0x7fec3c102080 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30022bd0 con 0x7fec3c102080 2023-10-06T21:18:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.869+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c19f030 con 0x7fec3c102080 2023-10-06T21:18:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.870+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c10c610 con 0x7fec3c102080 2023-10-06T21:18:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.872+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fec30003b70 con 0x7fec3c102080 2023-10-06T21:18:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.873+0000 7fec397fa700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 0x7fec240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.874+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec300cdf10 con 0x7fec3c102080 2023-10-06T21:18:44.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.874+0000 7fec3bfff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 0x7fec240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:44.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.874+0000 7fec3bfff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 0x7fec240a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005fd0 tx=0x7fec2c005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:44.878+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec30097a00 con 0x7fec3c102080 2023-10-06T21:18:45.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:44 smithi031 ceph-mon[98539]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:45.010 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:44 smithi031 podman[147333]: 2023-10-06T21:18:45.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:44 smithi146 ceph-mon[102809]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:44 smithi142 ceph-mon[102951]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.158+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec3c107ec0 con 0x7fec3c102080 2023-10-06T21:18:45.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.158+0000 7fec397fa700 1 -- 172.21.15.31:0/523005031 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7fec3009b020 con 0x7fec3c102080 2023-10-06T21:18:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:45.161 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T21:18:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 msgr2=0x7fec240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 0x7fec240a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005fd0 tx=0x7fec2c005f60 comp rx=0 tx=0).stop 2023-10-06T21:18:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 msgr2=0x7fec3c197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec300095c0 tx=0x7fec30005740 comp rx=0 tx=0).stop 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 shutdown_connections 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fec240a14d0 0x7fec240a3990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec3c102a40 0x7fec3c19e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c100ea0 0x7fec3c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 --2- 172.21.15.31:0/523005031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec3c102080 0x7fec3c197080 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 >> 172.21.15.31:0/523005031 conn(0x7fec3c0fc420 msgr2=0x7fec3c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 shutdown_connections 2023-10-06T21:18:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:45.161+0000 7fec423ff700 1 -- 172.21.15.31:0/523005031 wait complete. 2023-10-06T21:18:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2023-10-06T21:18:45.510 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:45 smithi031 bash[147333]: a2f388d69f85a031310342ca6f11d6ab8f2023ec31b948be5fc2613423e5eab0 2023-10-06T21:18:45.510 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mds[147416]: -- 172.21.15.31:0/1388633032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ff870f31e0 con 0x55ff870c9000 2023-10-06T21:18:45.510 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-a[147392]: starting mds.a at 2023-10-06T21:18:45.510 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:18:45 smithi031 systemd[1]: Started Ceph mds.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/523005031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:46.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:45 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/523005031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:46.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:46.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:45 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/523005031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:18:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:45 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:46 smithi146 ceph-mon[102809]: Deploying daemon mds.b on smithi142 2023-10-06T21:18:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:46 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.31:6840/847340369,v1:172.21.15.31:6841/847340369] up:boot 2023-10-06T21:18:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:46 smithi146 ceph-mon[102809]: fsmap 11 up:standby 2023-10-06T21:18:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:46 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:18:47.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:46 smithi146 ceph-mon[102809]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:46 smithi142 ceph-mon[102951]: Deploying daemon mds.b on smithi142 2023-10-06T21:18:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:46 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.31:6840/847340369,v1:172.21.15.31:6841/847340369] up:boot 2023-10-06T21:18:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:46 smithi142 ceph-mon[102951]: fsmap 11 up:standby 2023-10-06T21:18:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:46 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:18:47.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:46 smithi142 ceph-mon[102951]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:47.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:47.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:46 smithi031 ceph-mon[98539]: Deploying daemon mds.b on smithi142 2023-10-06T21:18:47.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:46 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.31:6840/847340369,v1:172.21.15.31:6841/847340369] up:boot 2023-10-06T21:18:47.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:46 smithi031 ceph-mon[98539]: fsmap 11 up:standby 2023-10-06T21:18:47.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:46 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:18:47.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:46 smithi031 ceph-mon[98539]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 -- 172.21.15.31:0/2091072255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100e90 msgr2=0x7febe01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 --2- 172.21.15.31:0/2091072255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100e90 0x7febe01012b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7febd4009990 tx=0x7febd4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 -- 172.21.15.31:0/2091072255 shutdown_connections 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 --2- 172.21.15.31:0/2091072255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102a30 0x7febe0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 --2- 172.21.15.31:0/2091072255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0100e90 0x7febe01012b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 --2- 172.21.15.31:0/2091072255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0102070 0x7febe01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 -- 172.21.15.31:0/2091072255 >> 172.21.15.31:0/2091072255 conn(0x7febe00fc430 msgr2=0x7febe00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.398+0000 7febe71ff700 1 -- 172.21.15.31:0/2091072255 shutdown_connections 2023-10-06T21:18:47.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.399+0000 7febe71ff700 1 -- 172.21.15.31:0/2091072255 wait complete. 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.399+0000 7febe71ff700 1 Processor -- start 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 -- start start 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 0x7febe0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 0x7febe00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0073f80 con 0x7febe0102a30 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00740f0 con 0x7febe0100e90 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe71ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01a0b90 con 0x7febe0102070 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe4f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 0x7febe0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:47.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48310/0 (socket says 172.21.15.31:48310) 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe4f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 0x7febe0074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45994/0 (socket says 172.21.15.31:45994) 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.400+0000 7febe579c700 1 -- 172.21.15.31:0/1370717023 learned_addr learned my addr 172.21.15.31:0/1370717023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febdffff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 0x7febe00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 -- 172.21.15.31:0/1370717023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 msgr2=0x7febe00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 0x7febe00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 -- 172.21.15.31:0/1370717023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 msgr2=0x7febe0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 0x7febe0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 -- 172.21.15.31:0/1370717023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40095f0 con 0x7febe0102a30 2023-10-06T21:18:47.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febdffff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 0x7febe00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe579c700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7febd8007ee0 tx=0x7febd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8017070 con 0x7febe0102a30 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febd800f040 con 0x7febe0102a30 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe01a0e70 con 0x7febe0102a30 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8005710 con 0x7febe0102a30 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.401+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe01a12a0 con 0x7febe0102a30 2023-10-06T21:18:47.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.402+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe010c600 con 0x7febe0102a30 2023-10-06T21:18:47.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.405+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7febd8005870 con 0x7febe0102a30 2023-10-06T21:18:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.406+0000 7febddffb700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 0x7febd00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.406+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7febd80cc760 con 0x7febe0102a30 2023-10-06T21:18:47.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.406+0000 7febe4f9b700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 0x7febd00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:47.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.407+0000 7febe4f9b700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 0x7febd00a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7febd40098e0 tx=0x7febd400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:47.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.408+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7febd80961c0 con 0x7febe0102a30 2023-10-06T21:18:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.665+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febe01a1490 con 0x7febe0102a30 2023-10-06T21:18:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.666+0000 7febddffb700 1 -- 172.21.15.31:0/1370717023 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7febd80997e0 con 0x7febe0102a30 2023-10-06T21:18:47.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:47.669 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.668+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 msgr2=0x7febd00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.668+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 0x7febd00a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7febd40098e0 tx=0x7febd400b260 comp rx=0 tx=0).stop 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 msgr2=0x7febe00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7febd8007ee0 tx=0x7febd800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 shutdown_connections 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7febd00a14d0 0x7febd00a3990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0102a30 0x7febe00737e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe0100e90 0x7febe0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 --2- 172.21.15.31:0/1370717023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febe0102070 0x7febe00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 >> 172.21.15.31:0/1370717023 conn(0x7febe00fc430 msgr2=0x7febe00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 shutdown_connections 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:47.669+0000 7febe71ff700 1 -- 172.21.15.31:0/1370717023 wait complete. 2023-10-06T21:18:47.673 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2023-10-06T21:18:47.793 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:47 smithi142 systemd[1]: Starting Ceph mds.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:18:48.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:47 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1370717023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:48.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:47 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1370717023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:48.111 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:47 smithi142 podman[140581]: 2023-10-06T21:18:48.111 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:47 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1370717023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:48.401 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:48 smithi142 bash[140581]: 1c6624c21e611baf236df4f40d1a603013b6453ad9b29b8c3effcea2197da5db 2023-10-06T21:18:48.401 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mds[140621]: -- 172.21.15.142:0/3868493829 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c75a4791e0 con 0x55c75a44e800 2023-10-06T21:18:48.402 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b[140597]: starting mds.b at 2023-10-06T21:18:48.402 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:48 smithi142 systemd[1]: Started Ceph mds.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.049 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:48 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:48 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:48 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:49.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:18:49.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 -- 172.21.15.31:0/2175647696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 msgr2=0x7fd5ac101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/2175647696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac101410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd598009990 tx=0x7fd598009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 -- 172.21.15.31:0/2175647696 shutdown_connections 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/2175647696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/2175647696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/2175647696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac101410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 -- 172.21.15.31:0/2175647696 >> 172.21.15.31:0/2175647696 conn(0x7fd5ac0fb310 msgr2=0x7fd5ac0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 -- 172.21.15.31:0/2175647696 shutdown_connections 2023-10-06T21:18:49.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.904+0000 7fd5b31ff700 1 -- 172.21.15.31:0/2175647696 wait complete. 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 Processor -- start 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 -- start start 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac19e5f0 con 0x7fd5ac101950 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac19e730 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.906+0000 7fd5b31ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac19e8c0 con 0x7fd5ac100fb0 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:46012/0 (socket says 172.21.15.31:46012) 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 -- 172.21.15.31:0/3833966489 learned_addr learned my addr 172.21.15.31:0/3833966489 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5abfff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 -- 172.21.15.31:0/3833966489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 msgr2=0x7fd5ac196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 -- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 msgr2=0x7fd5ac19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 -- 172.21.15.31:0/3833966489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980095f0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b179c700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.907+0000 7fd5b0f9b700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e970 tx=0x7fd5a000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.908+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0009790 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.908+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac19eba0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.908+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5a000c7e0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.908+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0003bd0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.908+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac19f100 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.909+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac10b4d0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.912+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd5a0003d50 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.913+0000 7fd5a9ffb700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 0x7fd59c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.913+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd5a0014070 con 0x7fd5ac0ffdc0 2023-10-06T21:18:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.913+0000 7fd5abfff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 0x7fd59c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.914+0000 7fd5abfff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 0x7fd59c0a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b2e0 tx=0x7fd598005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:49.917+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5a0096460 con 0x7fd5ac0ffdc0 2023-10-06T21:18:50.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:49 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:boot 2023-10-06T21:18:50.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:49 smithi031 ceph-mon[98539]: fsmap 12 up:standby 2023-10-06T21:18:50.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:49 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:18:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:49 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:boot 2023-10-06T21:18:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:49 smithi142 ceph-mon[102951]: fsmap 12 up:standby 2023-10-06T21:18:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:49 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:18:50.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.193+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5ac10b6c0 con 0x7fd5ac0ffdc0 2023-10-06T21:18:50.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.194+0000 7fd5a9ffb700 1 -- 172.21.15.31:0/3833966489 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7fd5a0099a80 con 0x7fd5ac0ffdc0 2023-10-06T21:18:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:50.197 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-06T21:18:50.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 msgr2=0x7fd59c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 0x7fd59c0a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b2e0 tx=0x7fd598005fb0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 msgr2=0x7fd5ac196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e970 tx=0x7fd5a000ec80 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 shutdown_connections 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd59c0a1550 0x7fd59c0a3a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ac101950 0x7fd5ac19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5ac0ffdc0 0x7fd5ac196a70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 --2- 172.21.15.31:0/3833966489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac100fb0 0x7fd5ac196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.196+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 >> 172.21.15.31:0/3833966489 conn(0x7fd5ac0fb310 msgr2=0x7fd5ac0680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.197+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 shutdown_connections 2023-10-06T21:18:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.197+0000 7fd5b31ff700 1 -- 172.21.15.31:0/3833966489 wait complete. 2023-10-06T21:18:50.201 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2023-10-06T21:18:50.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:18:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:49 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:boot 2023-10-06T21:18:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:49 smithi146 ceph-mon[102809]: fsmap 12 up:standby 2023-10-06T21:18:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:49 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:18:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3053490298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4073910 msgr2=0x7f61e4073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3053490298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4073910 0x7f61e4073d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009990 tx=0x7f61d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3053490298 shutdown_connections 2023-10-06T21:18:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3053490298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4108db0 0x7f61e410b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3053490298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 0x7f61e4073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3053490298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4073910 0x7f61e4073d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.439+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3053490298 >> 172.21.15.31:0/3053490298 conn(0x7f61e40fc420 msgr2=0x7f61e40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.440+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3053490298 shutdown_connections 2023-10-06T21:18:50.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.440+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3053490298 wait complete. 2023-10-06T21:18:50.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 Processor -- start 2023-10-06T21:18:50.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 -- start start 2023-10-06T21:18:50.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 0x7f61e4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 0x7f61e4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a0870 con 0x7f61e4108db0 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a09e0 con 0x7f61e4074ef0 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ec3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a0ba0 con 0x7f61e4073910 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ea99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ea99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48346/0 (socket says 172.21.15.31:48346) 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ea99c700 1 -- 172.21.15.31:0/3996947071 learned_addr learned my addr 172.21.15.31:0/3996947071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61ea19b700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 0x7f61e4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.441+0000 7f61e999a700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 0x7f61e4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 -- 172.21.15.31:0/3996947071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 msgr2=0x7f61e4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 0x7f61e4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 -- 172.21.15.31:0/3996947071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 msgr2=0x7f61e4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 0x7f61e4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 -- 172.21.15.31:0/3996947071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40095f0 con 0x7f61e4108db0 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea19b700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 0x7f61e4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ea99c700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00ef30 tx=0x7f61dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc018070 con 0x7f61e4108db0 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e41a0e20 con 0x7f61e4108db0 2023-10-06T21:18:50.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61dc00f040 con 0x7f61e4108db0 2023-10-06T21:18:50.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.442+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00cbc0 con 0x7f61e4108db0 2023-10-06T21:18:50.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.443+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e41a11f0 con 0x7f61e4108db0 2023-10-06T21:18:50.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.444+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e4107460 con 0x7f61e4108db0 2023-10-06T21:18:50.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.446+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f61dc007500 con 0x7f61e4108db0 2023-10-06T21:18:50.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.447+0000 7f61db7fe700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 0x7f61d00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.447+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f61dc0cc390 con 0x7f61e4108db0 2023-10-06T21:18:50.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.448+0000 7f61ea19b700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 0x7f61d00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.449+0000 7f61ea19b700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 0x7f61d00a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61e419a200 tx=0x7f61e0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.456+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f61dc095e80 con 0x7f61e4108db0 2023-10-06T21:18:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.683+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e410e7e0 con 0x7f61e4108db0 2023-10-06T21:18:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.684+0000 7f61db7fe700 1 -- 172.21.15.31:0/3996947071 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f61dc0994a0 con 0x7f61e4108db0 2023-10-06T21:18:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:50.688 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:17:58.784810+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 msgr2=0x7f61d00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 0x7f61d00a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61e419a200 tx=0x7f61e0009260 comp rx=0 tx=0).stop 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 msgr2=0x7f61e41a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00ef30 tx=0x7f61dc00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 shutdown_connections 2023-10-06T21:18:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f61d00a14d0 0x7f61d00a3990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4108db0 0x7f61e41a02a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e4073910 0x7f61e4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 --2- 172.21.15.31:0/3996947071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e4074ef0 0x7f61e4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 >> 172.21.15.31:0/3996947071 conn(0x7f61e40fc420 msgr2=0x7f61e40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 shutdown_connections 2023-10-06T21:18:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.687+0000 7f61ec3ff700 1 -- 172.21.15.31:0/3996947071 wait complete. 2023-10-06T21:18:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:18:50.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2339994204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 msgr2=0x7f6e581012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2339994204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e581012c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009990 tx=0x7f6e48009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2339994204 shutdown_connections 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2339994204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102a40 0x7f6e58109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2339994204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e581012c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2339994204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102080 0x7f6e58102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.926+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2339994204 >> 172.21.15.31:0/2339994204 conn(0x7f6e580fc460 msgr2=0x7f6e580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.927+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2339994204 shutdown_connections 2023-10-06T21:18:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.927+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2339994204 wait complete. 2023-10-06T21:18:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.927+0000 7f6e5e5ff700 1 Processor -- start 2023-10-06T21:18:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 -- start start 2023-10-06T21:18:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102080 0x7f6e581992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102a40 0x7f6e581a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a08f0 con 0x7f6e58102a40 2023-10-06T21:18:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a0a30 con 0x7f6e58102080 2023-10-06T21:18:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5e5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a0bc0 con 0x7f6e58100ea0 2023-10-06T21:18:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e57fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e57fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51712/0 (socket says 172.21.15.31:51712) 2023-10-06T21:18:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e57fff700 1 -- 172.21.15.31:0/2505741079 learned_addr learned my addr 172.21.15.31:0/2505741079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e5cb9c700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102a40 0x7f6e581a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.928+0000 7f6e577fe700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102080 0x7f6e581992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 -- 172.21.15.31:0/2505741079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102080 msgr2=0x7f6e581992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102080 0x7f6e581992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 -- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102a40 msgr2=0x7f6e581a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102a40 0x7f6e581a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 -- 172.21.15.31:0/2505741079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c009520 con 0x7f6e58100ea0 2023-10-06T21:18:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e57fff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005910 tx=0x7f6e48005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801d070 con 0x7f6e58100ea0 2023-10-06T21:18:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480095f0 con 0x7f6e58100ea0 2023-10-06T21:18:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e4800beb0 con 0x7f6e58100ea0 2023-10-06T21:18:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48022de0 con 0x7f6e58100ea0 2023-10-06T21:18:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.929+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581a10d0 con 0x7f6e58100ea0 2023-10-06T21:18:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.930+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5810c610 con 0x7f6e58100ea0 2023-10-06T21:18:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.932+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6e480225e0 con 0x7f6e58100ea0 2023-10-06T21:18:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.934+0000 7f6e557fa700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 0x7f6e400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.934+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e480ce500 con 0x7f6e58100ea0 2023-10-06T21:18:50.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.934+0000 7f6e577fe700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 0x7f6e400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.935+0000 7f6e577fe700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 0x7f6e400a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e58100bc0 tx=0x7f6e4c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:50.938+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e48097ff0 con 0x7f6e58100ea0 2023-10-06T21:18:51.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:50 smithi031 ceph-mon[98539]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:51.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:50 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3833966489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:50 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3996947071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:50 smithi142 ceph-mon[102951]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:50 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3833966489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:50 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3996947071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.155+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e581a1260 con 0x7f6e58100ea0 2023-10-06T21:18:51.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.156+0000 7f6e557fa700 1 -- 172.21.15.31:0/2505741079 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e4809b610 con 0x7f6e58100ea0 2023-10-06T21:18:51.159 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:18:51.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 msgr2=0x7f6e400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 0x7f6e400a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e58100bc0 tx=0x7f6e4c009260 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 msgr2=0x7f6e58198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005910 tx=0x7f6e48005740 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 shutdown_connections 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f6e400a1480 0x7f6e400a3940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58102a40 0x7f6e581a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e58100ea0 0x7f6e58198d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 --2- 172.21.15.31:0/2505741079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e58102080 0x7f6e581992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.158+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 >> 172.21.15.31:0/2505741079 conn(0x7f6e580fc460 msgr2=0x7f6e580fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:51.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.159+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 shutdown_connections 2023-10-06T21:18:51.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.159+0000 7f6e5e5ff700 1 -- 172.21.15.31:0/2505741079 wait complete. 2023-10-06T21:18:51.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T21:18:51.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:50 smithi146 ceph-mon[102809]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:50 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3833966489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:50 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3996947071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 -- 172.21.15.31:0/1027579016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538101940 msgr2=0x7f7538103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 --2- 172.21.15.31:0/1027579016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538101940 0x7f7538103d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 -- 172.21.15.31:0/1027579016 shutdown_connections 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 --2- 172.21.15.31:0/1027579016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538106ba0 0x7f7538109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 --2- 172.21.15.31:0/1027579016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538101940 0x7f7538103d30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 --2- 172.21.15.31:0/1027579016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538104270 0x7f7538106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 -- 172.21.15.31:0/1027579016 >> 172.21.15.31:0/1027579016 conn(0x7f75380fb310 msgr2=0x7f75380fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 -- 172.21.15.31:0/1027579016 shutdown_connections 2023-10-06T21:18:51.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.390+0000 7f753e1ff700 1 -- 172.21.15.31:0/1027579016 wait complete. 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 Processor -- start 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 -- start start 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 0x7f75381012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 0x7f75380ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 0x7f75380ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381005d0 con 0x7f7538106ba0 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7538100740 con 0x7f7538104270 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.392+0000 7f753e1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381a0b20 con 0x7f7538101940 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f75377fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 0x7f75381012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7537fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 0x7f75380ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 0x7f75380ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f75377fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 0x7f75381012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51738/0 (socket says 172.21.15.31:51738) 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f75377fe700 1 -- 172.21.15.31:0/378596962 learned_addr learned my addr 172.21.15.31:0/378596962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 -- 172.21.15.31:0/378596962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 msgr2=0x7f75381012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 0x7f75381012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 -- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 msgr2=0x7f75380ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 0x7f75380ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 -- 172.21.15.31:0/378596962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280095f0 con 0x7f7538104270 2023-10-06T21:18:51.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7537fff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 0x7f75380ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:51.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.393+0000 7f7536ffd700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 0x7f75380ff8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d750 tx=0x7f752c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.394+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c004020 con 0x7f7538104270 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.394+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75381a0e00 con 0x7f7538104270 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.394+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f752c004830 con 0x7f7538104270 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.394+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c005400 con 0x7f7538104270 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.394+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75381a1290 con 0x7f7538104270 2023-10-06T21:18:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.395+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75380fce90 con 0x7f7538104270 2023-10-06T21:18:51.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.398+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f752c004c80 con 0x7f7538104270 2023-10-06T21:18:51.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.399+0000 7f7534ff9700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 0x7f75200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.399+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f752c0cc280 con 0x7f7538104270 2023-10-06T21:18:51.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.399+0000 7f75377fe700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 0x7f75200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.400+0000 7f75377fe700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 0x7f75200a3990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f752800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:51.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.403+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f752c095ce0 con 0x7f7538104270 2023-10-06T21:18:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.624+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f753810c5a0 con 0x7f7538104270 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.625+0000 7f7534ff9700 1 -- 172.21.15.31:0/378596962 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f752c099300 con 0x7f7538104270 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:18:51.628 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T21:18:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 msgr2=0x7f75200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 0x7f75200a3990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f752800b540 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 msgr2=0x7f75380ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 0x7f75380ff8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d750 tx=0x7f752c00da60 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 shutdown_connections 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75200a14d0 0x7f75200a3990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7538106ba0 0x7f75380ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7538101940 0x7f75381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 --2- 172.21.15.31:0/378596962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7538104270 0x7f75380ff8f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 >> 172.21.15.31:0/378596962 conn(0x7f75380fb310 msgr2=0x7f75381025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 shutdown_connections 2023-10-06T21:18:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.627+0000 7f753e1ff700 1 -- 172.21.15.31:0/378596962 wait complete. 2023-10-06T21:18:51.655 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T21:18:51.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T21:18:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.870+0000 7fa96ddff700 1 -- 172.21.15.31:0/1118676517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968102090 msgr2=0x7fa968102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.870+0000 7fa96ddff700 1 --2- 172.21.15.31:0/1118676517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968102090 0x7fa968102510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa950009990 tx=0x7fa950009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 -- 172.21.15.31:0/1118676517 shutdown_connections 2023-10-06T21:18:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 --2- 172.21.15.31:0/1118676517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 0x7fa968109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 --2- 172.21.15.31:0/1118676517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968100f40 0x7fa968101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 --2- 172.21.15.31:0/1118676517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968102090 0x7fa968102510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 -- 172.21.15.31:0/1118676517 >> 172.21.15.31:0/1118676517 conn(0x7fa9680fc4d0 msgr2=0x7fa9680fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 -- 172.21.15.31:0/1118676517 shutdown_connections 2023-10-06T21:18:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.871+0000 7fa96ddff700 1 -- 172.21.15.31:0/1118676517 wait complete. 2023-10-06T21:18:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.872+0000 7fa96ddff700 1 Processor -- start 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.872+0000 7fa96ddff700 1 -- start start 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 0x7fa9681970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 0x7fa96819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819e700 con 0x7fa968100f40 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819e840 con 0x7fa968102090 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa96ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819e9d0 con 0x7fa968102a50 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48378/0 (socket says 172.21.15.31:48378) 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa9677fe700 1 -- 172.21.15.31:0/508810563 learned_addr learned my addr 172.21.15.31:0/508810563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa95ffff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 0x7fa9681970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.873+0000 7fa967fff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 0x7fa96819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 -- 172.21.15.31:0/508810563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 msgr2=0x7fa96819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 0x7fa96819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 -- 172.21.15.31:0/508810563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 msgr2=0x7fa9681970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 0x7fa9681970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 -- 172.21.15.31:0/508810563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500095f0 con 0x7fa968100f40 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa95ffff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 0x7fa9681970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9677fe700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9a0 tx=0x7fa95800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958009790 con 0x7fa968100f40 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96819ecb0 con 0x7fa968100f40 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa95800cbd0 con 0x7fa968100f40 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958018680 con 0x7fa968100f40 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.874+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96819f1c0 con 0x7fa968100f40 2023-10-06T21:18:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.875+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9681085c0 con 0x7fa968100f40 2023-10-06T21:18:51.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.877+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa95800cd40 con 0x7fa968100f40 2023-10-06T21:18:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.880+0000 7fa9657fa700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 0x7fa9540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.881+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa958014070 con 0x7fa968100f40 2023-10-06T21:18:51.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.882+0000 7fa95ffff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 0x7fa9540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:51.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.883+0000 7fa95ffff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 0x7fa9540a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa950006010 tx=0x7fa950005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:51.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:51.883+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa9580969b0 con 0x7fa968100f40 2023-10-06T21:18:52.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:51 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:52.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:51 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2505741079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:52.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:51 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/378596962' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:18:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.109+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa96810c880 con 0x7fa968100f40 2023-10-06T21:18:52.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:51 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:52.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:51 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2505741079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:52.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:51 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/378596962' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:18:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:51 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:51 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2505741079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:18:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:51 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/378596962' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:18:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.816+0000 7fa9657fa700 1 -- 172.21.15.31:0/508810563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa958099fd0 con 0x7fa968100f40 2023-10-06T21:18:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.819+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 msgr2=0x7fa9540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.819+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 0x7fa9540a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa950006010 tx=0x7fa950005c00 comp rx=0 tx=0).stop 2023-10-06T21:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.819+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 msgr2=0x7fa968196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9a0 tx=0x7fa95800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 shutdown_connections 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa9540a1480 0x7fa9540a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa968102a50 0x7fa96819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa968100f40 0x7fa968196b80 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9a0 tx=0x7fa95800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 --2- 172.21.15.31:0/508810563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa968102090 0x7fa9681970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 >> 172.21.15.31:0/508810563 conn(0x7fa9680fc4d0 msgr2=0x7fa968105700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.820+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 shutdown_connections 2023-10-06T21:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:52.821+0000 7fa96ddff700 1 -- 172.21.15.31:0/508810563 wait complete. 2023-10-06T21:18:52.827 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T21:18:52.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.067+0000 7f7c721ff700 1 -- 172.21.15.31:0/345093141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 msgr2=0x7f7c6c103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.067+0000 7f7c721ff700 1 --2- 172.21.15.31:0/345093141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c103510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0099c0 tx=0x7f7c5c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 -- 172.21.15.31:0/345093141 shutdown_connections 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 --2- 172.21.15.31:0/345093141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c104c80 0x7f7c6c10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 --2- 172.21.15.31:0/345093141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c103510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 --2- 172.21.15.31:0/345093141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 -- 172.21.15.31:0/345093141 >> 172.21.15.31:0/345093141 conn(0x7f7c6c0fe660 msgr2=0x7f7c6c100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 -- 172.21.15.31:0/345093141 shutdown_connections 2023-10-06T21:18:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.068+0000 7f7c721ff700 1 -- 172.21.15.31:0/345093141 wait complete. 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 Processor -- start 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 -- start start 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c6c104c80 0x7f7c6c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1a08f0 con 0x7f7c6c1030f0 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1a0a60 con 0x7f7c6c104c80 2023-10-06T21:18:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c721ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1a0c20 con 0x7f7c6c1042e0 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54908/0 (socket says 172.21.15.31:54908) 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.069+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/364496361 learned_addr learned my addr 172.21.15.31:0/364496361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6affd700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/364496361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 msgr2=0x7f7c6c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/364496361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c6c104c80 msgr2=0x7f7c6c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c6c104c80 0x7f7c6c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/364496361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c0095f0 con 0x7f7c6c1030f0 2023-10-06T21:18:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6affd700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c6b7fe700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c01d070 con 0x7f7c6c1030f0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c5c0054e0 con 0x7f7c6c1030f0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c004cb0 con 0x7f7c6c1030f0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.070+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c1a0ea0 con 0x7f7c6c1030f0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.071+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c1a1210 con 0x7f7c6c1030f0 2023-10-06T21:18:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.072+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c10e810 con 0x7f7c6c1030f0 2023-10-06T21:18:53.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.074+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7c5c005050 con 0x7f7c6c1030f0 2023-10-06T21:18:53.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.075+0000 7f7c68ff9700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 0x7f7c540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:53.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.075+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7c5c0cda50 con 0x7f7c6c1030f0 2023-10-06T21:18:53.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.075+0000 7f7c6affd700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 0x7f7c540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.076+0000 7f7c6affd700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 0x7f7c540a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009be0 tx=0x7f7c60009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.078+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7c5c097470 con 0x7f7c6c1030f0 2023-10-06T21:18:53.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:53.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:52 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:52 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.295+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7c6c10a100 con 0x7f7c6c1030f0 2023-10-06T21:18:53.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:53.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:52 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.823+0000 7f7c68ff9700 1 -- 172.21.15.31:0/364496361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7c5c09aa90 con 0x7f7c6c1030f0 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 msgr2=0x7f7c540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 0x7f7c540a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009be0 tx=0x7f7c60009260 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 msgr2=0x7f7c6c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c00fab0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 shutdown_connections 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7c540a1500 0x7f7c540a39c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c6c104c80 0x7f7c6c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c6c1030f0 0x7f7c6c198db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 --2- 172.21.15.31:0/364496361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6c1042e0 0x7f7c6c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:53.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 >> 172.21.15.31:0/364496361 conn(0x7f7c6c0fe660 msgr2=0x7f7c6c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:53.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 shutdown_connections 2023-10-06T21:18:53.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:53.827+0000 7f7c721ff700 1 -- 172.21.15.31:0/364496361 wait complete. 2023-10-06T21:18:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T21:18:53.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.074+0000 7f73be5ff700 1 -- 172.21.15.31:0/1966279513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8102a30 msgr2=0x7f73b8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.074+0000 7f73be5ff700 1 --2- 172.21.15.31:0/1966279513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8102a30 0x7f73b8109350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73b000b0a0 tx=0x7f73b000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 -- 172.21.15.31:0/1966279513 shutdown_connections 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 --2- 172.21.15.31:0/1966279513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8102a30 0x7f73b8109350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 --2- 172.21.15.31:0/1966279513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b8100e90 0x7f73b81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 --2- 172.21.15.31:0/1966279513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 0x7f73b81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 -- 172.21.15.31:0/1966279513 >> 172.21.15.31:0/1966279513 conn(0x7f73b80fc430 msgr2=0x7f73b80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 -- 172.21.15.31:0/1966279513 shutdown_connections 2023-10-06T21:18:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.075+0000 7f73be5ff700 1 -- 172.21.15.31:0/1966279513 wait complete. 2023-10-06T21:18:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 Processor -- start 2023-10-06T21:18:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 -- start start 2023-10-06T21:18:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 0x7f73b80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b8102a30 0x7f73b80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8073f80 con 0x7f73b8100e90 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80740f0 con 0x7f73b8102a30 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73be5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b81a0c40 con 0x7f73b8102070 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54936/0 (socket says 172.21.15.31:54936) 2023-10-06T21:18:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73affff700 1 -- 172.21.15.31:0/860855462 learned_addr learned my addr 172.21.15.31:0/860855462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.076+0000 7f73b7fff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 0x7f73b80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 -- 172.21.15.31:0/860855462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 msgr2=0x7f73b80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 0x7f73b80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 -- 172.21.15.31:0/860855462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b8102a30 msgr2=0x7f73b80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b8102a30 0x7f73b80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 -- 172.21.15.31:0/860855462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b0009d00 con 0x7f73b8100e90 2023-10-06T21:18:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73affff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e970 tx=0x7f73a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8009790 con 0x7f73b8100e90 2023-10-06T21:18:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73a800eec0 con 0x7f73b8100e90 2023-10-06T21:18:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8003c10 con 0x7f73b8100e90 2023-10-06T21:18:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.077+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b81a0f20 con 0x7f73b8100e90 2023-10-06T21:18:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.078+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b81a1350 con 0x7f73b8100e90 2023-10-06T21:18:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.078+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b8107e40 con 0x7f73b8100e90 2023-10-06T21:18:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.080+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f73a8004870 con 0x7f73b8100e90 2023-10-06T21:18:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.082+0000 7f73b5ffb700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 0x7f73a40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.082+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f73a8014070 con 0x7f73b8100e90 2023-10-06T21:18:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.082+0000 7f73b7fff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 0x7f73a40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.083+0000 7f73b7fff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 0x7f73a40a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73b8100bb0 tx=0x7f73a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.084+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f73a8096630 con 0x7f73b8100e90 2023-10-06T21:18:54.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:18:54.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:18:54.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:18:54.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:54.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:54.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:53 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:18:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:18:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:18:54.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:54.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:54.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:53 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/508810563' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:18:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:18:54.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:18:54.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:54.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:54.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:53 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.344+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f73b802d400 con 0x7f73b8100e90 2023-10-06T21:18:54.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.857+0000 7f73b5ffb700 1 -- 172.21.15.31:0/860855462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f73a8099c50 con 0x7f73b8100e90 2023-10-06T21:18:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.863+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 msgr2=0x7f73a40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.863+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 0x7f73a40a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73b8100bb0 tx=0x7f73a0009260 comp rx=0 tx=0).stop 2023-10-06T21:18:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.863+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 msgr2=0x7f73b8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.863+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e970 tx=0x7f73a800ec80 comp rx=0 tx=0).stop 2023-10-06T21:18:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 shutdown_connections 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f73a40a1500 0x7f73a40a39c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b8102a30 0x7f73b80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73b8100e90 0x7f73b8074c60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 --2- 172.21.15.31:0/860855462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b8102070 0x7f73b80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 >> 172.21.15.31:0/860855462 conn(0x7f73b80fc430 msgr2=0x7f73b80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 shutdown_connections 2023-10-06T21:18:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:54.864+0000 7f73be5ff700 1 -- 172.21.15.31:0/860855462 wait complete. 2023-10-06T21:18:54.869 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2023-10-06T21:18:54.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.104+0000 7f9014fff700 1 -- 172.21.15.31:0/845037922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 msgr2=0x7f9010104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.104+0000 7f9014fff700 1 --2- 172.21.15.31:0/845037922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f9010104740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90040099c0 tx=0x7f9004009cd0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 -- 172.21.15.31:0/845037922 shutdown_connections 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 --2- 172.21.15.31:0/845037922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 0x7f901010b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 --2- 172.21.15.31:0/845037922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 0x7f9010103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 --2- 172.21.15.31:0/845037922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f9010104740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 -- 172.21.15.31:0/845037922 >> 172.21.15.31:0/845037922 conn(0x7f90100fe660 msgr2=0x7f9010100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 -- 172.21.15.31:0/845037922 shutdown_connections 2023-10-06T21:18:55.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.105+0000 7f9014fff700 1 -- 172.21.15.31:0/845037922 wait complete. 2023-10-06T21:18:55.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 Processor -- start 2023-10-06T21:18:55.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 -- start start 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 0x7f9010198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 0x7f90101a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101a08e0 con 0x7f90101042e0 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101a0a20 con 0x7f9010104c80 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.106+0000 7f9014fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101a0bb0 con 0x7f90101030f0 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54952/0 (socket says 172.21.15.31:54952) 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 -- 172.21.15.31:0/2213183983 learned_addr learned my addr 172.21.15.31:0/2213183983 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900e59c700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 0x7f9010198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900ed9d700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 0x7f90101a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 -- 172.21.15.31:0/2213183983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 msgr2=0x7f9010198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 0x7f9010198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 -- 172.21.15.31:0/2213183983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 msgr2=0x7f90101a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 0x7f90101a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 -- 172.21.15.31:0/2213183983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040095f0 con 0x7f90101042e0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f900dd9b700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9004000c00 tx=0x7f900400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.107+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900401d070 con 0x7f90101042e0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.108+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f900400fdc0 con 0x7f90101042e0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.108+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90101a0e30 con 0x7f90101042e0 2023-10-06T21:18:55.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.108+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040173f0 con 0x7f90101042e0 2023-10-06T21:18:55.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.108+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101a11f0 con 0x7f90101042e0 2023-10-06T21:18:55.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.109+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901010a6b0 con 0x7f90101042e0 2023-10-06T21:18:55.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.111+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9004022470 con 0x7f90101042e0 2023-10-06T21:18:55.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.113+0000 7f8fff7fe700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 0x7f8ff80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.113+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f90040ce290 con 0x7f90101042e0 2023-10-06T21:18:55.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.113+0000 7f900e59c700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 0x7f8ff80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.113+0000 7f900e59c700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 0x7f8ff80a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9010104140 tx=0x7f9000009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:55.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.115+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9004097a30 con 0x7f90101042e0 2023-10-06T21:18:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: pgmap v297: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:18:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:18:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:18:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:18:55.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:54 smithi142 ceph-mon[102951]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:18:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: pgmap v297: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:18:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:18:55.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:18:55.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:55.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:18:55.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:54 smithi031 ceph-mon[98539]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: pgmap v297: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/364496361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:18:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:54 smithi146 ceph-mon[102809]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:18:55.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.360+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f901010e8a0 con 0x7f90101042e0 2023-10-06T21:18:55.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.362+0000 7f8fff7fe700 1 -- 172.21.15.31:0/2213183983 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f900409b050 con 0x7f90101042e0 2023-10-06T21:18:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:55.365 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:18:54.346102+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:18:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 msgr2=0x7f8ff80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 0x7f8ff80a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9010104140 tx=0x7f9000009190 comp rx=0 tx=0).stop 2023-10-06T21:18:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 msgr2=0x7f90101992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9004000c00 tx=0x7f900400fa00 comp rx=0 tx=0).stop 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 shutdown_connections 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ff80a14d0 0x7f8ff80a3990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010104c80 0x7f90101a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90101030f0 0x7f9010198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 --2- 172.21.15.31:0/2213183983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90101042e0 0x7f90101992a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.364+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 >> 172.21.15.31:0/2213183983 conn(0x7f90100fe660 msgr2=0x7f90101077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.365+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 shutdown_connections 2023-10-06T21:18:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.365+0000 7f9014fff700 1 -- 172.21.15.31:0/2213183983 wait complete. 2023-10-06T21:18:55.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 -- 172.21.15.31:0/1280364988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 msgr2=0x7fd35010b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 --2- 172.21.15.31:0/1280364988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd35010b550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd348009f90 tx=0x7fd34800b330 comp rx=0 tx=0).stop 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 -- 172.21.15.31:0/1280364988 shutdown_connections 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 --2- 172.21.15.31:0/1280364988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd35010b550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 --2- 172.21.15.31:0/1280364988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 --2- 172.21.15.31:0/1280364988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 -- 172.21.15.31:0/1280364988 >> 172.21.15.31:0/1280364988 conn(0x7fd3500fe660 msgr2=0x7fd350100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 -- 172.21.15.31:0/1280364988 shutdown_connections 2023-10-06T21:18:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.595+0000 7fd3549ff700 1 -- 172.21.15.31:0/1280364988 wait complete. 2023-10-06T21:18:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.596+0000 7fd3549ff700 1 Processor -- start 2023-10-06T21:18:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 -- start start 2023-10-06T21:18:55.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd3501a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a0960 con 0x7fd350104300 2023-10-06T21:18:55.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a0aa0 con 0x7fd350103110 2023-10-06T21:18:55.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd3549ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a0c30 con 0x7fd350104ca0 2023-10-06T21:18:55.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.597+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55700/0 (socket says 172.21.15.31:55700) 2023-10-06T21:18:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34e59c700 1 -- 172.21.15.31:0/2186490303 learned_addr learned my addr 172.21.15.31:0/2186490303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd3501a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34dd9b700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 -- 172.21.15.31:0/2186490303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 msgr2=0x7fd350198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 -- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 msgr2=0x7fd350199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 -- 172.21.15.31:0/2186490303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400095f0 con 0x7fd350104ca0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34dd9b700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34e59c700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.598+0000 7fd34ed9d700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd3501a0390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3480078c0 tx=0x7fd348008cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.599+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800e040 con 0x7fd350104ca0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.599+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd34800bc00 con 0x7fd350104ca0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.599+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480052a0 con 0x7fd350104ca0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.599+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd344009520 con 0x7fd350104ca0 2023-10-06T21:18:55.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.599+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd348009c40 con 0x7fd350104ca0 2023-10-06T21:18:55.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.600+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35010e830 con 0x7fd350104ca0 2023-10-06T21:18:55.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.603+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd348005400 con 0x7fd350104ca0 2023-10-06T21:18:55.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.605+0000 7fd33f7fe700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 0x7fd3380a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.605+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3480ce870 con 0x7fd350104ca0 2023-10-06T21:18:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.605+0000 7fd34e59c700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 0x7fd3380a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:55.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.606+0000 7fd34e59c700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 0x7fd3380a3a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd340005950 tx=0x7fd340009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:55.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.607+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd348098130 con 0x7fd350104ca0 2023-10-06T21:18:55.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.827+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd35010ea40 con 0x7fd350104ca0 2023-10-06T21:18:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.832+0000 7fd33f7fe700 1 -- 172.21.15.31:0/2186490303 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22130 (secure 0 0 0) 0x7fd34800bd70 con 0x7fd350104ca0 2023-10-06T21:18:55.835 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:18:55.835 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:18:54.346102+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:18:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 msgr2=0x7fd3380a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 0x7fd3380a3a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd340005950 tx=0x7fd340009310 comp rx=0 tx=0).stop 2023-10-06T21:18:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 msgr2=0x7fd3501a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd3501a0390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3480078c0 tx=0x7fd348008cd0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 shutdown_connections 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd3380a1550 0x7fd3380a3a10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350104ca0 0x7fd3501a0390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd350103110 0x7fd350198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 --2- 172.21.15.31:0/2186490303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd350104300 0x7fd350199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 >> 172.21.15.31:0/2186490303 conn(0x7fd3500fe660 msgr2=0x7fd350100860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 shutdown_connections 2023-10-06T21:18:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:55.835+0000 7fd3549ff700 1 -- 172.21.15.31:0/2186490303 wait complete. 2023-10-06T21:18:55.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 -- 172.21.15.31:0/130024654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40073910 msgr2=0x7f3a40073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 --2- 172.21.15.31:0/130024654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40073910 0x7f3a40073d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009990 tx=0x7f3a2c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 -- 172.21.15.31:0/130024654 shutdown_connections 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 --2- 172.21.15.31:0/130024654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40108da0 0x7f3a4010b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 --2- 172.21.15.31:0/130024654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40074ef0 0x7f3a40073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 --2- 172.21.15.31:0/130024654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40073910 0x7f3a40073d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 -- 172.21.15.31:0/130024654 >> 172.21.15.31:0/130024654 conn(0x7f3a400fc430 msgr2=0x7f3a400fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 -- 172.21.15.31:0/130024654 shutdown_connections 2023-10-06T21:18:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.071+0000 7f3a46bff700 1 -- 172.21.15.31:0/130024654 wait complete. 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.072+0000 7f3a46bff700 1 Processor -- start 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 -- start start 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 0x7f3a40198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 0x7f3a401a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401a08d0 con 0x7f3a40074ef0 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401a0a10 con 0x7f3a40108da0 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a46bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401a0ba0 con 0x7f3a40073910 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54998/0 (socket says 172.21.15.31:54998) 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a4499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 0x7f3a40198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a3ffff700 1 -- 172.21.15.31:0/2292990864 learned_addr learned my addr 172.21.15.31:0/2292990864 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a4519c700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 0x7f3a401a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a3ffff700 1 -- 172.21.15.31:0/2292990864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 msgr2=0x7f3a40198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.073+0000 7f3a3ffff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 0x7f3a40198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3ffff700 1 -- 172.21.15.31:0/2292990864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 msgr2=0x7f3a401a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3ffff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 0x7f3a401a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3ffff700 1 -- 172.21.15.31:0/2292990864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0095f0 con 0x7f3a40074ef0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a4519c700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 0x7f3a401a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a4499b700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 0x7f3a40198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3ffff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c005910 tx=0x7f3a2c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c01d070 con 0x7f3a40074ef0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a401a0e20 con 0x7f3a40074ef0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a2c00be30 con 0x7f3a40074ef0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c022dd0 con 0x7f3a40074ef0 2023-10-06T21:18:56.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.074+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a401a1190 con 0x7f3a40074ef0 2023-10-06T21:18:56.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.075+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a40107450 con 0x7f3a40074ef0 2023-10-06T21:18:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.077+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3a2c022570 con 0x7f3a40074ef0 2023-10-06T21:18:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.078+0000 7f3a3dffb700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 0x7f3a300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.079+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a2c0ce0c0 con 0x7f3a40074ef0 2023-10-06T21:18:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.079+0000 7f3a4499b700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 0x7f3a300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.079+0000 7f3a4499b700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 0x7f3a300a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a34011490 tx=0x7f3a34009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:56.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.082+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3a2c097980 con 0x7f3a40074ef0 2023-10-06T21:18:56.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:55 smithi142 ceph-mon[102951]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:18:56.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:55 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:18:56.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:55 smithi142 ceph-mon[102951]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:18:56.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:55 smithi142 ceph-mon[102951]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:18:56.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:55 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2213183983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:56.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:55 smithi031 ceph-mon[98539]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:18:56.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:55 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:18:56.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:55 smithi031 ceph-mon[98539]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:18:56.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:55 smithi031 ceph-mon[98539]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:18:56.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:55 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2213183983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:55 smithi146 ceph-mon[102809]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:18:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:55 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/860855462' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:18:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:55 smithi146 ceph-mon[102809]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:18:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:55 smithi146 ceph-mon[102809]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:18:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:55 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2213183983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.348+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3a4002d090 con 0x7f3a40074ef0 2023-10-06T21:18:56.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.867+0000 7f3a3dffb700 1 -- 172.21.15.31:0/2292990864 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3a2c09afa0 con 0x7f3a40074ef0 2023-10-06T21:18:56.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 msgr2=0x7f3a300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:56.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 0x7f3a300a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a34011490 tx=0x7f3a34009260 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 msgr2=0x7f3a40199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c005910 tx=0x7f3a2c005740 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 shutdown_connections 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3a300a1480 0x7f3a300a3940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a40108da0 0x7f3a401a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a40073910 0x7f3a40198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 --2- 172.21.15.31:0/2292990864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a40074ef0 0x7f3a40199290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.871+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 >> 172.21.15.31:0/2292990864 conn(0x7f3a400fc430 msgr2=0x7f3a400fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.872+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 shutdown_connections 2023-10-06T21:18:56.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:56.872+0000 7f3a46bff700 1 -- 172.21.15.31:0/2292990864 wait complete. 2023-10-06T21:18:56.895 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T21:18:56.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-06T21:18:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 -- 172.21.15.31:0/469963650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102a30 msgr2=0x7f173c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 --2- 172.21.15.31:0/469963650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102a30 0x7f173c109350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f173400b0a0 tx=0x7f173400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 -- 172.21.15.31:0/469963650 shutdown_connections 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 --2- 172.21.15.31:0/469963650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102a30 0x7f173c109350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 --2- 172.21.15.31:0/469963650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 --2- 172.21.15.31:0/469963650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102070 0x7f173c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 -- 172.21.15.31:0/469963650 >> 172.21.15.31:0/469963650 conn(0x7f173c0fc430 msgr2=0x7f173c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 -- 172.21.15.31:0/469963650 shutdown_connections 2023-10-06T21:18:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.108+0000 7f17429ff700 1 -- 172.21.15.31:0/469963650 wait complete. 2023-10-06T21:18:57.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.109+0000 7f17429ff700 1 Processor -- start 2023-10-06T21:18:57.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.109+0000 7f17429ff700 1 -- start start 2023-10-06T21:18:57.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102a30 0x7f173c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19e760 con 0x7f173c102070 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19e8d0 con 0x7f173c102a30 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f17429ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19ea90 con 0x7f173c100e90 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f1733fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55026/0 (socket says 172.21.15.31:55026) 2023-10-06T21:18:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 -- 172.21.15.31:0/2013113608 learned_addr learned my addr 172.21.15.31:0/2013113608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f1740f9c700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102a30 0x7f173c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 -- 172.21.15.31:0/2013113608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 msgr2=0x7f173c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 -- 172.21.15.31:0/2013113608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102a30 msgr2=0x7f173c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102a30 0x7f173c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f173bfff700 1 -- 172.21.15.31:0/2013113608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009d00 con 0x7f173c102070 2023-10-06T21:18:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.110+0000 7f1733fff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f173bfff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f172400d980 tx=0x7f172400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724004020 con 0x7f173c102070 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f172400f690 con 0x7f173c102070 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c19ed70 con 0x7f173c102070 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240039e0 con 0x7f173c102070 2023-10-06T21:18:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.111+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c19f210 con 0x7f173c102070 2023-10-06T21:18:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.112+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c005320 con 0x7f173c102070 2023-10-06T21:18:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.115+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1724004180 con 0x7f173c102070 2023-10-06T21:18:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.116+0000 7f1739ffb700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 0x7f17280a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.117+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f17240cc9e0 con 0x7f173c102070 2023-10-06T21:18:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.117+0000 7f1733fff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 0x7f17280a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.117+0000 7f1733fff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 0x7f17280a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f173c1980f0 tx=0x7f172c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:57.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.119+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1724096170 con 0x7f173c102070 2023-10-06T21:18:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: pgmap v300: 97 pgs: 15 active+clean, 29 creating+peering, 53 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2186490303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: Cluster is now healthy 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:active 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:56 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:18:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: pgmap v300: 97 pgs: 15 active+clean, 29 creating+peering, 53 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2186490303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:18:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:18:57.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: Cluster is now healthy 2023-10-06T21:18:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:active 2023-10-06T21:18:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:56 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:18:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: pgmap v300: 97 pgs: 15 active+clean, 29 creating+peering, 53 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:18:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2186490303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:18:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:18:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:18:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: Cluster is now healthy 2023-10-06T21:18:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] up:active 2023-10-06T21:18:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:56 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:18:57.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.393+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f171c005b50 con 0x7f173c102070 2023-10-06T21:18:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.875+0000 7f1739ffb700 1 -- 172.21.15.31:0/2013113608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f17240170b0 con 0x7f173c102070 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 msgr2=0x7f17280a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 0x7f17280a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f173c1980f0 tx=0x7f172c009260 comp rx=0 tx=0).stop 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 msgr2=0x7f173c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f172400d980 tx=0x7f172400dc90 comp rx=0 tx=0).stop 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 shutdown_connections 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f17280a1500 0x7f17280a39c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c102a30 0x7f173c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173c100e90 0x7f173c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 --2- 172.21.15.31:0/2013113608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c102070 0x7f173c197160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:57.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 >> 172.21.15.31:0/2013113608 conn(0x7f173c0fc430 msgr2=0x7f173c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:57.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 shutdown_connections 2023-10-06T21:18:57.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:57.878+0000 7f17429ff700 1 -- 172.21.15.31:0/2013113608 wait complete. 2023-10-06T21:18:57.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-06T21:18:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 -- 172.21.15.31:0/3008790829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc073910 msgr2=0x7fb4cc073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/3008790829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc073910 0x7fb4cc073d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009990 tx=0x7fb4c0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 -- 172.21.15.31:0/3008790829 shutdown_connections 2023-10-06T21:18:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/3008790829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/3008790829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc074ef0 0x7fb4cc073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/3008790829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc073910 0x7fb4cc073d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 -- 172.21.15.31:0/3008790829 >> 172.21.15.31:0/3008790829 conn(0x7fb4cc0fc460 msgr2=0x7fb4cc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 -- 172.21.15.31:0/3008790829 shutdown_connections 2023-10-06T21:18:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.111+0000 7fb4d13ff700 1 -- 172.21.15.31:0/3008790829 wait complete. 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 Processor -- start 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 -- start start 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 0x7fb4cc19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc074ef0 0x7fb4cc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1a4c90 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1a4dd0 con 0x7fb4cc074ef0 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.112+0000 7fb4d13ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1a4f60 con 0x7fb4cc073910 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4caffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 0x7fb4cc19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4caffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 0x7fb4cc19d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:38656/0 (socket says 172.21.15.31:38656) 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4caffd700 1 -- 172.21.15.31:0/1067707733 learned_addr learned my addr 172.21.15.31:0/1067707733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4ca7fc700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc074ef0 0x7fb4cc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 -- 172.21.15.31:0/1067707733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 msgr2=0x7fb4cc19d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 0x7fb4cc19d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 -- 172.21.15.31:0/1067707733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc074ef0 msgr2=0x7fb4cc19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc074ef0 0x7fb4cc19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 -- 172.21.15.31:0/1067707733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00095f0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.113+0000 7fb4cb7fe700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc1a46c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400efc0 tx=0x7fb4c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.114+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4018070 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.114+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc1a51e0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.114+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4c400f040 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.114+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400cba0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.114+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc1a56a0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.115+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc103010 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.117+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb4c4003a00 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.118+0000 7fb4b3fff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 0x7fb4b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.118+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4c40cc750 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.118+0000 7fb4caffd700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 0x7fb4b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.119+0000 7fb4caffd700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 0x7fb4b40a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005950 tx=0x7fb4bc00b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.122+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb4c4095ef0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:18:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:58.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:57 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:57 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2292990864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:18:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T21:18:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:58.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:57 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:18:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.382+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb4cc112bb0 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.808+0000 7fb4b3fff700 1 -- 172.21.15.31:0/1067707733 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb4c4099510 con 0x7fb4cc10d4c0 2023-10-06T21:18:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 msgr2=0x7fb4b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 0x7fb4b40a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005950 tx=0x7fb4bc00b500 comp rx=0 tx=0).stop 2023-10-06T21:18:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 msgr2=0x7fb4cc1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc1a46c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400efc0 tx=0x7fb4c400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 shutdown_connections 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb4b40a1480 0x7fb4b40a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc10d4c0 0x7fb4cc1a46c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc073910 0x7fb4cc19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 --2- 172.21.15.31:0/1067707733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc074ef0 0x7fb4cc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 >> 172.21.15.31:0/1067707733 conn(0x7fb4cc0fc460 msgr2=0x7fb4cc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 shutdown_connections 2023-10-06T21:18:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:58.811+0000 7fb4d13ff700 1 -- 172.21.15.31:0/1067707733 wait complete. 2023-10-06T21:18:58.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-06T21:18:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 -- 172.21.15.31:0/3033913308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128073910 msgr2=0x7fa128073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 --2- 172.21.15.31:0/3033913308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128073910 0x7fa128073d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa118009990 tx=0x7fa118009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 -- 172.21.15.31:0/3033913308 shutdown_connections 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 --2- 172.21.15.31:0/3033913308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 0x7fa12810b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 --2- 172.21.15.31:0/3033913308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128074ef0 0x7fa128073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 --2- 172.21.15.31:0/3033913308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128073910 0x7fa128073d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 -- 172.21.15.31:0/3033913308 >> 172.21.15.31:0/3033913308 conn(0x7fa1280fc430 msgr2=0x7fa1280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 -- 172.21.15.31:0/3033913308 shutdown_connections 2023-10-06T21:18:59.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.049+0000 7fa12fbff700 1 -- 172.21.15.31:0/3033913308 wait complete. 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.050+0000 7fa12fbff700 1 Processor -- start 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.050+0000 7fa12fbff700 1 -- start start 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128073910 0x7fa128198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 0x7fa1281a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281a08f0 con 0x7fa128074ef0 2023-10-06T21:18:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281a0a60 con 0x7fa128108da0 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281a0c20 con 0x7fa128073910 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55068/0 (socket says 172.21.15.31:55068) 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 -- 172.21.15.31:0/2344898461 learned_addr learned my addr 172.21.15.31:0/2344898461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12e19c700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 0x7fa1281a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d99b700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128073910 0x7fa128198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 -- 172.21.15.31:0/2344898461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128073910 msgr2=0x7fa128198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128073910 0x7fa128198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 -- 172.21.15.31:0/2344898461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 msgr2=0x7fa1281a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 0x7fa1281a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.051+0000 7fa12d19a700 1 -- 172.21.15.31:0/2344898461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa128074ef0 2023-10-06T21:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa12d19a700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa118003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11801d070 con 0x7fa128074ef0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1281a0ea0 con 0x7fa128074ef0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa11800bd70 con 0x7fa128074ef0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118022c40 con 0x7fa128074ef0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.052+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1281a1210 con 0x7fa128074ef0 2023-10-06T21:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.053+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa128107450 con 0x7fa128074ef0 2023-10-06T21:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.055+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa118022470 con 0x7fa128074ef0 2023-10-06T21:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.056+0000 7fa11effd700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 0x7fa1140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.057+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1180ce2d0 con 0x7fa128074ef0 2023-10-06T21:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.057+0000 7fa12d99b700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 0x7fa1140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.057+0000 7fa12d99b700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 0x7fa1140a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa12819a280 tx=0x7fa12400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.060+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa118097c30 con 0x7fa128074ef0 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: pgmap v302: 97 pgs: 64 active+clean, 29 creating+peering, 4 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:active 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-mon[102951]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-10-06T21:18:59.153 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:18:58 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b[140597]: 2023-10-06T21:18:58.803+0000 7fa27b688700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: pgmap v302: 97 pgs: 64 active+clean, 29 creating+peering, 4 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:18:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:18:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:active 2023-10-06T21:18:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:18:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-06T21:18:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-10-06T21:18:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:58 smithi031 ceph-mon[98539]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: pgmap v302: 97 pgs: 64 active+clean, 29 creating+peering, 4 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2013113608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:18:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1067707733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:18:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] up:active 2023-10-06T21:18:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:18:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-06T21:18:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-10-06T21:18:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:58 smithi146 ceph-mon[102809]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-10-06T21:18:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.319+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fa12810e780 con 0x7fa128074ef0 2023-10-06T21:18:59.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.816+0000 7fa11effd700 1 -- 172.21.15.31:0/2344898461 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa118027020 con 0x7fa128074ef0 2023-10-06T21:18:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 msgr2=0x7fa1140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 0x7fa1140a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa12819a280 tx=0x7fa12400b410 comp rx=0 tx=0).stop 2023-10-06T21:18:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 msgr2=0x7fa1281992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa118003a50 comp rx=0 tx=0).stop 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 shutdown_connections 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa1140a1480 0x7fa1140a3940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa128108da0 0x7fa1281a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa128073910 0x7fa128198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 --2- 172.21.15.31:0/2344898461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa128074ef0 0x7fa1281992f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 >> 172.21.15.31:0/2344898461 conn(0x7fa1280fc430 msgr2=0x7fa1280fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.819+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 shutdown_connections 2023-10-06T21:18:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:18:59.820+0000 7fa12fbff700 1 -- 172.21.15.31:0/2344898461 wait complete. 2023-10-06T21:18:59.848 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-06T21:18:59.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-06T21:19:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 -- 172.21.15.31:0/1154915551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102a30 msgr2=0x7fb3d4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/1154915551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102a30 0x7fb3d4109350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b0a0 tx=0x7fb3cc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 -- 172.21.15.31:0/1154915551 shutdown_connections 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/1154915551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102a30 0x7fb3d4109350 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/1154915551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4100e90 0x7fb3d41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/1154915551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4102070 0x7fb3d41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 -- 172.21.15.31:0/1154915551 >> 172.21.15.31:0/1154915551 conn(0x7fb3d40fc430 msgr2=0x7fb3d40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 -- 172.21.15.31:0/1154915551 shutdown_connections 2023-10-06T21:19:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.057+0000 7fb3dbfff700 1 -- 172.21.15.31:0/1154915551 wait complete. 2023-10-06T21:19:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 Processor -- start 2023-10-06T21:19:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 -- start start 2023-10-06T21:19:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 0x7fb3d4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4102a30 0x7fb3d41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d41a09a0 con 0x7fb3d4102070 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d41a0ae0 con 0x7fb3d4102a30 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3dbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d41a0c70 con 0x7fb3d4100e90 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3d959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3d959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55074/0 (socket says 172.21.15.31:55074) 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.058+0000 7fb3d959a700 1 -- 172.21.15.31:0/3243126991 learned_addr learned my addr 172.21.15.31:0/3243126991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d9d9b700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 0x7fb3d4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3da59c700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4102a30 0x7fb3d41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 -- 172.21.15.31:0/3243126991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 msgr2=0x7fb3d4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 0x7fb3d4198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 -- 172.21.15.31:0/3243126991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4102a30 msgr2=0x7fb3d41a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4102a30 0x7fb3d41a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 -- 172.21.15.31:0/3243126991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009d00 con 0x7fb3d4102070 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d9d9b700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 0x7fb3d4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3d959a700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400d980 tx=0x7fb3c400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400de60 con 0x7fb3d4102070 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c40044b0 con 0x7fb3d4102070 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d41a0f50 con 0x7fb3d4102070 2023-10-06T21:19:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4003b40 con 0x7fb3d4102070 2023-10-06T21:19:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.059+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d41a13d0 con 0x7fb3d4102070 2023-10-06T21:19:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.061+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d41933f0 con 0x7fb3d4102070 2023-10-06T21:19:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.062+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb3c4003ca0 con 0x7fb3d4102070 2023-10-06T21:19:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.064+0000 7fb3caffd700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 0x7fb3c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:00.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.064+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3c40cc8c0 con 0x7fb3d4102070 2023-10-06T21:19:00.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.064+0000 7fb3d9d9b700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 0x7fb3c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:00.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.065+0000 7fb3d9d9b700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 0x7fb3c00a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4101be0 tx=0x7fb3d000b670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:00.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.068+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb3c4095fd0 con 0x7fb3d4102070 2023-10-06T21:19:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:19:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:19:00.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:19:00.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T21:19:00.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:19:00.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:00.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:18:59 smithi142 ceph-mon[102951]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:18:59 smithi031 ceph-mon[98539]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2344898461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:19:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:00.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:18:59 smithi146 ceph-mon[102809]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-10-06T21:19:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.313+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d402d320 con 0x7fb3c00a1480 2023-10-06T21:19:00.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:19:00.320+0000 7fa24af57700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:19:00.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:19:00.320+0000 7fa24af57700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:19:00.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:19:00.320+0000 7fa24af57700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:19:00.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:19:00.320+0000 7fa24af57700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:19:00.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:19:00.320+0000 7fa24af57700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:19:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.870+0000 7fb3caffd700 1 -- 172.21.15.31:0/3243126991 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb3d402d320 con 0x7fb3c00a1480 2023-10-06T21:19:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 msgr2=0x7fb3c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 0x7fb3c00a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4101be0 tx=0x7fb3d000b670 comp rx=0 tx=0).stop 2023-10-06T21:19:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 msgr2=0x7fb3d4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400d980 tx=0x7fb3c400dd40 comp rx=0 tx=0).stop 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 shutdown_connections 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb3c00a1480 0x7fb3c00a3940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4102a30 0x7fb3d41a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d4100e90 0x7fb3d4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 --2- 172.21.15.31:0/3243126991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d4102070 0x7fb3d4199360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 >> 172.21.15.31:0/3243126991 conn(0x7fb3d40fc430 msgr2=0x7fb3d4107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 shutdown_connections 2023-10-06T21:19:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:00.873+0000 7fb3dbfff700 1 -- 172.21.15.31:0/3243126991 wait complete. 2023-10-06T21:19:00.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.104+0000 7f5922fff700 1 -- 172.21.15.31:0/758694528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102a30 msgr2=0x7f591c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.104+0000 7f5922fff700 1 --2- 172.21.15.31:0/758694528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102a30 0x7f591c109350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f591400b0a0 tx=0x7f591400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 -- 172.21.15.31:0/758694528 shutdown_connections 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 --2- 172.21.15.31:0/758694528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102a30 0x7f591c109350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 --2- 172.21.15.31:0/758694528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 0x7f591c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 --2- 172.21.15.31:0/758694528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102070 0x7f591c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 -- 172.21.15.31:0/758694528 >> 172.21.15.31:0/758694528 conn(0x7f591c0fc430 msgr2=0x7f591c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 -- 172.21.15.31:0/758694528 shutdown_connections 2023-10-06T21:19:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.105+0000 7f5922fff700 1 -- 172.21.15.31:0/758694528 wait complete. 2023-10-06T21:19:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 Processor -- start 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 -- start start 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 0x7f591c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102a30 0x7f591c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c19e760 con 0x7f591c102070 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c19e8a0 con 0x7f591c100e90 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.106+0000 7f5922fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c19ea30 con 0x7f591c102a30 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f5920d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 0x7f591c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55104/0 (socket says 172.21.15.31:55104) 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 -- 172.21.15.31:0/3695809563 learned_addr learned my addr 172.21.15.31:0/3695809563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f592159c700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102a30 0x7f591c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 -- 172.21.15.31:0/3695809563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102a30 msgr2=0x7f591c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102a30 0x7f591c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 -- 172.21.15.31:0/3695809563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 msgr2=0x7f591c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 0x7f591c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 -- 172.21.15.31:0/3695809563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914009d00 con 0x7f591c102070 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f591bfff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f590800e970 tx=0x7f590800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.107+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908009790 con 0x7f591c102070 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.108+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590800eec0 con 0x7f591c102070 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.108+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908003c10 con 0x7f591c102070 2023-10-06T21:19:01.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.108+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c19ed10 con 0x7f591c102070 2023-10-06T21:19:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.108+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c19f1d0 con 0x7f591c102070 2023-10-06T21:19:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.109+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c10c560 con 0x7f591c102070 2023-10-06T21:19:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.111+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5908003db0 con 0x7f591c102070 2023-10-06T21:19:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.113+0000 7f5919ffb700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 0x7f590c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.114+0000 7f5920d9b700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 0x7f590c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.114+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5908014070 con 0x7f591c102070 2023-10-06T21:19:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.114+0000 7f5920d9b700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 0x7f590c0a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f591c101be0 tx=0x7f591000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.117+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59080965d0 con 0x7f591c102070 2023-10-06T21:19:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.2 KiB/s wr, 13 op/s 2023-10-06T21:19:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:active 2023-10-06T21:19:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T21:19:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:19:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: Cluster is now healthy 2023-10-06T21:19:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-06T21:19:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:00 smithi142 ceph-mon[102951]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.2 KiB/s wr, 13 op/s 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:active 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: Cluster is now healthy 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-06T21:19:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:00 smithi031 ceph-mon[98539]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.2 KiB/s wr, 13 op/s 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] up:active 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: Cluster is now healthy 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-06T21:19:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:00 smithi146 ceph-mon[102809]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-06T21:19:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.367+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f591c0612f0 con 0x7f590c0a1480 2023-10-06T21:19:01.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.371+0000 7f5919ffb700 1 -- 172.21.15.31:0/3695809563 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f591c0612f0 con 0x7f590c0a1480 2023-10-06T21:19:01.374 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 msgr2=0x7f590c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 0x7f590c0a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f591c101be0 tx=0x7f591000b410 comp rx=0 tx=0).stop 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 msgr2=0x7f591c197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f590800e970 tx=0x7f590800ec80 comp rx=0 tx=0).stop 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 shutdown_connections 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f590c0a1480 0x7f590c0a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f591c102a30 0x7f591c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f591c100e90 0x7f591c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 --2- 172.21.15.31:0/3695809563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f591c102070 0x7f591c197120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 >> 172.21.15.31:0/3695809563 conn(0x7f591c0fc430 msgr2=0x7f591c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:01.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 shutdown_connections 2023-10-06T21:19:01.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.374+0000 7f5922fff700 1 -- 172.21.15.31:0/3695809563 wait complete. 2023-10-06T21:19:01.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-06T21:19:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 -- 172.21.15.31:0/2484977054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec102070 msgr2=0x7f72ec1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 --2- 172.21.15.31:0/2484977054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec102070 0x7f72ec1024f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009990 tx=0x7f72d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 -- 172.21.15.31:0/2484977054 shutdown_connections 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 --2- 172.21.15.31:0/2484977054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 0x7f72ec109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 --2- 172.21.15.31:0/2484977054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec100e90 0x7f72ec1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 --2- 172.21.15.31:0/2484977054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec102070 0x7f72ec1024f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.601+0000 7f72f13ff700 1 -- 172.21.15.31:0/2484977054 >> 172.21.15.31:0/2484977054 conn(0x7f72ec0fc430 msgr2=0x7f72ec0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.602+0000 7f72f13ff700 1 -- 172.21.15.31:0/2484977054 shutdown_connections 2023-10-06T21:19:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.602+0000 7f72f13ff700 1 -- 172.21.15.31:0/2484977054 wait complete. 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 Processor -- start 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 -- start start 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec102070 0x7f72ec1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 0x7f72ec1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1a0930 con 0x7f72ec100e90 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1a0a70 con 0x7f72ec102070 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72f13ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1a0c00 con 0x7f72ec102a30 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55118/0 (socket says 172.21.15.31:55118) 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.603+0000 7f72e2ffd700 1 -- 172.21.15.31:0/575780137 learned_addr learned my addr 172.21.15.31:0/575780137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72eaffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec102070 0x7f72ec1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72eb7fe700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 0x7f72ec1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 -- 172.21.15.31:0/575780137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 msgr2=0x7f72ec1a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 0x7f72ec1a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 -- 172.21.15.31:0/575780137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec102070 msgr2=0x7f72ec1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec102070 0x7f72ec1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 -- 172.21.15.31:0/575780137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40095f0 con 0x7f72ec100e90 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e2ffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e970 tx=0x7f72dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc009790 con 0x7f72ec100e90 2023-10-06T21:19:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72dc00eec0 con 0x7f72ec100e90 2023-10-06T21:19:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec1a0ee0 con 0x7f72ec100e90 2023-10-06T21:19:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc003c10 con 0x7f72ec100e90 2023-10-06T21:19:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.604+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec1a1300 con 0x7f72ec100e90 2023-10-06T21:19:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.606+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec1933f0 con 0x7f72ec100e90 2023-10-06T21:19:01.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.607+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f72dc003db0 con 0x7f72ec100e90 2023-10-06T21:19:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.610+0000 7f72e8ff9700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 0x7f72d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.610+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72dc014070 con 0x7f72ec100e90 2023-10-06T21:19:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.610+0000 7f72eaffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 0x7f72d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.610+0000 7f72eaffd700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 0x7f72d80a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72d40098e0 tx=0x7f72d400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:01.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.614+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f72dc0965d0 con 0x7f72ec100e90 2023-10-06T21:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.866+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f72ec02d350 con 0x7f72d80a1480 2023-10-06T21:19:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.893+0000 7f72e8ff9700 1 -- 172.21.15.31:0/575780137 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f72ec02d350 con 0x7f72d80a1480 2023-10-06T21:19:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 msgr2=0x7f72d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 0x7f72d80a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72d40098e0 tx=0x7f72d400b260 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 msgr2=0x7f72ec198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e970 tx=0x7f72dc00ec80 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 shutdown_connections 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f72d80a1480 0x7f72d80a3940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72ec102a30 0x7f72ec1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ec100e90 0x7f72ec198db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 --2- 172.21.15.31:0/575780137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72ec102070 0x7f72ec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 >> 172.21.15.31:0/575780137 conn(0x7f72ec0fc430 msgr2=0x7f72ec107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:01.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 shutdown_connections 2023-10-06T21:19:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:01.895+0000 7f72f13ff700 1 -- 172.21.15.31:0/575780137 wait complete. 2023-10-06T21:19:01.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-06T21:19:02.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 -- 172.21.15.31:0/190688491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 msgr2=0x7efec01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 --2- 172.21.15.31:0/190688491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec01012b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efebc009990 tx=0x7efebc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 -- 172.21.15.31:0/190688491 shutdown_connections 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 --2- 172.21.15.31:0/190688491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102a30 0x7efec0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 --2- 172.21.15.31:0/190688491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec01012b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 --2- 172.21.15.31:0/190688491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102070 0x7efec01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 -- 172.21.15.31:0/190688491 >> 172.21.15.31:0/190688491 conn(0x7efec00fc430 msgr2=0x7efec00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 -- 172.21.15.31:0/190688491 shutdown_connections 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.121+0000 7efec7fff700 1 -- 172.21.15.31:0/190688491 wait complete. 2023-10-06T21:19:02.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 Processor -- start 2023-10-06T21:19:02.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 -- start start 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102070 0x7efec0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 0x7efec01a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01a0900 con 0x7efec0100e90 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01a0a70 con 0x7efec0102a30 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec7fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01a0c30 con 0x7efec0102070 2023-10-06T21:19:02.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55140/0 (socket says 172.21.15.31:55140) 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec5d9b700 1 -- 172.21.15.31:0/209645893 learned_addr learned my addr 172.21.15.31:0/209645893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec559a700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102070 0x7efec0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.123+0000 7efec659c700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 0x7efec01a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 -- 172.21.15.31:0/209645893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102070 msgr2=0x7efec0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102070 0x7efec0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 -- 172.21.15.31:0/209645893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 msgr2=0x7efec01a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 0x7efec01a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 -- 172.21.15.31:0/209645893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0095f0 con 0x7efec0100e90 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec659c700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 0x7efec01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec5d9b700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efebc0095c0 tx=0x7efebc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc01d070 con 0x7efec0100e90 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efebc004000 con 0x7efec0100e90 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec01a0eb0 con 0x7efec0100e90 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.124+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc022bd0 con 0x7efec0100e90 2023-10-06T21:19:02.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.125+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec01a1270 con 0x7efec0100e90 2023-10-06T21:19:02.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.126+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec010c600 con 0x7efec0100e90 2023-10-06T21:19:02.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.128+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efebc003b70 con 0x7efec0100e90 2023-10-06T21:19:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.129+0000 7efeb6ffd700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 0x7efeac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.129+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efebc0ce1c0 con 0x7efec0100e90 2023-10-06T21:19:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.129+0000 7efec559a700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 0x7efeac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.130+0000 7efec559a700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 0x7efeac0a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efec0100bb0 tx=0x7efeb000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.133+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efebc097960 con 0x7efec0100e90 2023-10-06T21:19:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:01 smithi142 ceph-mon[102951]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:01 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:active 2023-10-06T21:19:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:01 smithi142 ceph-mon[102951]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-06T21:19:02.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:01 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:19:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:01 smithi031 ceph-mon[98539]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:01 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:active 2023-10-06T21:19:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:01 smithi031 ceph-mon[98539]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-06T21:19:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:01 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:19:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:01 smithi146 ceph-mon[102809]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:01 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] up:active 2023-10-06T21:19:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:01 smithi146 ceph-mon[102809]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-06T21:19:02.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:01 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:19:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.381+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efec0108b30 con 0x7efeac0a1500 2023-10-06T21:19:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.385+0000 7efeb6ffd700 1 -- 172.21.15.31:0/209645893 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7efec0108b30 con 0x7efeac0a1500 2023-10-06T21:19:02.388 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/sv_1/236d7ccf-15c8-4226-a2e0-5ad8f8a2ca46 2023-10-06T21:19:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 msgr2=0x7efeac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 0x7efeac0a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efec0100bb0 tx=0x7efeb000b410 comp rx=0 tx=0).stop 2023-10-06T21:19:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 msgr2=0x7efec0198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efebc0095c0 tx=0x7efebc005740 comp rx=0 tx=0).stop 2023-10-06T21:19:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 shutdown_connections 2023-10-06T21:19:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efeac0a1500 0x7efeac0a39c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec0102a30 0x7efec01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec0100e90 0x7efec0198dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 --2- 172.21.15.31:0/209645893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec0102070 0x7efec0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 >> 172.21.15.31:0/209645893 conn(0x7efec00fc430 msgr2=0x7efec00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 shutdown_connections 2023-10-06T21:19:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.388+0000 7efec7fff700 1 -- 172.21.15.31:0/209645893 wait complete. 2023-10-06T21:19:02.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 -- 172.21.15.31:0/2101280229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 msgr2=0x7fedc0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 --2- 172.21.15.31:0/2101280229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc0109360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fedb800b0a0 tx=0x7fedb800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 -- 172.21.15.31:0/2101280229 shutdown_connections 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 --2- 172.21.15.31:0/2101280229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc0109360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 --2- 172.21.15.31:0/2101280229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0100ea0 0x7fedc01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 --2- 172.21.15.31:0/2101280229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0102080 0x7fedc0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 -- 172.21.15.31:0/2101280229 >> 172.21.15.31:0/2101280229 conn(0x7fedc00fc420 msgr2=0x7fedc00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 -- 172.21.15.31:0/2101280229 shutdown_connections 2023-10-06T21:19:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.611+0000 7fedc63ff700 1 -- 172.21.15.31:0/2101280229 wait complete. 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.613+0000 7fedc63ff700 1 Processor -- start 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.613+0000 7fedc63ff700 1 -- start start 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0100ea0 0x7fedc0196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0102080 0x7fedc01970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019e700 con 0x7fedc0102a40 2023-10-06T21:19:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019e840 con 0x7fedc0100ea0 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019e9d0 con 0x7fedc0102080 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55154/0 (socket says 172.21.15.31:55154) 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 -- 172.21.15.31:0/309108064 learned_addr learned my addr 172.21.15.31:0/309108064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedbffff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0100ea0 0x7fedc0196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedbf7fe700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0102080 0x7fedc01970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 -- 172.21.15.31:0/309108064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0102080 msgr2=0x7fedc01970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0102080 0x7fedc01970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 -- 172.21.15.31:0/309108064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0100ea0 msgr2=0x7fedc0196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0100ea0 0x7fedc0196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.614+0000 7fedc499c700 1 -- 172.21.15.31:0/309108064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009d00 con 0x7fedc0102a40 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedc499c700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d010 tx=0x7fedb800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800e040 con 0x7fedc0102a40 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedb8003e10 con 0x7fedc0102a40 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc019ec50 con 0x7fedc0102a40 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80052a0 con 0x7fedc0102a40 2023-10-06T21:19:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.615+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc019f0d0 con 0x7fedc0102a40 2023-10-06T21:19:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.616+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0108490 con 0x7fedc0102a40 2023-10-06T21:19:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.618+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fedb801d9a0 con 0x7fedc0102a40 2023-10-06T21:19:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.619+0000 7fedbd7fa700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 0x7feda80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.620+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fedb80cdc60 con 0x7fedc0102a40 2023-10-06T21:19:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.620+0000 7fedbffff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 0x7feda80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.620+0000 7fedbffff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 0x7feda80a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fedc0101bf0 tx=0x7fedb0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:02.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.623+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fedb8097400 con 0x7fedc0102a40 2023-10-06T21:19:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.887+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc010c5c0 con 0x7fedc0102a40 2023-10-06T21:19:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.888+0000 7fedbd7fa700 1 -- 172.21.15.31:0/309108064 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7fedb809aa20 con 0x7fedc0102a40 2023-10-06T21:19:02.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:02.892 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 msgr2=0x7feda80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 0x7feda80a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fedc0101bf0 tx=0x7fedb0009260 comp rx=0 tx=0).stop 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 msgr2=0x7fedc019e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d010 tx=0x7fedb800bcc0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 shutdown_connections 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feda80a1550 0x7feda80a3a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedc0102a40 0x7fedc019e130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc0100ea0 0x7fedc0196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 --2- 172.21.15.31:0/309108064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc0102080 0x7fedc01970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 >> 172.21.15.31:0/309108064 conn(0x7fedc00fc420 msgr2=0x7fedc01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 shutdown_connections 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:02.891+0000 7fedc63ff700 1 -- 172.21.15.31:0/309108064 wait complete. 2023-10-06T21:19:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:19:02.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:19:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 -- 172.21.15.31:0/215068169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4102a40 msgr2=0x7f91d4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 --2- 172.21.15.31:0/215068169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4102a40 0x7f91d4109360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00b0a0 tx=0x7f91cc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 -- 172.21.15.31:0/215068169 shutdown_connections 2023-10-06T21:19:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 --2- 172.21.15.31:0/215068169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4102a40 0x7f91d4109360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 --2- 172.21.15.31:0/215068169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4100ea0 0x7f91d41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 --2- 172.21.15.31:0/215068169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102080 0x7f91d4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 -- 172.21.15.31:0/215068169 >> 172.21.15.31:0/215068169 conn(0x7f91d40fc420 msgr2=0x7f91d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 -- 172.21.15.31:0/215068169 shutdown_connections 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.123+0000 7f91dadff700 1 -- 172.21.15.31:0/215068169 wait complete. 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.124+0000 7f91dadff700 1 Processor -- start 2023-10-06T21:19:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.124+0000 7f91dadff700 1 -- start start 2023-10-06T21:19:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 0x7f91d40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102a40 0x7f91d40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4073f80 con 0x7f91d4100ea0 2023-10-06T21:19:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40740f0 con 0x7f91d4102080 2023-10-06T21:19:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91dadff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41a0c50 con 0x7f91d4102a40 2023-10-06T21:19:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54650/0 (socket says 172.21.15.31:54650) 2023-10-06T21:19:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 -- 172.21.15.31:0/3382474946 learned_addr learned my addr 172.21.15.31:0/3382474946 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d939c700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102a40 0x7f91d40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d3fff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 0x7f91d40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 -- 172.21.15.31:0/3382474946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102a40 msgr2=0x7f91d40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102a40 0x7f91d40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 -- 172.21.15.31:0/3382474946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 msgr2=0x7f91d40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 0x7f91d40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 -- 172.21.15.31:0/3382474946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009d00 con 0x7f91d4100ea0 2023-10-06T21:19:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d3fff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 0x7f91d40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.125+0000 7f91d8b9b700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e970 tx=0x7f91c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.126+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8009790 con 0x7f91d4100ea0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.126+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91c80044b0 con 0x7f91d4100ea0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.126+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d41a0f30 con 0x7f91d4100ea0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.126+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8003dd0 con 0x7f91d4100ea0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.126+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d41a13b0 con 0x7f91d4100ea0 2023-10-06T21:19:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.127+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d4108510 con 0x7f91d4100ea0 2023-10-06T21:19:03.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.129+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f91c8004000 con 0x7f91d4100ea0 2023-10-06T21:19:03.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.130+0000 7f91d1ffb700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 0x7f91c40a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.130+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91c8014070 con 0x7f91d4100ea0 2023-10-06T21:19:03.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.130+0000 7f91d3fff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 0x7f91c40a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.131+0000 7f91d3fff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 0x7f91c40a3a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005e10 tx=0x7f91c0005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.134+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91c8096620 con 0x7f91d4100ea0 2023-10-06T21:19:03.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:02 smithi031 ceph-mon[98539]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:02 smithi031 ceph-mon[98539]: pgmap v304: 97 pgs: 97 active+clean; 599 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.5 KiB/s wr, 20 op/s 2023-10-06T21:19:03.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:02 smithi031 ceph-mon[98539]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:02 smithi146 ceph-mon[102809]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:02 smithi146 ceph-mon[102809]: pgmap v304: 97 pgs: 97 active+clean; 599 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.5 KiB/s wr, 20 op/s 2023-10-06T21:19:03.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:02 smithi146 ceph-mon[102809]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.347+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d410c830 con 0x7f91d4100ea0 2023-10-06T21:19:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.348+0000 7f91d1ffb700 1 -- 172.21.15.31:0/3382474946 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22117 (secure 0 0 0) 0x7f91c8099c40 con 0x7f91d4100ea0 2023-10-06T21:19:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:03.351 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:18:55.827472+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:19:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.350+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 msgr2=0x7f91c40a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.350+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 0x7f91c40a3a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005e10 tx=0x7f91c0005d20 comp rx=0 tx=0).stop 2023-10-06T21:19:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.350+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 msgr2=0x7f91d4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.350+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e970 tx=0x7f91c800ed30 comp rx=0 tx=0).stop 2023-10-06T21:19:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 shutdown_connections 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91c40a15d0 0x7f91c40a3a90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d4102a40 0x7f91d40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91d4100ea0 0x7f91d4074c60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 --2- 172.21.15.31:0/3382474946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d4102080 0x7f91d40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 >> 172.21.15.31:0/3382474946 conn(0x7f91d40fc420 msgr2=0x7f91d4105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 shutdown_connections 2023-10-06T21:19:03.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.351+0000 7f91dadff700 1 -- 172.21.15.31:0/3382474946 wait complete. 2023-10-06T21:19:03.373 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T21:19:03.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:19:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:02 smithi142 ceph-mon[102951]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:02 smithi142 ceph-mon[102951]: pgmap v304: 97 pgs: 97 active+clean; 599 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.5 KiB/s wr, 20 op/s 2023-10-06T21:19:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:02 smithi142 ceph-mon[102951]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:03.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 -- 172.21.15.31:0/436712243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4100ea0 msgr2=0x7fecb41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/436712243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4100ea0 0x7fecb41012c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009990 tx=0x7fecb0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 -- 172.21.15.31:0/436712243 shutdown_connections 2023-10-06T21:19:03.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/436712243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4102a40 0x7fecb4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/436712243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4100ea0 0x7fecb41012c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/436712243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 0x7fecb4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 -- 172.21.15.31:0/436712243 >> 172.21.15.31:0/436712243 conn(0x7fecb40fc420 msgr2=0x7fecb40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 -- 172.21.15.31:0/436712243 shutdown_connections 2023-10-06T21:19:03.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.576+0000 7fecbc3ff700 1 -- 172.21.15.31:0/436712243 wait complete. 2023-10-06T21:19:03.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 Processor -- start 2023-10-06T21:19:03.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 -- start start 2023-10-06T21:19:03.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4100ea0 0x7fecb4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 0x7fecb4197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419e6d0 con 0x7fecb4102a40 2023-10-06T21:19:03.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419e810 con 0x7fecb4102080 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.578+0000 7fecbc3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419e9a0 con 0x7fecb4100ea0 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54674/0 (socket says 172.21.15.31:54674) 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4100ea0 0x7fecb4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 -- 172.21.15.31:0/645327510 learned_addr learned my addr 172.21.15.31:0/645327510 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecb999a700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 0x7fecb4197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 -- 172.21.15.31:0/645327510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4100ea0 msgr2=0x7fecb4196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4100ea0 0x7fecb4196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 -- 172.21.15.31:0/645327510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 msgr2=0x7fecb4197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 0x7fecb4197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 -- 172.21.15.31:0/645327510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00095f0 con 0x7fecb4102a40 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.579+0000 7fecba99c700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fecac007ee0 tx=0x7fecac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.580+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac017070 con 0x7fecb4102a40 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.580+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecac00f040 con 0x7fecb4102a40 2023-10-06T21:19:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.580+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac005570 con 0x7fecb4102a40 2023-10-06T21:19:03.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.580+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb419ec80 con 0x7fecb4102a40 2023-10-06T21:19:03.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.580+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb419f140 con 0x7fecb4102a40 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.581+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb41085e0 con 0x7fecb4102a40 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.583+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fecac00db20 con 0x7fecb4102a40 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.584+0000 7fecab7fe700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 0x7feca00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.584+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fecac0cc350 con 0x7fecb4102a40 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.584+0000 7fecba19b700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 0x7feca00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.585+0000 7fecba19b700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 0x7feca00a3a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb000b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.588+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fecac095a00 con 0x7fecb4102a40 2023-10-06T21:19:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.797+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecb402d2b0 con 0x7feca00a15d0 2023-10-06T21:19:03.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.811+0000 7fecab7fe700 1 -- 172.21.15.31:0/645327510 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336048 (secure 0 0 0) 0x7fecb402d2b0 con 0x7feca00a15d0 2023-10-06T21:19:03.819 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 msgr2=0x7feca00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 0x7feca00a3a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb000b560 comp rx=0 tx=0).stop 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 msgr2=0x7fecb419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fecac007ee0 tx=0x7fecac00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 shutdown_connections 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7feca00a15d0 0x7feca00a3a90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecb4102a40 0x7fecb419e100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecb4100ea0 0x7fecb4196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 --2- 172.21.15.31:0/645327510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb4102080 0x7fecb4197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 >> 172.21.15.31:0/645327510 conn(0x7fecb40fc420 msgr2=0x7fecb4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 shutdown_connections 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:03.816+0000 7fecbc3ff700 1 -- 172.21.15.31:0/645327510 wait complete. 2023-10-06T21:19:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:19:03.839 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-10-06T21:19:01.806414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613704,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":161,"num_read_kb":138,"num_write":321,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335380,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606444,"statfs":{"total":1151940427776,"available":1151596998656,"internally_reserved":0,"allocated":11440128,"data_stored":7090928,"data_compressed":107196,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23336,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.978552"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356315+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:18:56.356315+0000","last_peered":"2023-10-06T21:18:56.356315+0000","last_clean":"2023-10-06T21:18:56.356315+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:18:56.356315+0000","last_undegraded":"2023-10-06T21:18:56.356315+0000","last_fullsized":"2023-10-06T21:18:56.356315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697863+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:18:56.697863+0000","last_peered":"2023-10-06T21:18:56.697863+0000","last_clean":"2023-10-06T21:18:56.697863+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:18:56.697863+0000","last_undegraded":"2023-10-06T21:18:56.697863+0000","last_fullsized":"2023-10-06T21:18:56.697863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697629+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:18:56.697629+0000","last_peered":"2023-10-06T21:18:56.697629+0000","last_clean":"2023-10-06T21:18:56.697629+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:18:56.697629+0000","last_undegraded":"2023-10-06T21:18:56.697629+0000","last_fullsized":"2023-10-06T21:18:56.697629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.838022+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:18:55.838022+0000","last_peered":"2023-10-06T21:18:55.838022+0000","last_clean":"2023-10-06T21:18:55.838022+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:18:55.838022+0000","last_undegraded":"2023-10-06T21:18:55.838022+0000","last_fullsized":"2023-10-06T21:18:55.838022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697329+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:18:56.697329+0000","last_peered":"2023-10-06T21:18:56.697329+0000","last_clean":"2023-10-06T21:18:56.697329+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:18:56.697329+0000","last_undegraded":"2023-10-06T21:18:56.697329+0000","last_fullsized":"2023-10-06T21:18:56.697329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682507+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:18:56.682507+0000","last_peered":"2023-10-06T21:18:56.682507+0000","last_clean":"2023-10-06T21:18:56.682507+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:18:56.682507+0000","last_undegraded":"2023-10-06T21:18:56.682507+0000","last_fullsized":"2023-10-06T21:18:56.682507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389064+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:18:57.389064+0000","last_peered":"2023-10-06T21:18:57.389064+0000","last_clean":"2023-10-06T21:18:57.389064+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:18:57.389064+0000","last_undegraded":"2023-10-06T21:18:57.389064+0000","last_fullsized":"2023-10-06T21:18:57.389064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835555+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:18:55.835555+0000","last_peered":"2023-10-06T21:18:55.835555+0000","last_clean":"2023-10-06T21:18:55.835555+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:18:55.835555+0000","last_undegraded":"2023-10-06T21:18:55.835555+0000","last_fullsized":"2023-10-06T21:18:55.835555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697617+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:18:56.697617+0000","last_peered":"2023-10-06T21:18:56.697617+0000","last_clean":"2023-10-06T21:18:56.697617+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:18:56.697617+0000","last_undegraded":"2023-10-06T21:18:56.697617+0000","last_fullsized":"2023-10-06T21:18:56.697617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682779+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:18:56.682779+0000","last_peered":"2023-10-06T21:18:56.682779+0000","last_clean":"2023-10-06T21:18:56.682779+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:18:56.682779+0000","last_undegraded":"2023-10-06T21:18:56.682779+0000","last_fullsized":"2023-10-06T21:18:56.682779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389285+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:18:57.389285+0000","last_peered":"2023-10-06T21:18:57.389285+0000","last_clean":"2023-10-06T21:18:57.389285+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:18:57.389285+0000","last_undegraded":"2023-10-06T21:18:57.389285+0000","last_fullsized":"2023-10-06T21:18:57.389285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.837899+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:18:55.837899+0000","last_peered":"2023-10-06T21:18:55.837899+0000","last_clean":"2023-10-06T21:18:55.837899+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:18:55.837899+0000","last_undegraded":"2023-10-06T21:18:55.837899+0000","last_fullsized":"2023-10-06T21:18:55.837899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697092+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:18:56.697092+0000","last_peered":"2023-10-06T21:18:56.697092+0000","last_clean":"2023-10-06T21:18:56.697092+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:18:56.697092+0000","last_undegraded":"2023-10-06T21:18:56.697092+0000","last_fullsized":"2023-10-06T21:18:56.697092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698017+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:18:56.698017+0000","last_peered":"2023-10-06T21:18:56.698017+0000","last_clean":"2023-10-06T21:18:56.698017+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:18:56.698017+0000","last_undegraded":"2023-10-06T21:18:56.698017+0000","last_fullsized":"2023-10-06T21:18:56.698017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697471+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:18:56.697471+0000","last_peered":"2023-10-06T21:18:56.697471+0000","last_clean":"2023-10-06T21:18:56.697471+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:18:56.697471+0000","last_undegraded":"2023-10-06T21:18:56.697471+0000","last_fullsized":"2023-10-06T21:18:56.697471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697171+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:18:56.697171+0000","last_peered":"2023-10-06T21:18:56.697171+0000","last_clean":"2023-10-06T21:18:56.697171+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:18:56.697171+0000","last_undegraded":"2023-10-06T21:18:56.697171+0000","last_fullsized":"2023-10-06T21:18:56.697171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356400+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:18:56.356400+0000","last_peered":"2023-10-06T21:18:56.356400+0000","last_clean":"2023-10-06T21:18:56.356400+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:18:56.356400+0000","last_undegraded":"2023-10-06T21:18:56.356400+0000","last_fullsized":"2023-10-06T21:18:56.356400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698352+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:18:56.698352+0000","last_peered":"2023-10-06T21:18:56.698352+0000","last_clean":"2023-10-06T21:18:56.698352+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:18:56.698352+0000","last_undegraded":"2023-10-06T21:18:56.698352+0000","last_fullsized":"2023-10-06T21:18:56.698352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836559+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:18:55.836559+0000","last_peered":"2023-10-06T21:18:55.836559+0000","last_clean":"2023-10-06T21:18:55.836559+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:18:55.836559+0000","last_undegraded":"2023-10-06T21:18:55.836559+0000","last_fullsized":"2023-10-06T21:18:55.836559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698307+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:18:56.698307+0000","last_peered":"2023-10-06T21:18:56.698307+0000","last_clean":"2023-10-06T21:18:56.698307+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:18:56.698307+0000","last_undegraded":"2023-10-06T21:18:56.698307+0000","last_fullsized":"2023-10-06T21:18:56.698307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835945+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:18:55.835945+0000","last_peered":"2023-10-06T21:18:55.835945+0000","last_clean":"2023-10-06T21:18:55.835945+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:18:55.835945+0000","last_undegraded":"2023-10-06T21:18:55.835945+0000","last_fullsized":"2023-10-06T21:18:55.835945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836108+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:18:55.836108+0000","last_peered":"2023-10-06T21:18:55.836108+0000","last_clean":"2023-10-06T21:18:55.836108+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:18:55.836108+0000","last_undegraded":"2023-10-06T21:18:55.836108+0000","last_fullsized":"2023-10-06T21:18:55.836108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698118+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:18:56.698118+0000","last_peered":"2023-10-06T21:18:56.698118+0000","last_clean":"2023-10-06T21:18:56.698118+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:18:56.698118+0000","last_undegraded":"2023-10-06T21:18:56.698118+0000","last_fullsized":"2023-10-06T21:18:56.698118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.828068+0000","last_change":"2023-10-06T21:18:53.852079+0000","last_active":"2023-10-06T21:18:58.828068+0000","last_peered":"2023-10-06T21:18:58.828068+0000","last_clean":"2023-10-06T21:18:58.828068+0000","last_became_active":"2023-10-06T21:18:53.851472+0000","last_became_peered":"2023-10-06T21:18:53.851472+0000","last_unstale":"2023-10-06T21:18:58.828068+0000","last_undegraded":"2023-10-06T21:18:58.828068+0000","last_fullsized":"2023-10-06T21:18:58.828068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:00:45.421872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697281+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:18:56.697281+0000","last_peered":"2023-10-06T21:18:56.697281+0000","last_clean":"2023-10-06T21:18:56.697281+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:18:56.697281+0000","last_undegraded":"2023-10-06T21:18:56.697281+0000","last_fullsized":"2023-10-06T21:18:56.697281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.876281+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:18:54.876281+0000","last_peered":"2023-10-06T21:18:54.876281+0000","last_clean":"2023-10-06T21:18:54.876281+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:18:54.876281+0000","last_undegraded":"2023-10-06T21:18:54.876281+0000","last_fullsized":"2023-10-06T21:18:54.876281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698250+0000","last_change":"2023-10-06T21:18:54.849038+0000","last_active":"2023-10-06T21:18:56.698250+0000","last_peered":"2023-10-06T21:18:56.698250+0000","last_clean":"2023-10-06T21:18:56.698250+0000","last_became_active":"2023-10-06T21:18:54.848645+0000","last_became_peered":"2023-10-06T21:18:54.848645+0000","last_unstale":"2023-10-06T21:18:56.698250+0000","last_undegraded":"2023-10-06T21:18:56.698250+0000","last_fullsized":"2023-10-06T21:18:56.698250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:21:01.954014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.823402+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:18:59.823402+0000","last_peered":"2023-10-06T21:18:59.823402+0000","last_clean":"2023-10-06T21:18:59.823402+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:18:59.823402+0000","last_undegraded":"2023-10-06T21:18:59.823402+0000","last_fullsized":"2023-10-06T21:18:59.823402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697489+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:18:56.697489+0000","last_peered":"2023-10-06T21:18:56.697489+0000","last_clean":"2023-10-06T21:18:56.697489+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:18:56.697489+0000","last_undegraded":"2023-10-06T21:18:56.697489+0000","last_fullsized":"2023-10-06T21:18:56.697489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835491+0000","last_change":"2023-10-06T21:18:53.852076+0000","last_active":"2023-10-06T21:18:59.835491+0000","last_peered":"2023-10-06T21:18:59.835491+0000","last_clean":"2023-10-06T21:18:59.835491+0000","last_became_active":"2023-10-06T21:18:53.851622+0000","last_became_peered":"2023-10-06T21:18:53.851622+0000","last_unstale":"2023-10-06T21:18:59.835491+0000","last_undegraded":"2023-10-06T21:18:59.835491+0000","last_fullsized":"2023-10-06T21:18:59.835491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:42.339452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698241+0000","last_change":"2023-10-06T21:18:54.846408+0000","last_active":"2023-10-06T21:18:56.698241+0000","last_peered":"2023-10-06T21:18:56.698241+0000","last_clean":"2023-10-06T21:18:56.698241+0000","last_became_active":"2023-10-06T21:18:54.846113+0000","last_became_peered":"2023-10-06T21:18:54.846113+0000","last_unstale":"2023-10-06T21:18:56.698241+0000","last_undegraded":"2023-10-06T21:18:56.698241+0000","last_fullsized":"2023-10-06T21:18:56.698241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:36.454140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.830159+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:18:58.830159+0000","last_peered":"2023-10-06T21:18:58.830159+0000","last_clean":"2023-10-06T21:18:58.830159+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:18:58.830159+0000","last_undegraded":"2023-10-06T21:18:58.830159+0000","last_fullsized":"2023-10-06T21:18:58.830159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835642+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:18:55.835642+0000","last_peered":"2023-10-06T21:18:55.835642+0000","last_clean":"2023-10-06T21:18:55.835642+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:18:55.835642+0000","last_undegraded":"2023-10-06T21:18:55.835642+0000","last_fullsized":"2023-10-06T21:18:55.835642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.833932+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:18:59.833932+0000","last_peered":"2023-10-06T21:18:59.833932+0000","last_clean":"2023-10-06T21:18:59.833932+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:18:59.833932+0000","last_undegraded":"2023-10-06T21:18:59.833932+0000","last_fullsized":"2023-10-06T21:18:59.833932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698211+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:18:56.698211+0000","last_peered":"2023-10-06T21:18:56.698211+0000","last_clean":"2023-10-06T21:18:56.698211+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:18:56.698211+0000","last_undegraded":"2023-10-06T21:18:56.698211+0000","last_fullsized":"2023-10-06T21:18:56.698211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.836607+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:18:59.836607+0000","last_peered":"2023-10-06T21:18:59.836607+0000","last_clean":"2023-10-06T21:18:59.836607+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:18:59.836607+0000","last_undegraded":"2023-10-06T21:18:59.836607+0000","last_fullsized":"2023-10-06T21:18:59.836607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697569+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:18:56.697569+0000","last_peered":"2023-10-06T21:18:56.697569+0000","last_clean":"2023-10-06T21:18:56.697569+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:18:56.697569+0000","last_undegraded":"2023-10-06T21:18:56.697569+0000","last_fullsized":"2023-10-06T21:18:56.697569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895772+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:18:57.895772+0000","last_peered":"2023-10-06T21:18:57.895772+0000","last_clean":"2023-10-06T21:18:57.895772+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:18:57.895772+0000","last_undegraded":"2023-10-06T21:18:57.895772+0000","last_fullsized":"2023-10-06T21:18:57.895772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697891+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:18:56.697891+0000","last_peered":"2023-10-06T21:18:56.697891+0000","last_clean":"2023-10-06T21:18:56.697891+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:18:56.697891+0000","last_undegraded":"2023-10-06T21:18:56.697891+0000","last_fullsized":"2023-10-06T21:18:56.697891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.820062+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:18:58.820062+0000","last_peered":"2023-10-06T21:18:58.820062+0000","last_clean":"2023-10-06T21:18:58.820062+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:18:58.820062+0000","last_undegraded":"2023-10-06T21:18:58.820062+0000","last_fullsized":"2023-10-06T21:18:58.820062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697442+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:18:56.697442+0000","last_peered":"2023-10-06T21:18:56.697442+0000","last_clean":"2023-10-06T21:18:56.697442+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:18:56.697442+0000","last_undegraded":"2023-10-06T21:18:56.697442+0000","last_fullsized":"2023-10-06T21:18:56.697442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.861202+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:19:00.861202+0000","last_peered":"2023-10-06T21:19:00.861202+0000","last_clean":"2023-10-06T21:19:00.861202+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:19:00.861202+0000","last_undegraded":"2023-10-06T21:19:00.861202+0000","last_fullsized":"2023-10-06T21:19:00.861202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697565+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:18:56.697565+0000","last_peered":"2023-10-06T21:18:56.697565+0000","last_clean":"2023-10-06T21:18:56.697565+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:18:56.697565+0000","last_undegraded":"2023-10-06T21:18:56.697565+0000","last_fullsized":"2023-10-06T21:18:56.697565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.894388+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:18:57.894388+0000","last_peered":"2023-10-06T21:18:57.894388+0000","last_clean":"2023-10-06T21:18:57.894388+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:18:57.894388+0000","last_undegraded":"2023-10-06T21:18:57.894388+0000","last_fullsized":"2023-10-06T21:18:57.894388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697030+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:18:56.697030+0000","last_peered":"2023-10-06T21:18:56.697030+0000","last_clean":"2023-10-06T21:18:56.697030+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:18:56.697030+0000","last_undegraded":"2023-10-06T21:18:56.697030+0000","last_fullsized":"2023-10-06T21:18:56.697030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835118+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:18:59.835118+0000","last_peered":"2023-10-06T21:18:59.835118+0000","last_clean":"2023-10-06T21:18:59.835118+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:18:59.835118+0000","last_undegraded":"2023-10-06T21:18:59.835118+0000","last_fullsized":"2023-10-06T21:18:59.835118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389435+0000","last_change":"2023-10-06T21:18:54.860485+0000","last_active":"2023-10-06T21:18:57.389435+0000","last_peered":"2023-10-06T21:18:57.389435+0000","last_clean":"2023-10-06T21:18:57.389435+0000","last_became_active":"2023-10-06T21:18:54.860171+0000","last_became_peered":"2023-10-06T21:18:54.860171+0000","last_unstale":"2023-10-06T21:18:57.389435+0000","last_undegraded":"2023-10-06T21:18:57.389435+0000","last_fullsized":"2023-10-06T21:18:57.389435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:46:50.890861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835407+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:18:59.835407+0000","last_peered":"2023-10-06T21:18:59.835407+0000","last_clean":"2023-10-06T21:18:59.835407+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:18:59.835407+0000","last_undegraded":"2023-10-06T21:18:59.835407+0000","last_fullsized":"2023-10-06T21:18:59.835407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389002+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:18:57.389002+0000","last_peered":"2023-10-06T21:18:57.389002+0000","last_clean":"2023-10-06T21:18:57.389002+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:18:57.389002+0000","last_undegraded":"2023-10-06T21:18:57.389002+0000","last_fullsized":"2023-10-06T21:18:57.389002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.831557+0000","last_change":"2023-10-06T21:18:53.850300+0000","last_active":"2023-10-06T21:18:58.831557+0000","last_peered":"2023-10-06T21:18:58.831557+0000","last_clean":"2023-10-06T21:18:58.831557+0000","last_became_active":"2023-10-06T21:18:53.849911+0000","last_became_peered":"2023-10-06T21:18:53.849911+0000","last_unstale":"2023-10-06T21:18:58.831557+0000","last_undegraded":"2023-10-06T21:18:58.831557+0000","last_fullsized":"2023-10-06T21:18:58.831557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:44:27.734481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697985+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:18:56.697985+0000","last_peered":"2023-10-06T21:18:56.697985+0000","last_clean":"2023-10-06T21:18:56.697985+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:18:56.697985+0000","last_undegraded":"2023-10-06T21:18:56.697985+0000","last_fullsized":"2023-10-06T21:18:56.697985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697638+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:18:56.697638+0000","last_peered":"2023-10-06T21:18:56.697638+0000","last_clean":"2023-10-06T21:18:56.697638+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:18:56.697638+0000","last_undegraded":"2023-10-06T21:18:56.697638+0000","last_fullsized":"2023-10-06T21:18:56.697638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356245+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:18:56.356245+0000","last_peered":"2023-10-06T21:18:56.356245+0000","last_clean":"2023-10-06T21:18:56.356245+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:18:56.356245+0000","last_undegraded":"2023-10-06T21:18:56.356245+0000","last_fullsized":"2023-10-06T21:18:56.356245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698197+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:18:56.698197+0000","last_peered":"2023-10-06T21:18:56.698197+0000","last_clean":"2023-10-06T21:18:56.698197+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:18:56.698197+0000","last_undegraded":"2023-10-06T21:18:56.698197+0000","last_fullsized":"2023-10-06T21:18:56.698197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.799250+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:18:56.799250+0000","last_peered":"2023-10-06T21:18:56.799250+0000","last_clean":"2023-10-06T21:18:56.799250+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:18:56.799250+0000","last_undegraded":"2023-10-06T21:18:56.799250+0000","last_fullsized":"2023-10-06T21:18:56.799250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389158+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:18:57.389158+0000","last_peered":"2023-10-06T21:18:57.389158+0000","last_clean":"2023-10-06T21:18:57.389158+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:18:57.389158+0000","last_undegraded":"2023-10-06T21:18:57.389158+0000","last_fullsized":"2023-10-06T21:18:57.389158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682587+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:18:56.682587+0000","last_peered":"2023-10-06T21:18:56.682587+0000","last_clean":"2023-10-06T21:18:56.682587+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:18:56.682587+0000","last_undegraded":"2023-10-06T21:18:56.682587+0000","last_fullsized":"2023-10-06T21:18:56.682587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697857+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:18:56.697857+0000","last_peered":"2023-10-06T21:18:56.697857+0000","last_clean":"2023-10-06T21:18:56.697857+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:18:56.697857+0000","last_undegraded":"2023-10-06T21:18:56.697857+0000","last_fullsized":"2023-10-06T21:18:56.697857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.831415+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:18:58.831415+0000","last_peered":"2023-10-06T21:18:58.831415+0000","last_clean":"2023-10-06T21:18:58.831415+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:18:58.831415+0000","last_undegraded":"2023-10-06T21:18:58.831415+0000","last_fullsized":"2023-10-06T21:18:58.831415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836116+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:18:55.836116+0000","last_peered":"2023-10-06T21:18:55.836116+0000","last_clean":"2023-10-06T21:18:55.836116+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:18:55.836116+0000","last_undegraded":"2023-10-06T21:18:55.836116+0000","last_fullsized":"2023-10-06T21:18:55.836116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.836823+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:18:59.836823+0000","last_peered":"2023-10-06T21:18:59.836823+0000","last_clean":"2023-10-06T21:18:59.836823+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:18:59.836823+0000","last_undegraded":"2023-10-06T21:18:59.836823+0000","last_fullsized":"2023-10-06T21:18:59.836823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":318,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.837351+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:18:54.837351+0000","last_peered":"2023-10-06T21:18:54.837351+0000","last_clean":"2023-10-06T21:18:54.837351+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:18:54.837351+0000","last_undegraded":"2023-10-06T21:18:54.837351+0000","last_fullsized":"2023-10-06T21:18:54.837351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.696803+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:18:56.696803+0000","last_peered":"2023-10-06T21:18:56.696803+0000","last_clean":"2023-10-06T21:18:56.696803+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:18:56.696803+0000","last_undegraded":"2023-10-06T21:18:56.696803+0000","last_fullsized":"2023-10-06T21:18:56.696803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.828115+0000","last_change":"2023-10-06T21:18:53.853472+0000","last_active":"2023-10-06T21:18:58.828115+0000","last_peered":"2023-10-06T21:18:58.828115+0000","last_clean":"2023-10-06T21:18:58.828115+0000","last_became_active":"2023-10-06T21:18:53.853165+0000","last_became_peered":"2023-10-06T21:18:53.853165+0000","last_unstale":"2023-10-06T21:18:58.828115+0000","last_undegraded":"2023-10-06T21:18:58.828115+0000","last_fullsized":"2023-10-06T21:18:58.828115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:08:24.148890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895944+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:18:57.895944+0000","last_peered":"2023-10-06T21:18:57.895944+0000","last_clean":"2023-10-06T21:18:57.895944+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:18:57.895944+0000","last_undegraded":"2023-10-06T21:18:57.895944+0000","last_fullsized":"2023-10-06T21:18:57.895944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697237+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:18:56.697237+0000","last_peered":"2023-10-06T21:18:56.697237+0000","last_clean":"2023-10-06T21:18:56.697237+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:18:56.697237+0000","last_undegraded":"2023-10-06T21:18:56.697237+0000","last_fullsized":"2023-10-06T21:18:56.697237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698432+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:18:56.698432+0000","last_peered":"2023-10-06T21:18:56.698432+0000","last_clean":"2023-10-06T21:18:56.698432+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:18:56.698432+0000","last_undegraded":"2023-10-06T21:18:56.698432+0000","last_fullsized":"2023-10-06T21:18:56.698432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836190+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:18:55.836190+0000","last_peered":"2023-10-06T21:18:55.836190+0000","last_clean":"2023-10-06T21:18:55.836190+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:18:55.836190+0000","last_undegraded":"2023-10-06T21:18:55.836190+0000","last_fullsized":"2023-10-06T21:18:55.836190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.843143+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:19:00.843143+0000","last_peered":"2023-10-06T21:19:00.843143+0000","last_clean":"2023-10-06T21:19:00.843143+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:19:00.843143+0000","last_undegraded":"2023-10-06T21:19:00.843143+0000","last_fullsized":"2023-10-06T21:19:00.843143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697861+0000","last_change":"2023-10-06T21:18:54.857218+0000","last_active":"2023-10-06T21:18:56.697861+0000","last_peered":"2023-10-06T21:18:56.697861+0000","last_clean":"2023-10-06T21:18:56.697861+0000","last_became_active":"2023-10-06T21:18:54.856856+0000","last_became_peered":"2023-10-06T21:18:54.856856+0000","last_unstale":"2023-10-06T21:18:56.697861+0000","last_undegraded":"2023-10-06T21:18:56.697861+0000","last_fullsized":"2023-10-06T21:18:56.697861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:41.188229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895681+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:18:57.895681+0000","last_peered":"2023-10-06T21:18:57.895681+0000","last_clean":"2023-10-06T21:18:57.895681+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:18:57.895681+0000","last_undegraded":"2023-10-06T21:18:57.895681+0000","last_fullsized":"2023-10-06T21:18:57.895681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697161+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:18:56.697161+0000","last_peered":"2023-10-06T21:18:56.697161+0000","last_clean":"2023-10-06T21:18:56.697161+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:18:56.697161+0000","last_undegraded":"2023-10-06T21:18:56.697161+0000","last_fullsized":"2023-10-06T21:18:56.697161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.896562+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:18:57.896562+0000","last_peered":"2023-10-06T21:18:57.896562+0000","last_clean":"2023-10-06T21:18:57.896562+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:18:57.896562+0000","last_undegraded":"2023-10-06T21:18:57.896562+0000","last_fullsized":"2023-10-06T21:18:57.896562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356068+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:18:56.356068+0000","last_peered":"2023-10-06T21:18:56.356068+0000","last_clean":"2023-10-06T21:18:56.356068+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:18:56.356068+0000","last_undegraded":"2023-10-06T21:18:56.356068+0000","last_fullsized":"2023-10-06T21:18:56.356068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.838071+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:18:54.838071+0000","last_peered":"2023-10-06T21:18:54.838071+0000","last_clean":"2023-10-06T21:18:54.838071+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:18:54.838071+0000","last_undegraded":"2023-10-06T21:18:54.838071+0000","last_fullsized":"2023-10-06T21:18:54.838071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835866+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:18:55.835866+0000","last_peered":"2023-10-06T21:18:55.835866+0000","last_clean":"2023-10-06T21:18:55.835866+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:18:55.835866+0000","last_undegraded":"2023-10-06T21:18:55.835866+0000","last_fullsized":"2023-10-06T21:18:55.835866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895755+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:18:57.895755+0000","last_peered":"2023-10-06T21:18:57.895755+0000","last_clean":"2023-10-06T21:18:57.895755+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:18:57.895755+0000","last_undegraded":"2023-10-06T21:18:57.895755+0000","last_fullsized":"2023-10-06T21:18:57.895755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389347+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:18:57.389347+0000","last_peered":"2023-10-06T21:18:57.389347+0000","last_clean":"2023-10-06T21:18:57.389347+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:18:57.389347+0000","last_undegraded":"2023-10-06T21:18:57.389347+0000","last_fullsized":"2023-10-06T21:18:57.389347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.838565+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:18:54.838565+0000","last_peered":"2023-10-06T21:18:54.838565+0000","last_clean":"2023-10-06T21:18:54.838565+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:18:54.838565+0000","last_undegraded":"2023-10-06T21:18:54.838565+0000","last_fullsized":"2023-10-06T21:18:54.838565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389508+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:18:57.389508+0000","last_peered":"2023-10-06T21:18:57.389508+0000","last_clean":"2023-10-06T21:18:57.389508+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:18:57.389508+0000","last_undegraded":"2023-10-06T21:18:57.389508+0000","last_fullsized":"2023-10-06T21:18:57.389508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835786+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:18:59.835786+0000","last_peered":"2023-10-06T21:18:59.835786+0000","last_clean":"2023-10-06T21:18:59.835786+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:18:59.835786+0000","last_undegraded":"2023-10-06T21:18:59.835786+0000","last_fullsized":"2023-10-06T21:18:59.835786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389216+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:18:57.389216+0000","last_peered":"2023-10-06T21:18:57.389216+0000","last_clean":"2023-10-06T21:18:57.389216+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:18:57.389216+0000","last_undegraded":"2023-10-06T21:18:57.389216+0000","last_fullsized":"2023-10-06T21:18:57.389216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.873606+0000","last_change":"2023-10-06T21:18:53.838670+0000","last_active":"2023-10-06T21:18:54.873606+0000","last_peered":"2023-10-06T21:18:54.873606+0000","last_clean":"2023-10-06T21:18:54.873606+0000","last_became_active":"2023-10-06T21:18:53.838228+0000","last_became_peered":"2023-10-06T21:18:53.838228+0000","last_unstale":"2023-10-06T21:18:54.873606+0000","last_undegraded":"2023-10-06T21:18:54.873606+0000","last_fullsized":"2023-10-06T21:18:54.873606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:09:31.132174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389409+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:18:57.389409+0000","last_peered":"2023-10-06T21:18:57.389409+0000","last_clean":"2023-10-06T21:18:57.389409+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:18:57.389409+0000","last_undegraded":"2023-10-06T21:18:57.389409+0000","last_fullsized":"2023-10-06T21:18:57.389409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356189+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:18:56.356189+0000","last_peered":"2023-10-06T21:18:56.356189+0000","last_clean":"2023-10-06T21:18:56.356189+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:18:56.356189+0000","last_undegraded":"2023-10-06T21:18:56.356189+0000","last_fullsized":"2023-10-06T21:18:56.356189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836454+0000","last_change":"2023-10-06T21:18:54.850526+0000","last_active":"2023-10-06T21:18:55.836454+0000","last_peered":"2023-10-06T21:18:55.836454+0000","last_clean":"2023-10-06T21:18:55.836454+0000","last_became_active":"2023-10-06T21:18:54.850080+0000","last_became_peered":"2023-10-06T21:18:54.850080+0000","last_unstale":"2023-10-06T21:18:55.836454+0000","last_undegraded":"2023-10-06T21:18:55.836454+0000","last_fullsized":"2023-10-06T21:18:55.836454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:02:11.554480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836407+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:18:55.836407+0000","last_peered":"2023-10-06T21:18:55.836407+0000","last_clean":"2023-10-06T21:18:55.836407+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:18:55.836407+0000","last_undegraded":"2023-10-06T21:18:55.836407+0000","last_fullsized":"2023-10-06T21:18:55.836407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697750+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:18:56.697750+0000","last_peered":"2023-10-06T21:18:56.697750+0000","last_clean":"2023-10-06T21:18:56.697750+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:18:56.697750+0000","last_undegraded":"2023-10-06T21:18:56.697750+0000","last_fullsized":"2023-10-06T21:18:56.697750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697094+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:18:56.697094+0000","last_peered":"2023-10-06T21:18:56.697094+0000","last_clean":"2023-10-06T21:18:56.697094+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:18:56.697094+0000","last_undegraded":"2023-10-06T21:18:56.697094+0000","last_fullsized":"2023-10-06T21:18:56.697094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356310+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:18:56.356310+0000","last_peered":"2023-10-06T21:18:56.356310+0000","last_clean":"2023-10-06T21:18:56.356310+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:18:56.356310+0000","last_undegraded":"2023-10-06T21:18:56.356310+0000","last_fullsized":"2023-10-06T21:18:56.356310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836836+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:18:55.836836+0000","last_peered":"2023-10-06T21:18:55.836836+0000","last_clean":"2023-10-06T21:18:55.836836+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:18:55.836836+0000","last_undegraded":"2023-10-06T21:18:55.836836+0000","last_fullsized":"2023-10-06T21:18:55.836836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835839+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:18:55.835839+0000","last_peered":"2023-10-06T21:18:55.835839+0000","last_clean":"2023-10-06T21:18:55.835839+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:18:55.835839+0000","last_undegraded":"2023-10-06T21:18:55.835839+0000","last_fullsized":"2023-10-06T21:18:55.835839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697860+0000","last_change":"2023-10-06T21:18:54.847695+0000","last_active":"2023-10-06T21:18:56.697860+0000","last_peered":"2023-10-06T21:18:56.697860+0000","last_clean":"2023-10-06T21:18:56.697860+0000","last_became_active":"2023-10-06T21:18:54.847297+0000","last_became_peered":"2023-10-06T21:18:54.847297+0000","last_unstale":"2023-10-06T21:18:56.697860+0000","last_undegraded":"2023-10-06T21:18:56.697860+0000","last_fullsized":"2023-10-06T21:18:56.697860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:07:16.600335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835616+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:18:55.835616+0000","last_peered":"2023-10-06T21:18:55.835616+0000","last_clean":"2023-10-06T21:18:55.835616+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:18:55.835616+0000","last_undegraded":"2023-10-06T21:18:55.835616+0000","last_fullsized":"2023-10-06T21:18:55.835616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697337+0000","last_change":"2023-10-06T21:18:54.859301+0000","last_active":"2023-10-06T21:18:56.697337+0000","last_peered":"2023-10-06T21:18:56.697337+0000","last_clean":"2023-10-06T21:18:56.697337+0000","last_became_active":"2023-10-06T21:18:54.858936+0000","last_became_peered":"2023-10-06T21:18:54.858936+0000","last_unstale":"2023-10-06T21:18:56.697337+0000","last_undegraded":"2023-10-06T21:18:56.697337+0000","last_fullsized":"2023-10-06T21:18:56.697337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:29:35.239801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697134+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:18:56.697134+0000","last_peered":"2023-10-06T21:18:56.697134+0000","last_clean":"2023-10-06T21:18:56.697134+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:18:56.697134+0000","last_undegraded":"2023-10-06T21:18:56.697134+0000","last_fullsized":"2023-10-06T21:18:56.697134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356216+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:18:56.356216+0000","last_peered":"2023-10-06T21:18:56.356216+0000","last_clean":"2023-10-06T21:18:56.356216+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:18:56.356216+0000","last_undegraded":"2023-10-06T21:18:56.356216+0000","last_fullsized":"2023-10-06T21:18:56.356216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23213,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":139,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":53674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1052623,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79400000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.755},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.70499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.016},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.85599999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8479999999999999,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89600000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":433372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.624},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.79400000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.753}]},{"osd":6,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.85599999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.92200000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97699999999999998,"5min":1.284,"15min":1.284},"last":1.216},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0009999999999999,"5min":1.218,"15min":1.218},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72799999999999998},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.94199999999999995,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.48699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90300000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0209999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.71899999999999997},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.3580000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.74299999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.508,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82099999999999995,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.095,"5min":1.125,"15min":1.125},"last":1.3220000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.877,"15min":0.88200000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.266}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.4219999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.286},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.3879999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":1031979,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82999999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.871},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.67300000000000004},"min":{"1min":0.52000000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":1.044,"15min":1.044},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96999999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56000000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0269999999999999,"5min":1.073,"15min":1.073},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.57799999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82299999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.83999999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":1.0269999999999999,"15min":1.0649999999999999},"min":{"1min":0.53200000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.996,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.111,"5min":1.119,"15min":1.119},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.96299999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0720000000000001,"5min":1.121,"15min":1.121},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.81200000000000006,"5min":1.887,"15min":1.887},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.441},"max":{"1min":1.0469999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.92800000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.034,"5min":1.097,"15min":1.097},"last":0.442},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.66400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0509999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.88600000000000001,"15min":0.91800000000000004},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97399999999999998,"5min":1.827,"15min":1.827},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.65700000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.115,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.86799999999999999,"15min":0.85899999999999999},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1910000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82899999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.016,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:18:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94699999999999995,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.623,"15min":0.627},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.52700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.56399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.88100000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.79000000000000004,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.76100000000000001},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.97099999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0940000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.58899999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0409999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.1419999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.085}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.0289999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":443538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1499999999999999,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.034,"5min":1.738,"15min":1.738},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.204,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.749},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97799999999999998,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.95899999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.45100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.67900000000000005,"15min":0.64900000000000002},"min":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0329999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.67500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.129}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":1.008},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.88200000000000001,"15min":0.879},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.159}]},{"osd":9,"last update":"Fri Oct 6 21:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.181}]},{"osd":10,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.089}]},{"osd":11,"last update":"Fri Oct 6 21:18:59 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.218},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.1120000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.60199999999999998,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3650000000000002,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.747},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.626},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91200000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.997}]},{"osd":4,"last update":"Fri Oct 6 21:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50900000000000001,"15min":0.47999999999999998},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.498},{"interface":"front","average":{"1min":0.62,"5min":0.52400000000000002,"15min":0.47999999999999998},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.056,"15min":1.056},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0569999999999999,"5min":1.298,"15min":1.298},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.1040000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.077},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.42699999999999999,"15min":0.40200000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"min":{"1min":0.33200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.74199999999999999,"5min":1.101,"15min":1.101},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65800000000000003,"15min":0.625},"min":{"1min":0.51100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.026,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.995,"5min":1.135,"15min":1.135},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69699999999999995,"15min":0.64900000000000002},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.65300000000000002},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":0.999,"15min":0.999},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0609999999999999,"5min":1.206,"15min":1.206},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68000000000000005,"15min":0.66600000000000004},"min":{"1min":0.53100000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.26,"15min":1.26},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.81100000000000005,"15min":0.79800000000000004},"min":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81399999999999995,"15min":0.80500000000000005},"min":{"1min":0.66400000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.94899999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.034},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.93000000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86799999999999999,"5min":1.601,"15min":1.601},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.745},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67400000000000004,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1659999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.47199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.69399999999999995,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0049999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.758,"15min":1.758},"last":0.755},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.878}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.95799999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.91000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1034548,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83499999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65000000000000002,"15min":0.61799999999999999},"min":{"1min":0.501,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":1.089,"15min":1.089},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60099999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.96699999999999997,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.498,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93200000000000005,"5min":1.196,"15min":1.196},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89400000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97699999999999998,"5min":1.119,"15min":1.119},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.80000000000000004,"15min":0.83399999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90300000000000002,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":20729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":21103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:19:03.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 -- 172.21.15.31:0/2060297174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c102a40 msgr2=0x7f525c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 --2- 172.21.15.31:0/2060297174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c102a40 0x7f525c109360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f525400b0a0 tx=0x7f525400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 -- 172.21.15.31:0/2060297174 shutdown_connections 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 --2- 172.21.15.31:0/2060297174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c102a40 0x7f525c109360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 --2- 172.21.15.31:0/2060297174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c100ea0 0x7f525c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 --2- 172.21.15.31:0/2060297174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102080 0x7f525c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 -- 172.21.15.31:0/2060297174 >> 172.21.15.31:0/2060297174 conn(0x7f525c0fc420 msgr2=0x7f525c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 -- 172.21.15.31:0/2060297174 shutdown_connections 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.083+0000 7f52615ff700 1 -- 172.21.15.31:0/2060297174 wait complete. 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 Processor -- start 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 -- start start 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c102080 0x7f525c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102a40 0x7f525c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c1a0970 con 0x7f525c100ea0 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c1a0ab0 con 0x7f525c102080 2023-10-06T21:19:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f52615ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c1a0c40 con 0x7f525c102a40 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.084+0000 7f525affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54698/0 (socket says 172.21.15.31:54698) 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 -- 172.21.15.31:0/1233825659 learned_addr learned my addr 172.21.15.31:0/1233825659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525b7fe700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102a40 0x7f525c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525a7fc700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c102080 0x7f525c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 -- 172.21.15.31:0/1233825659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102a40 msgr2=0x7f525c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102a40 0x7f525c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 -- 172.21.15.31:0/1233825659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c102080 msgr2=0x7f525c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:04.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c102080 0x7f525c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 -- 172.21.15.31:0/1233825659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254009d00 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f525affd700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e970 tx=0x7f524c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c009790 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524c00eec0 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.085+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c1a0f20 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.086+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c003c10 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.086+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c1a1340 con 0x7f525c100ea0 2023-10-06T21:19:04.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.087+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c10c5c0 con 0x7f525c100ea0 2023-10-06T21:19:04.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.088+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f524c003db0 con 0x7f525c100ea0 2023-10-06T21:19:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.090+0000 7f5243fff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 0x7f52440a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.090+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f524c014070 con 0x7f525c100ea0 2023-10-06T21:19:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.090+0000 7f525a7fc700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 0x7f52440a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.091+0000 7f525a7fc700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 0x7f52440a3a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f525c100bc0 tx=0x7f5250009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.094+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f524c096600 con 0x7f525c100ea0 2023-10-06T21:19:04.151 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:19:03 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b[140597]: 2023-10-06T21:19:03.804+0000 7fa27b688700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-06T21:19:04.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:03 smithi142 ceph-mon[102951]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:04.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:03 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/309108064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:03 smithi142 ceph-mon[102951]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-06T21:19:04.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:03 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3382474946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:03 smithi031 ceph-mon[98539]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:03 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/309108064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:03 smithi031 ceph-mon[98539]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-06T21:19:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:03 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3382474946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:03 smithi146 ceph-mon[102809]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:19:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:03 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/309108064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:03 smithi146 ceph-mon[102809]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-06T21:19:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:03 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3382474946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.300+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f525c108a50 con 0x7f52440a1550 2023-10-06T21:19:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.311+0000 7f5243fff700 1 -- 172.21.15.31:0/1233825659 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339925 (secure 0 0 0) 0x7f525c108a50 con 0x7f52440a1550 2023-10-06T21:19:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 msgr2=0x7f52440a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 0x7f52440a3a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f525c100bc0 tx=0x7f5250009260 comp rx=0 tx=0).stop 2023-10-06T21:19:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 msgr2=0x7f525c198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e970 tx=0x7f524c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 shutdown_connections 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f52440a1550 0x7f52440a3a10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f525c102a40 0x7f525c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f525c100ea0 0x7f525c198df0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 --2- 172.21.15.31:0/1233825659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525c102080 0x7f525c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 >> 172.21.15.31:0/1233825659 conn(0x7f525c0fc420 msgr2=0x7f525c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 shutdown_connections 2023-10-06T21:19:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:04.315+0000 7f52615ff700 1 -- 172.21.15.31:0/1233825659 wait complete. 2023-10-06T21:19:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:19:04.337 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2023-10-06T21:19:03.811801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614998,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":169,"num_read_kb":144,"num_write":355,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335484,"kb_used_data":11288,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606340,"statfs":{"total":1151940427776,"available":1151596892160,"internally_reserved":0,"allocated":11558912,"data_stored":7139276,"data_compressed":109033,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24630,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":174,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.982105"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356315+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:18:56.356315+0000","last_peered":"2023-10-06T21:18:56.356315+0000","last_clean":"2023-10-06T21:18:56.356315+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:18:56.356315+0000","last_undegraded":"2023-10-06T21:18:56.356315+0000","last_fullsized":"2023-10-06T21:18:56.356315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697863+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:18:56.697863+0000","last_peered":"2023-10-06T21:18:56.697863+0000","last_clean":"2023-10-06T21:18:56.697863+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:18:56.697863+0000","last_undegraded":"2023-10-06T21:18:56.697863+0000","last_fullsized":"2023-10-06T21:18:56.697863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697629+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:18:56.697629+0000","last_peered":"2023-10-06T21:18:56.697629+0000","last_clean":"2023-10-06T21:18:56.697629+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:18:56.697629+0000","last_undegraded":"2023-10-06T21:18:56.697629+0000","last_fullsized":"2023-10-06T21:18:56.697629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.838022+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:18:55.838022+0000","last_peered":"2023-10-06T21:18:55.838022+0000","last_clean":"2023-10-06T21:18:55.838022+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:18:55.838022+0000","last_undegraded":"2023-10-06T21:18:55.838022+0000","last_fullsized":"2023-10-06T21:18:55.838022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697329+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:18:56.697329+0000","last_peered":"2023-10-06T21:18:56.697329+0000","last_clean":"2023-10-06T21:18:56.697329+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:18:56.697329+0000","last_undegraded":"2023-10-06T21:18:56.697329+0000","last_fullsized":"2023-10-06T21:18:56.697329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682507+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:18:56.682507+0000","last_peered":"2023-10-06T21:18:56.682507+0000","last_clean":"2023-10-06T21:18:56.682507+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:18:56.682507+0000","last_undegraded":"2023-10-06T21:18:56.682507+0000","last_fullsized":"2023-10-06T21:18:56.682507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389064+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:18:57.389064+0000","last_peered":"2023-10-06T21:18:57.389064+0000","last_clean":"2023-10-06T21:18:57.389064+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:18:57.389064+0000","last_undegraded":"2023-10-06T21:18:57.389064+0000","last_fullsized":"2023-10-06T21:18:57.389064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835555+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:18:55.835555+0000","last_peered":"2023-10-06T21:18:55.835555+0000","last_clean":"2023-10-06T21:18:55.835555+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:18:55.835555+0000","last_undegraded":"2023-10-06T21:18:55.835555+0000","last_fullsized":"2023-10-06T21:18:55.835555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697617+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:18:56.697617+0000","last_peered":"2023-10-06T21:18:56.697617+0000","last_clean":"2023-10-06T21:18:56.697617+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:18:56.697617+0000","last_undegraded":"2023-10-06T21:18:56.697617+0000","last_fullsized":"2023-10-06T21:18:56.697617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682779+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:18:56.682779+0000","last_peered":"2023-10-06T21:18:56.682779+0000","last_clean":"2023-10-06T21:18:56.682779+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:18:56.682779+0000","last_undegraded":"2023-10-06T21:18:56.682779+0000","last_fullsized":"2023-10-06T21:18:56.682779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389285+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:18:57.389285+0000","last_peered":"2023-10-06T21:18:57.389285+0000","last_clean":"2023-10-06T21:18:57.389285+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:18:57.389285+0000","last_undegraded":"2023-10-06T21:18:57.389285+0000","last_fullsized":"2023-10-06T21:18:57.389285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.837899+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:18:55.837899+0000","last_peered":"2023-10-06T21:18:55.837899+0000","last_clean":"2023-10-06T21:18:55.837899+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:18:55.837899+0000","last_undegraded":"2023-10-06T21:18:55.837899+0000","last_fullsized":"2023-10-06T21:18:55.837899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697092+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:18:56.697092+0000","last_peered":"2023-10-06T21:18:56.697092+0000","last_clean":"2023-10-06T21:18:56.697092+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:18:56.697092+0000","last_undegraded":"2023-10-06T21:18:56.697092+0000","last_fullsized":"2023-10-06T21:18:56.697092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698017+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:18:56.698017+0000","last_peered":"2023-10-06T21:18:56.698017+0000","last_clean":"2023-10-06T21:18:56.698017+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:18:56.698017+0000","last_undegraded":"2023-10-06T21:18:56.698017+0000","last_fullsized":"2023-10-06T21:18:56.698017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697471+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:18:56.697471+0000","last_peered":"2023-10-06T21:18:56.697471+0000","last_clean":"2023-10-06T21:18:56.697471+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:18:56.697471+0000","last_undegraded":"2023-10-06T21:18:56.697471+0000","last_fullsized":"2023-10-06T21:18:56.697471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697171+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:18:56.697171+0000","last_peered":"2023-10-06T21:18:56.697171+0000","last_clean":"2023-10-06T21:18:56.697171+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:18:56.697171+0000","last_undegraded":"2023-10-06T21:18:56.697171+0000","last_fullsized":"2023-10-06T21:18:56.697171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356400+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:18:56.356400+0000","last_peered":"2023-10-06T21:18:56.356400+0000","last_clean":"2023-10-06T21:18:56.356400+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:18:56.356400+0000","last_undegraded":"2023-10-06T21:18:56.356400+0000","last_fullsized":"2023-10-06T21:18:56.356400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698352+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:18:56.698352+0000","last_peered":"2023-10-06T21:18:56.698352+0000","last_clean":"2023-10-06T21:18:56.698352+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:18:56.698352+0000","last_undegraded":"2023-10-06T21:18:56.698352+0000","last_fullsized":"2023-10-06T21:18:56.698352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836559+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:18:55.836559+0000","last_peered":"2023-10-06T21:18:55.836559+0000","last_clean":"2023-10-06T21:18:55.836559+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:18:55.836559+0000","last_undegraded":"2023-10-06T21:18:55.836559+0000","last_fullsized":"2023-10-06T21:18:55.836559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.843760+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:19:00.843760+0000","last_peered":"2023-10-06T21:19:00.843760+0000","last_clean":"2023-10-06T21:19:00.843760+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:19:00.843760+0000","last_undegraded":"2023-10-06T21:19:00.843760+0000","last_fullsized":"2023-10-06T21:19:00.843760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835945+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:18:55.835945+0000","last_peered":"2023-10-06T21:18:55.835945+0000","last_clean":"2023-10-06T21:18:55.835945+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:18:55.835945+0000","last_undegraded":"2023-10-06T21:18:55.835945+0000","last_fullsized":"2023-10-06T21:18:55.835945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836108+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:18:55.836108+0000","last_peered":"2023-10-06T21:18:55.836108+0000","last_clean":"2023-10-06T21:18:55.836108+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:18:55.836108+0000","last_undegraded":"2023-10-06T21:18:55.836108+0000","last_fullsized":"2023-10-06T21:18:55.836108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698118+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:18:56.698118+0000","last_peered":"2023-10-06T21:18:56.698118+0000","last_clean":"2023-10-06T21:18:56.698118+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:18:56.698118+0000","last_undegraded":"2023-10-06T21:18:56.698118+0000","last_fullsized":"2023-10-06T21:18:56.698118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.828068+0000","last_change":"2023-10-06T21:18:53.852079+0000","last_active":"2023-10-06T21:18:58.828068+0000","last_peered":"2023-10-06T21:18:58.828068+0000","last_clean":"2023-10-06T21:18:58.828068+0000","last_became_active":"2023-10-06T21:18:53.851472+0000","last_became_peered":"2023-10-06T21:18:53.851472+0000","last_unstale":"2023-10-06T21:18:58.828068+0000","last_undegraded":"2023-10-06T21:18:58.828068+0000","last_fullsized":"2023-10-06T21:18:58.828068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:00:45.421872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697281+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:18:56.697281+0000","last_peered":"2023-10-06T21:18:56.697281+0000","last_clean":"2023-10-06T21:18:56.697281+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:18:56.697281+0000","last_undegraded":"2023-10-06T21:18:56.697281+0000","last_fullsized":"2023-10-06T21:18:56.697281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.876281+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:18:54.876281+0000","last_peered":"2023-10-06T21:18:54.876281+0000","last_clean":"2023-10-06T21:18:54.876281+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:18:54.876281+0000","last_undegraded":"2023-10-06T21:18:54.876281+0000","last_fullsized":"2023-10-06T21:18:54.876281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698250+0000","last_change":"2023-10-06T21:18:54.849038+0000","last_active":"2023-10-06T21:18:56.698250+0000","last_peered":"2023-10-06T21:18:56.698250+0000","last_clean":"2023-10-06T21:18:56.698250+0000","last_became_active":"2023-10-06T21:18:54.848645+0000","last_became_peered":"2023-10-06T21:18:54.848645+0000","last_unstale":"2023-10-06T21:18:56.698250+0000","last_undegraded":"2023-10-06T21:18:56.698250+0000","last_fullsized":"2023-10-06T21:18:56.698250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:21:01.954014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.823402+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:18:59.823402+0000","last_peered":"2023-10-06T21:18:59.823402+0000","last_clean":"2023-10-06T21:18:59.823402+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:18:59.823402+0000","last_undegraded":"2023-10-06T21:18:59.823402+0000","last_fullsized":"2023-10-06T21:18:59.823402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697489+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:18:56.697489+0000","last_peered":"2023-10-06T21:18:56.697489+0000","last_clean":"2023-10-06T21:18:56.697489+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:18:56.697489+0000","last_undegraded":"2023-10-06T21:18:56.697489+0000","last_fullsized":"2023-10-06T21:18:56.697489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835491+0000","last_change":"2023-10-06T21:18:53.852076+0000","last_active":"2023-10-06T21:18:59.835491+0000","last_peered":"2023-10-06T21:18:59.835491+0000","last_clean":"2023-10-06T21:18:59.835491+0000","last_became_active":"2023-10-06T21:18:53.851622+0000","last_became_peered":"2023-10-06T21:18:53.851622+0000","last_unstale":"2023-10-06T21:18:59.835491+0000","last_undegraded":"2023-10-06T21:18:59.835491+0000","last_fullsized":"2023-10-06T21:18:59.835491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:42.339452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698241+0000","last_change":"2023-10-06T21:18:54.846408+0000","last_active":"2023-10-06T21:18:56.698241+0000","last_peered":"2023-10-06T21:18:56.698241+0000","last_clean":"2023-10-06T21:18:56.698241+0000","last_became_active":"2023-10-06T21:18:54.846113+0000","last_became_peered":"2023-10-06T21:18:54.846113+0000","last_unstale":"2023-10-06T21:18:56.698241+0000","last_undegraded":"2023-10-06T21:18:56.698241+0000","last_fullsized":"2023-10-06T21:18:56.698241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:36.454140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.830159+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:18:58.830159+0000","last_peered":"2023-10-06T21:18:58.830159+0000","last_clean":"2023-10-06T21:18:58.830159+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:18:58.830159+0000","last_undegraded":"2023-10-06T21:18:58.830159+0000","last_fullsized":"2023-10-06T21:18:58.830159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835642+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:18:55.835642+0000","last_peered":"2023-10-06T21:18:55.835642+0000","last_clean":"2023-10-06T21:18:55.835642+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:18:55.835642+0000","last_undegraded":"2023-10-06T21:18:55.835642+0000","last_fullsized":"2023-10-06T21:18:55.835642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.833932+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:18:59.833932+0000","last_peered":"2023-10-06T21:18:59.833932+0000","last_clean":"2023-10-06T21:18:59.833932+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:18:59.833932+0000","last_undegraded":"2023-10-06T21:18:59.833932+0000","last_fullsized":"2023-10-06T21:18:59.833932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.698211+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:18:56.698211+0000","last_peered":"2023-10-06T21:18:56.698211+0000","last_clean":"2023-10-06T21:18:56.698211+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:18:56.698211+0000","last_undegraded":"2023-10-06T21:18:56.698211+0000","last_fullsized":"2023-10-06T21:18:56.698211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.836607+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:18:59.836607+0000","last_peered":"2023-10-06T21:18:59.836607+0000","last_clean":"2023-10-06T21:18:59.836607+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:18:59.836607+0000","last_undegraded":"2023-10-06T21:18:59.836607+0000","last_fullsized":"2023-10-06T21:18:59.836607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697569+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:18:56.697569+0000","last_peered":"2023-10-06T21:18:56.697569+0000","last_clean":"2023-10-06T21:18:56.697569+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:18:56.697569+0000","last_undegraded":"2023-10-06T21:18:56.697569+0000","last_fullsized":"2023-10-06T21:18:56.697569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895772+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:18:57.895772+0000","last_peered":"2023-10-06T21:18:57.895772+0000","last_clean":"2023-10-06T21:18:57.895772+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:18:57.895772+0000","last_undegraded":"2023-10-06T21:18:57.895772+0000","last_fullsized":"2023-10-06T21:18:57.895772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697891+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:18:56.697891+0000","last_peered":"2023-10-06T21:18:56.697891+0000","last_clean":"2023-10-06T21:18:56.697891+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:18:56.697891+0000","last_undegraded":"2023-10-06T21:18:56.697891+0000","last_fullsized":"2023-10-06T21:18:56.697891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.820062+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:18:58.820062+0000","last_peered":"2023-10-06T21:18:58.820062+0000","last_clean":"2023-10-06T21:18:58.820062+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:18:58.820062+0000","last_undegraded":"2023-10-06T21:18:58.820062+0000","last_fullsized":"2023-10-06T21:18:58.820062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697442+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:18:56.697442+0000","last_peered":"2023-10-06T21:18:56.697442+0000","last_clean":"2023-10-06T21:18:56.697442+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:18:56.697442+0000","last_undegraded":"2023-10-06T21:18:56.697442+0000","last_fullsized":"2023-10-06T21:18:56.697442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.861202+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:19:00.861202+0000","last_peered":"2023-10-06T21:19:00.861202+0000","last_clean":"2023-10-06T21:19:00.861202+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:19:00.861202+0000","last_undegraded":"2023-10-06T21:19:00.861202+0000","last_fullsized":"2023-10-06T21:19:00.861202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697565+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:18:56.697565+0000","last_peered":"2023-10-06T21:18:56.697565+0000","last_clean":"2023-10-06T21:18:56.697565+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:18:56.697565+0000","last_undegraded":"2023-10-06T21:18:56.697565+0000","last_fullsized":"2023-10-06T21:18:56.697565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.844286+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:19:00.844286+0000","last_peered":"2023-10-06T21:19:00.844286+0000","last_clean":"2023-10-06T21:19:00.844286+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:19:00.844286+0000","last_undegraded":"2023-10-06T21:19:00.844286+0000","last_fullsized":"2023-10-06T21:19:00.844286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697030+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:18:56.697030+0000","last_peered":"2023-10-06T21:18:56.697030+0000","last_clean":"2023-10-06T21:18:56.697030+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:18:56.697030+0000","last_undegraded":"2023-10-06T21:18:56.697030+0000","last_fullsized":"2023-10-06T21:18:56.697030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835118+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:18:59.835118+0000","last_peered":"2023-10-06T21:18:59.835118+0000","last_clean":"2023-10-06T21:18:59.835118+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:18:59.835118+0000","last_undegraded":"2023-10-06T21:18:59.835118+0000","last_fullsized":"2023-10-06T21:18:59.835118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389435+0000","last_change":"2023-10-06T21:18:54.860485+0000","last_active":"2023-10-06T21:18:57.389435+0000","last_peered":"2023-10-06T21:18:57.389435+0000","last_clean":"2023-10-06T21:18:57.389435+0000","last_became_active":"2023-10-06T21:18:54.860171+0000","last_became_peered":"2023-10-06T21:18:54.860171+0000","last_unstale":"2023-10-06T21:18:57.389435+0000","last_undegraded":"2023-10-06T21:18:57.389435+0000","last_fullsized":"2023-10-06T21:18:57.389435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:46:50.890861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835407+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:18:59.835407+0000","last_peered":"2023-10-06T21:18:59.835407+0000","last_clean":"2023-10-06T21:18:59.835407+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:18:59.835407+0000","last_undegraded":"2023-10-06T21:18:59.835407+0000","last_fullsized":"2023-10-06T21:18:59.835407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389002+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:18:57.389002+0000","last_peered":"2023-10-06T21:18:57.389002+0000","last_clean":"2023-10-06T21:18:57.389002+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:18:57.389002+0000","last_undegraded":"2023-10-06T21:18:57.389002+0000","last_fullsized":"2023-10-06T21:18:57.389002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.831557+0000","last_change":"2023-10-06T21:18:53.850300+0000","last_active":"2023-10-06T21:18:58.831557+0000","last_peered":"2023-10-06T21:18:58.831557+0000","last_clean":"2023-10-06T21:18:58.831557+0000","last_became_active":"2023-10-06T21:18:53.849911+0000","last_became_peered":"2023-10-06T21:18:53.849911+0000","last_unstale":"2023-10-06T21:18:58.831557+0000","last_undegraded":"2023-10-06T21:18:58.831557+0000","last_fullsized":"2023-10-06T21:18:58.831557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:44:27.734481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697985+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:18:56.697985+0000","last_peered":"2023-10-06T21:18:56.697985+0000","last_clean":"2023-10-06T21:18:56.697985+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:18:56.697985+0000","last_undegraded":"2023-10-06T21:18:56.697985+0000","last_fullsized":"2023-10-06T21:18:56.697985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697638+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:18:56.697638+0000","last_peered":"2023-10-06T21:18:56.697638+0000","last_clean":"2023-10-06T21:18:56.697638+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:18:56.697638+0000","last_undegraded":"2023-10-06T21:18:56.697638+0000","last_fullsized":"2023-10-06T21:18:56.697638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356245+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:18:56.356245+0000","last_peered":"2023-10-06T21:18:56.356245+0000","last_clean":"2023-10-06T21:18:56.356245+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:18:56.356245+0000","last_undegraded":"2023-10-06T21:18:56.356245+0000","last_fullsized":"2023-10-06T21:18:56.356245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.038024+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:18:57.038024+0000","last_peered":"2023-10-06T21:18:57.038024+0000","last_clean":"2023-10-06T21:18:57.038024+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:18:57.038024+0000","last_undegraded":"2023-10-06T21:18:57.038024+0000","last_fullsized":"2023-10-06T21:18:57.038024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.799250+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:18:56.799250+0000","last_peered":"2023-10-06T21:18:56.799250+0000","last_clean":"2023-10-06T21:18:56.799250+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:18:56.799250+0000","last_undegraded":"2023-10-06T21:18:56.799250+0000","last_fullsized":"2023-10-06T21:18:56.799250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389158+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:18:57.389158+0000","last_peered":"2023-10-06T21:18:57.389158+0000","last_clean":"2023-10-06T21:18:57.389158+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:18:57.389158+0000","last_undegraded":"2023-10-06T21:18:57.389158+0000","last_fullsized":"2023-10-06T21:18:57.389158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.682587+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:18:56.682587+0000","last_peered":"2023-10-06T21:18:56.682587+0000","last_clean":"2023-10-06T21:18:56.682587+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:18:56.682587+0000","last_undegraded":"2023-10-06T21:18:56.682587+0000","last_fullsized":"2023-10-06T21:18:56.682587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697857+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:18:56.697857+0000","last_peered":"2023-10-06T21:18:56.697857+0000","last_clean":"2023-10-06T21:18:56.697857+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:18:56.697857+0000","last_undegraded":"2023-10-06T21:18:56.697857+0000","last_fullsized":"2023-10-06T21:18:56.697857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.831415+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:18:58.831415+0000","last_peered":"2023-10-06T21:18:58.831415+0000","last_clean":"2023-10-06T21:18:58.831415+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:18:58.831415+0000","last_undegraded":"2023-10-06T21:18:58.831415+0000","last_fullsized":"2023-10-06T21:18:58.831415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836116+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:18:55.836116+0000","last_peered":"2023-10-06T21:18:55.836116+0000","last_clean":"2023-10-06T21:18:55.836116+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:18:55.836116+0000","last_undegraded":"2023-10-06T21:18:55.836116+0000","last_fullsized":"2023-10-06T21:18:55.836116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.836823+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:18:59.836823+0000","last_peered":"2023-10-06T21:18:59.836823+0000","last_clean":"2023-10-06T21:18:59.836823+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:18:59.836823+0000","last_undegraded":"2023-10-06T21:18:59.836823+0000","last_fullsized":"2023-10-06T21:18:59.836823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":318,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.837351+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:18:54.837351+0000","last_peered":"2023-10-06T21:18:54.837351+0000","last_clean":"2023-10-06T21:18:54.837351+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:18:54.837351+0000","last_undegraded":"2023-10-06T21:18:54.837351+0000","last_fullsized":"2023-10-06T21:18:54.837351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.696803+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:18:56.696803+0000","last_peered":"2023-10-06T21:18:56.696803+0000","last_clean":"2023-10-06T21:18:56.696803+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:18:56.696803+0000","last_undegraded":"2023-10-06T21:18:56.696803+0000","last_fullsized":"2023-10-06T21:18:56.696803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:58.828115+0000","last_change":"2023-10-06T21:18:53.853472+0000","last_active":"2023-10-06T21:18:58.828115+0000","last_peered":"2023-10-06T21:18:58.828115+0000","last_clean":"2023-10-06T21:18:58.828115+0000","last_became_active":"2023-10-06T21:18:53.853165+0000","last_became_peered":"2023-10-06T21:18:53.853165+0000","last_unstale":"2023-10-06T21:18:58.828115+0000","last_undegraded":"2023-10-06T21:18:58.828115+0000","last_fullsized":"2023-10-06T21:18:58.828115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:08:24.148890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895944+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:18:57.895944+0000","last_peered":"2023-10-06T21:18:57.895944+0000","last_clean":"2023-10-06T21:18:57.895944+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:18:57.895944+0000","last_undegraded":"2023-10-06T21:18:57.895944+0000","last_fullsized":"2023-10-06T21:18:57.895944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697237+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:18:56.697237+0000","last_peered":"2023-10-06T21:18:56.697237+0000","last_clean":"2023-10-06T21:18:56.697237+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:18:56.697237+0000","last_undegraded":"2023-10-06T21:18:56.697237+0000","last_fullsized":"2023-10-06T21:18:56.697237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.837586+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:18:59.837586+0000","last_peered":"2023-10-06T21:18:59.837586+0000","last_clean":"2023-10-06T21:18:59.837586+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:18:59.837586+0000","last_undegraded":"2023-10-06T21:18:59.837586+0000","last_fullsized":"2023-10-06T21:18:59.837586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836190+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:18:55.836190+0000","last_peered":"2023-10-06T21:18:55.836190+0000","last_clean":"2023-10-06T21:18:55.836190+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:18:55.836190+0000","last_undegraded":"2023-10-06T21:18:55.836190+0000","last_fullsized":"2023-10-06T21:18:55.836190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:00.843143+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:19:00.843143+0000","last_peered":"2023-10-06T21:19:00.843143+0000","last_clean":"2023-10-06T21:19:00.843143+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:19:00.843143+0000","last_undegraded":"2023-10-06T21:19:00.843143+0000","last_fullsized":"2023-10-06T21:19:00.843143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697861+0000","last_change":"2023-10-06T21:18:54.857218+0000","last_active":"2023-10-06T21:18:56.697861+0000","last_peered":"2023-10-06T21:18:56.697861+0000","last_clean":"2023-10-06T21:18:56.697861+0000","last_became_active":"2023-10-06T21:18:54.856856+0000","last_became_peered":"2023-10-06T21:18:54.856856+0000","last_unstale":"2023-10-06T21:18:56.697861+0000","last_undegraded":"2023-10-06T21:18:56.697861+0000","last_fullsized":"2023-10-06T21:18:56.697861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:41.188229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895681+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:18:57.895681+0000","last_peered":"2023-10-06T21:18:57.895681+0000","last_clean":"2023-10-06T21:18:57.895681+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:18:57.895681+0000","last_undegraded":"2023-10-06T21:18:57.895681+0000","last_fullsized":"2023-10-06T21:18:57.895681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697161+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:18:56.697161+0000","last_peered":"2023-10-06T21:18:56.697161+0000","last_clean":"2023-10-06T21:18:56.697161+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:18:56.697161+0000","last_undegraded":"2023-10-06T21:18:56.697161+0000","last_fullsized":"2023-10-06T21:18:56.697161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.896562+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:18:57.896562+0000","last_peered":"2023-10-06T21:18:57.896562+0000","last_clean":"2023-10-06T21:18:57.896562+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:18:57.896562+0000","last_undegraded":"2023-10-06T21:18:57.896562+0000","last_fullsized":"2023-10-06T21:18:57.896562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356068+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:18:56.356068+0000","last_peered":"2023-10-06T21:18:56.356068+0000","last_clean":"2023-10-06T21:18:56.356068+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:18:56.356068+0000","last_undegraded":"2023-10-06T21:18:56.356068+0000","last_fullsized":"2023-10-06T21:18:56.356068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835584+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:18:59.835584+0000","last_peered":"2023-10-06T21:18:59.835584+0000","last_clean":"2023-10-06T21:18:59.835584+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:18:59.835584+0000","last_undegraded":"2023-10-06T21:18:59.835584+0000","last_fullsized":"2023-10-06T21:18:59.835584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835866+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:18:55.835866+0000","last_peered":"2023-10-06T21:18:55.835866+0000","last_clean":"2023-10-06T21:18:55.835866+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:18:55.835866+0000","last_undegraded":"2023-10-06T21:18:55.835866+0000","last_fullsized":"2023-10-06T21:18:55.835866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.895755+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:18:57.895755+0000","last_peered":"2023-10-06T21:18:57.895755+0000","last_clean":"2023-10-06T21:18:57.895755+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:18:57.895755+0000","last_undegraded":"2023-10-06T21:18:57.895755+0000","last_fullsized":"2023-10-06T21:18:57.895755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389347+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:18:57.389347+0000","last_peered":"2023-10-06T21:18:57.389347+0000","last_clean":"2023-10-06T21:18:57.389347+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:18:57.389347+0000","last_undegraded":"2023-10-06T21:18:57.389347+0000","last_fullsized":"2023-10-06T21:18:57.389347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:19:01.848810+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:19:01.848810+0000","last_peered":"2023-10-06T21:19:01.848810+0000","last_clean":"2023-10-06T21:19:01.848810+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:19:01.848810+0000","last_undegraded":"2023-10-06T21:19:01.848810+0000","last_fullsized":"2023-10-06T21:19:01.848810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389508+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:18:57.389508+0000","last_peered":"2023-10-06T21:18:57.389508+0000","last_clean":"2023-10-06T21:18:57.389508+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:18:57.389508+0000","last_undegraded":"2023-10-06T21:18:57.389508+0000","last_fullsized":"2023-10-06T21:18:57.389508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:59.835786+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:18:59.835786+0000","last_peered":"2023-10-06T21:18:59.835786+0000","last_clean":"2023-10-06T21:18:59.835786+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:18:59.835786+0000","last_undegraded":"2023-10-06T21:18:59.835786+0000","last_fullsized":"2023-10-06T21:18:59.835786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389216+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:18:57.389216+0000","last_peered":"2023-10-06T21:18:57.389216+0000","last_clean":"2023-10-06T21:18:57.389216+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:18:57.389216+0000","last_undegraded":"2023-10-06T21:18:57.389216+0000","last_fullsized":"2023-10-06T21:18:57.389216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:18:54.873606+0000","last_change":"2023-10-06T21:18:53.838670+0000","last_active":"2023-10-06T21:18:54.873606+0000","last_peered":"2023-10-06T21:18:54.873606+0000","last_clean":"2023-10-06T21:18:54.873606+0000","last_became_active":"2023-10-06T21:18:53.838228+0000","last_became_peered":"2023-10-06T21:18:53.838228+0000","last_unstale":"2023-10-06T21:18:54.873606+0000","last_undegraded":"2023-10-06T21:18:54.873606+0000","last_fullsized":"2023-10-06T21:18:54.873606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:09:31.132174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:57.389409+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:18:57.389409+0000","last_peered":"2023-10-06T21:18:57.389409+0000","last_clean":"2023-10-06T21:18:57.389409+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:18:57.389409+0000","last_undegraded":"2023-10-06T21:18:57.389409+0000","last_fullsized":"2023-10-06T21:18:57.389409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356189+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:18:56.356189+0000","last_peered":"2023-10-06T21:18:56.356189+0000","last_clean":"2023-10-06T21:18:56.356189+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:18:56.356189+0000","last_undegraded":"2023-10-06T21:18:56.356189+0000","last_fullsized":"2023-10-06T21:18:56.356189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836454+0000","last_change":"2023-10-06T21:18:54.850526+0000","last_active":"2023-10-06T21:18:55.836454+0000","last_peered":"2023-10-06T21:18:55.836454+0000","last_clean":"2023-10-06T21:18:55.836454+0000","last_became_active":"2023-10-06T21:18:54.850080+0000","last_became_peered":"2023-10-06T21:18:54.850080+0000","last_unstale":"2023-10-06T21:18:55.836454+0000","last_undegraded":"2023-10-06T21:18:55.836454+0000","last_fullsized":"2023-10-06T21:18:55.836454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:02:11.554480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836407+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:18:55.836407+0000","last_peered":"2023-10-06T21:18:55.836407+0000","last_clean":"2023-10-06T21:18:55.836407+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:18:55.836407+0000","last_undegraded":"2023-10-06T21:18:55.836407+0000","last_fullsized":"2023-10-06T21:18:55.836407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697750+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:18:56.697750+0000","last_peered":"2023-10-06T21:18:56.697750+0000","last_clean":"2023-10-06T21:18:56.697750+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:18:56.697750+0000","last_undegraded":"2023-10-06T21:18:56.697750+0000","last_fullsized":"2023-10-06T21:18:56.697750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697094+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:18:56.697094+0000","last_peered":"2023-10-06T21:18:56.697094+0000","last_clean":"2023-10-06T21:18:56.697094+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:18:56.697094+0000","last_undegraded":"2023-10-06T21:18:56.697094+0000","last_fullsized":"2023-10-06T21:18:56.697094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356310+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:18:56.356310+0000","last_peered":"2023-10-06T21:18:56.356310+0000","last_clean":"2023-10-06T21:18:56.356310+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:18:56.356310+0000","last_undegraded":"2023-10-06T21:18:56.356310+0000","last_fullsized":"2023-10-06T21:18:56.356310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.836836+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:18:55.836836+0000","last_peered":"2023-10-06T21:18:55.836836+0000","last_clean":"2023-10-06T21:18:55.836836+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:18:55.836836+0000","last_undegraded":"2023-10-06T21:18:55.836836+0000","last_fullsized":"2023-10-06T21:18:55.836836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835839+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:18:55.835839+0000","last_peered":"2023-10-06T21:18:55.835839+0000","last_clean":"2023-10-06T21:18:55.835839+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:18:55.835839+0000","last_undegraded":"2023-10-06T21:18:55.835839+0000","last_fullsized":"2023-10-06T21:18:55.835839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697860+0000","last_change":"2023-10-06T21:18:54.847695+0000","last_active":"2023-10-06T21:18:56.697860+0000","last_peered":"2023-10-06T21:18:56.697860+0000","last_clean":"2023-10-06T21:18:56.697860+0000","last_became_active":"2023-10-06T21:18:54.847297+0000","last_became_peered":"2023-10-06T21:18:54.847297+0000","last_unstale":"2023-10-06T21:18:56.697860+0000","last_undegraded":"2023-10-06T21:18:56.697860+0000","last_fullsized":"2023-10-06T21:18:56.697860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:07:16.600335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:55.835616+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:18:55.835616+0000","last_peered":"2023-10-06T21:18:55.835616+0000","last_clean":"2023-10-06T21:18:55.835616+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:18:55.835616+0000","last_undegraded":"2023-10-06T21:18:55.835616+0000","last_fullsized":"2023-10-06T21:18:55.835616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697337+0000","last_change":"2023-10-06T21:18:54.859301+0000","last_active":"2023-10-06T21:18:56.697337+0000","last_peered":"2023-10-06T21:18:56.697337+0000","last_clean":"2023-10-06T21:18:56.697337+0000","last_became_active":"2023-10-06T21:18:54.858936+0000","last_became_peered":"2023-10-06T21:18:54.858936+0000","last_unstale":"2023-10-06T21:18:56.697337+0000","last_undegraded":"2023-10-06T21:18:56.697337+0000","last_fullsized":"2023-10-06T21:18:56.697337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:29:35.239801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.697134+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:18:56.697134+0000","last_peered":"2023-10-06T21:18:56.697134+0000","last_clean":"2023-10-06T21:18:56.697134+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:18:56.697134+0000","last_undegraded":"2023-10-06T21:18:56.697134+0000","last_fullsized":"2023-10-06T21:18:56.697134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:18:56.356216+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:18:56.356216+0000","last_peered":"2023-10-06T21:18:56.356216+0000","last_clean":"2023-10-06T21:18:56.356216+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:18:56.356216+0000","last_undegraded":"2023-10-06T21:18:56.356216+0000","last_fullsized":"2023-10-06T21:18:56.356216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24507,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":173,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":97427,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0209999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.92700000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.087}]},{"osd":10,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1052623,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79400000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.755},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.70499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.016},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.85599999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8479999999999999,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89600000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":442482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.505},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.54800000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.98399999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:18:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.93400000000000005}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97699999999999998,"5min":1.284,"15min":1.284},"last":1.216},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0009999999999999,"5min":1.218,"15min":1.218},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72799999999999998},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.94199999999999995,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.48699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90300000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0209999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.71899999999999997},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.3580000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.74299999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.508,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82099999999999995,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.095,"5min":1.125,"15min":1.125},"last":1.3220000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.877,"15min":0.88200000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.266}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.4219999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.286},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.3879999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":1031979,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82999999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.871},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.67300000000000004},"min":{"1min":0.52000000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":1.044,"15min":1.044},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96999999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56000000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0269999999999999,"5min":1.073,"15min":1.073},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.80600000000000005,"15min":0.76900000000000002},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.57799999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82299999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.83999999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":1.0269999999999999,"15min":1.0649999999999999},"min":{"1min":0.53200000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.996,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.111,"5min":1.119,"15min":1.119},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.96299999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0720000000000001,"5min":1.121,"15min":1.121},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.81200000000000006,"5min":1.887,"15min":1.887},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.441},"max":{"1min":1.0469999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.92800000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.034,"5min":1.097,"15min":1.097},"last":0.442},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.66400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0509999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.88600000000000001,"15min":0.91800000000000004},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97399999999999998,"5min":1.827,"15min":1.827},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.65700000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.115,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.86799999999999999,"15min":0.85899999999999999},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1910000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82899999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.016,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:18:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94699999999999995,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.623,"15min":0.627},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.621},"min":{"1min":0.52700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.56399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.88100000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.79000000000000004,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.76100000000000001},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.97099999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0940000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.58899999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0409999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.1419999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.085}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.0289999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":443538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1499999999999999,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.034,"5min":1.738,"15min":1.738},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.204,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.749},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97799999999999998,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.95899999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.45100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.67900000000000005,"15min":0.64900000000000002},"min":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0329999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.67500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.129}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":1.008},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.88200000000000001,"15min":0.879},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.159}]},{"osd":9,"last update":"Fri Oct 6 21:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.181}]},{"osd":10,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.089}]},{"osd":11,"last update":"Fri Oct 6 21:18:59 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.218},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.1120000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480779,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.60199999999999998,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3650000000000002,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.626},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91200000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50900000000000001,"15min":0.47999999999999998},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.52400000000000002,"15min":0.47999999999999998},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.056,"15min":1.056},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0569999999999999,"5min":1.298,"15min":1.298},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.92900000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.82099999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.42699999999999999,"15min":0.40200000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69099999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"min":{"1min":0.33200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.74199999999999999,"5min":1.101,"15min":1.101},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65800000000000003,"15min":0.625},"min":{"1min":0.51100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.026,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.66700000000000004,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.995,"5min":1.135,"15min":1.135},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69699999999999995,"15min":0.64900000000000002},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.65300000000000002},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":0.999,"15min":0.999},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0609999999999999,"5min":1.206,"15min":1.206},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68000000000000005,"15min":0.66600000000000004},"min":{"1min":0.53100000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.55300000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.26,"15min":1.26},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.81100000000000005,"15min":0.79800000000000004},"min":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81399999999999995,"15min":0.80500000000000005},"min":{"1min":0.66400000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.94899999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.034},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.93000000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86799999999999999,"5min":1.601,"15min":1.601},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.745},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67400000000000004,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1659999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.47199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.69399999999999995,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0049999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.758,"15min":1.758},"last":0.755},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.878}]},{"osd":9,"last update":"Fri Oct 6 21:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.95799999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.91000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1034548,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83499999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65000000000000002,"15min":0.61799999999999999},"min":{"1min":0.501,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94199999999999995,"5min":1.089,"15min":1.089},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60099999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.96699999999999997,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 21:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.498,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93200000000000005,"5min":1.196,"15min":1.196},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89400000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97699999999999998,"5min":1.119,"15min":1.119},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.80000000000000004,"15min":0.83399999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90300000000000002,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:18:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":20729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":44289,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":21103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:19:04.346 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T21:19:04.347 INFO:tasks.cephadm:Setting up client nodes... 2023-10-06T21:19:04.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-06T21:19:05.114 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:04 smithi031 ceph-mon[98539]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:04 smithi031 ceph-mon[98539]: pgmap v305: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2023-10-06T21:19:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:04 smithi146 ceph-mon[102809]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:04 smithi146 ceph-mon[102809]: pgmap v305: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2023-10-06T21:19:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:04 smithi142 ceph-mon[102951]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:04 smithi142 ceph-mon[102951]: pgmap v305: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2023-10-06T21:19:06.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:05 smithi031 ceph-mon[98539]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:05 smithi146 ceph-mon[102809]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:05 smithi142 ceph-mon[102951]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 -- 172.21.15.31:0/1109365951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 msgr2=0x7fe34c10b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 --2- 172.21.15.31:0/1109365951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c10b630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe34400b0d0 tx=0x7fe34400b3e0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 -- 172.21.15.31:0/1109365951 shutdown_connections 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 --2- 172.21.15.31:0/1109365951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c10b630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 --2- 172.21.15.31:0/1109365951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1031e0 0x7fe34c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 --2- 172.21.15.31:0/1109365951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34c1043e0 0x7fe34c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 -- 172.21.15.31:0/1109365951 >> 172.21.15.31:0/1109365951 conn(0x7fe34c0fe780 msgr2=0x7fe34c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 -- 172.21.15.31:0/1109365951 shutdown_connections 2023-10-06T21:19:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.669+0000 7fe3545dd700 1 -- 172.21.15.31:0/1109365951 wait complete. 2023-10-06T21:19:06.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.670+0000 7fe3545dd700 1 Processor -- start 2023-10-06T21:19:06.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.670+0000 7fe3545dd700 1 -- start start 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34c1031e0 0x7fe34c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1043e0 0x7fe34c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c1a0ab0 con 0x7fe34c104d80 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c1a0bf0 con 0x7fe34c1043e0 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe3545dd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c1a0d80 con 0x7fe34c1031e0 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe352b7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe352b7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54718/0 (socket says 172.21.15.31:54718) 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe352b7a700 1 -- 172.21.15.31:0/3276836506 learned_addr learned my addr 172.21.15.31:0/3276836506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.671+0000 7fe351b78700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1043e0 0x7fe34c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 -- 172.21.15.31:0/3276836506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34c1031e0 msgr2=0x7fe34c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34c1031e0 0x7fe34c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 -- 172.21.15.31:0/3276836506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1043e0 msgr2=0x7fe34c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1043e0 0x7fe34c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 -- 172.21.15.31:0/3276836506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe344009d00 con 0x7fe34c104d80 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe352b7a700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe344009470 tx=0x7fe3440094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400e040 con 0x7fe34c104d80 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe344012e10 con 0x7fe34c104d80 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c1a1000 con 0x7fe34c104d80 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34401b450 con 0x7fe34c104d80 2023-10-06T21:19:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.672+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c1a14f0 con 0x7fe34c104d80 2023-10-06T21:19:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.674+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c10a730 con 0x7fe34c104d80 2023-10-06T21:19:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.676+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe344012980 con 0x7fe34c104d80 2023-10-06T21:19:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.677+0000 7fe3437fe700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 0x7fe3380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.677+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe3440cec50 con 0x7fe34c104d80 2023-10-06T21:19:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.677+0000 7fe352379700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 0x7fe3380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.678+0000 7fe352379700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 0x7fe3380a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe34c104240 tx=0x7fe348006d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.681+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe344098290 con 0x7fe34c104d80 2023-10-06T21:19:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.973+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe34c10e980 con 0x7fe34c104d80 2023-10-06T21:19:06.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.976+0000 7fe3437fe700 1 -- 172.21.15.31:0/3276836506 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe34409b8b0 con 0x7fe34c104d80 2023-10-06T21:19:06.979 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-10-06T21:19:06.979 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDKeSBlpQEiOhAAjQfNGS+xsPH3KO3cMvumOA== 2023-10-06T21:19:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 msgr2=0x7fe3380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 0x7fe3380a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe34c104240 tx=0x7fe348006d20 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 msgr2=0x7fe34c1a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe344009470 tx=0x7fe3440094a0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 shutdown_connections 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe3380a1510 0x7fe3380a39d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34c104d80 0x7fe34c1a04e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34c1031e0 0x7fe34c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 --2- 172.21.15.31:0/3276836506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c1043e0 0x7fe34c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.978+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 >> 172.21.15.31:0/3276836506 conn(0x7fe34c0fe780 msgr2=0x7fe34c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.979+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 shutdown_connections 2023-10-06T21:19:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:06.979+0000 7fe3545dd700 1 -- 172.21.15.31:0/3276836506 wait complete. 2023-10-06T21:19:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:06 smithi031 ceph-mon[98539]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-10-06T21:19:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:06 smithi146 ceph-mon[102809]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-10-06T21:19:07.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:06 smithi142 ceph-mon[102951]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-10-06T21:19:07.592 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:19:07.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-06T21:19:07.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T21:19:07.695 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T21:19:07.695 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-06T21:19:07.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-06T21:19:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:07 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:19:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:07 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:19:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:07 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:19:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:07 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:19:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:07 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:19:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:07 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3276836506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:19:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:09.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:08 smithi031 ceph-mon[98539]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-06T21:19:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:08 smithi146 ceph-mon[102809]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-06T21:19:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:08 smithi142 ceph-mon[102951]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-06T21:19:09.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 -- 172.21.15.31:0/2859594714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc001013a0 msgr2=0x7efc00101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 --2- 172.21.15.31:0/2859594714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc001013a0 0x7efc00101780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efbf00099c0 tx=0x7efbf0009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 -- 172.21.15.31:0/2859594714 shutdown_connections 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 --2- 172.21.15.31:0/2859594714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 0x7efc0010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 --2- 172.21.15.31:0/2859594714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc001013a0 0x7efc00101780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 --2- 172.21.15.31:0/2859594714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc00101d50 0x7efc00105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 -- 172.21.15.31:0/2859594714 >> 172.21.15.31:0/2859594714 conn(0x7efc00078f30 msgr2=0x7efc00079340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 -- 172.21.15.31:0/2859594714 shutdown_connections 2023-10-06T21:19:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.990+0000 7efc05c5e700 1 -- 172.21.15.31:0/2859594714 wait complete. 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.991+0000 7efc05c5e700 1 Processor -- start 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 -- start start 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc001013a0 0x7efc00199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 0x7efc0019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0019e1f0 con 0x7efc00101d50 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0019e360 con 0x7efc001013a0 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efc05c5e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0019e520 con 0x7efc00106320 2023-10-06T21:19:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efbfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efbfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54734/0 (socket says 172.21.15.31:54734) 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efbfeffd700 1 -- 172.21.15.31:0/1997045200 learned_addr learned my addr 172.21.15.31:0/1997045200 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efbff7fe700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc001013a0 0x7efc00199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.992+0000 7efbfffff700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 0x7efc0019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 -- 172.21.15.31:0/1997045200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 msgr2=0x7efc0019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 0x7efc0019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 -- 172.21.15.31:0/1997045200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc001013a0 msgr2=0x7efc00199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc001013a0 0x7efc00199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 -- 172.21.15.31:0/1997045200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00095f0 con 0x7efc00101d50 2023-10-06T21:19:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfeffd700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efbf400d950 tx=0x7efbf400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4004220 con 0x7efc00101d50 2023-10-06T21:19:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efbf4004cc0 con 0x7efc00101d50 2023-10-06T21:19:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0019e800 con 0x7efc00101d50 2023-10-06T21:19:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.993+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4005600 con 0x7efc00101d50 2023-10-06T21:19:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.994+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc001a6060 con 0x7efc00101d50 2023-10-06T21:19:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.995+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0004ebb0 con 0x7efc00101d50 2023-10-06T21:19:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.997+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7efbf4004380 con 0x7efc00101d50 2023-10-06T21:19:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.998+0000 7efbfcff9700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 0x7efbe80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.998+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efbf40ccb70 con 0x7efc00101d50 2023-10-06T21:19:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.998+0000 7efbff7fe700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 0x7efbe80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:09.999+0000 7efbff7fe700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 0x7efbe80a3a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006010 tx=0x7efbf00051c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.002+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efbf4096120 con 0x7efc00101d50 2023-10-06T21:19:10.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.272+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efc000008d0 con 0x7efc00101d50 2023-10-06T21:19:10.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.278+0000 7efbfcff9700 1 -- 172.21.15.31:0/1997045200 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260403 (secure 0 0 0) 0x7efbf4016090 con 0x7efc00101d50 2023-10-06T21:19:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.282+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 msgr2=0x7efbe80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.282+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 0x7efbe80a3a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006010 tx=0x7efbf00051c0 comp rx=0 tx=0).stop 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.282+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 msgr2=0x7efc00199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.282+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efbf400d950 tx=0x7efbf400dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 shutdown_connections 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7efbe80a1590 0x7efbe80a3a50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc00106320 0x7efc0019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc001013a0 0x7efc00199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:10.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 --2- 172.21.15.31:0/1997045200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc00101d50 0x7efc00199770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:10.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 >> 172.21.15.31:0/1997045200 conn(0x7efc00078f30 msgr2=0x7efc000ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:10.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 shutdown_connections 2023-10-06T21:19:10.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:10.283+0000 7efc05c5e700 1 -- 172.21.15.31:0/1997045200 wait complete. 2023-10-06T21:19:10.756 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":964869793},{"type":"v1","addr":"172.21.15.31:6801","nonce":964869793}]},"active_addr":"172.21.15.31:6801/964869793","active_change":"2023-10-06T21:11:25.716575+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1070417538}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":751435865}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":126831737}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1598815521}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2960018587}]}]} 2023-10-06T21:19:10.763 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-06T21:19:10.763 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-06T21:19:10.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:19:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:10 smithi031 ceph-mon[98539]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2023-10-06T21:19:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:10 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1997045200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:19:11.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:10 smithi146 ceph-mon[102809]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2023-10-06T21:19:11.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:10 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1997045200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:19:11.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:10 smithi142 ceph-mon[102951]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2023-10-06T21:19:11.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:10 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1997045200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 -- 172.21.15.31:0/896062976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 msgr2=0x7f05b0113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 --2- 172.21.15.31:0/896062976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b0113fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05a800b0a0 tx=0x7f05a800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 -- 172.21.15.31:0/896062976 shutdown_connections 2023-10-06T21:19:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 --2- 172.21.15.31:0/896062976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b0113fb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 --2- 172.21.15.31:0/896062976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0074ef0 0x7f05b00730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 --2- 172.21.15.31:0/896062976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0073620 0x7f05b0073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 -- 172.21.15.31:0/896062976 >> 172.21.15.31:0/896062976 conn(0x7f05b00780a0 msgr2=0x7f05b00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 -- 172.21.15.31:0/896062976 shutdown_connections 2023-10-06T21:19:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.946+0000 7f05b77e7700 1 -- 172.21.15.31:0/896062976 wait complete. 2023-10-06T21:19:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.947+0000 7f05b77e7700 1 Processor -- start 2023-10-06T21:19:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.947+0000 7f05b77e7700 1 -- start start 2023-10-06T21:19:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0073620 0x7f05b019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0074ef0 0x7f05b019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:12.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a1590 con 0x7f05b0111bc0 2023-10-06T21:19:12.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a1700 con 0x7f05b0073620 2023-10-06T21:19:12.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b77e7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a18c0 con 0x7f05b0074ef0 2023-10-06T21:19:12.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b5d84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:12.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b5d84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54762/0 (socket says 172.21.15.31:54762) 2023-10-06T21:19:12.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b5d84700 1 -- 172.21.15.31:0/1904973535 learned_addr learned my addr 172.21.15.31:0/1904973535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:12.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.948+0000 7f05b5583700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0073620 0x7f05b019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:12.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b4d82700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0074ef0 0x7f05b019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 -- 172.21.15.31:0/1904973535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0074ef0 msgr2=0x7f05b019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0074ef0 0x7f05b019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 -- 172.21.15.31:0/1904973535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0073620 msgr2=0x7f05b019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0073620 0x7f05b019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 -- 172.21.15.31:0/1904973535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009d00 con 0x7f05b0111bc0 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b5d84700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009fd0 tx=0x7f05a8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800e040 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b01a1b40 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05a801d910 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80055e0 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.949+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b01aa4f0 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.951+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b004ebb0 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.952+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f05a801da80 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.954+0000 7f05a67fc700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 0x7f059c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.954+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05a80ce480 con 0x7f05b0111bc0 2023-10-06T21:19:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.954+0000 7f05b5583700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 0x7f059c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:12.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.954+0000 7f05b5583700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 0x7f059c0a39d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f05ac007b60 tx=0x7f05ac0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:12.958+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f05a8097ac0 con 0x7f05b0111bc0 2023-10-06T21:19:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.172+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05b019e9f0 con 0x7f05b0111bc0 2023-10-06T21:19:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.173+0000 7f05a67fc700 1 -- 172.21.15.31:0/1904973535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22117 (secure 0 0 0) 0x7f05a801dd80 con 0x7f05b0111bc0 2023-10-06T21:19:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:13.176 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:18:55.827472+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 msgr2=0x7f059c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 0x7f059c0a39d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f05ac007b60 tx=0x7f05ac0073d0 comp rx=0 tx=0).stop 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 msgr2=0x7f05b01a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009fd0 tx=0x7f05a8009440 comp rx=0 tx=0).stop 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 shutdown_connections 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f059c0a1510 0x7f059c0a39d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b0111bc0 0x7f05b01a0f90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0073620 0x7f05b019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 --2- 172.21.15.31:0/1904973535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05b0074ef0 0x7f05b019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 >> 172.21.15.31:0/1904973535 conn(0x7f05b00780a0 msgr2=0x7f05b01106c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 shutdown_connections 2023-10-06T21:19:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:13.176+0000 7f05b77e7700 1 -- 172.21.15.31:0/1904973535 wait complete. 2023-10-06T21:19:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:12 smithi031 ceph-mon[98539]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T21:19:13.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:12 smithi146 ceph-mon[102809]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T21:19:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:12 smithi142 ceph-mon[102951]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T21:19:13.687 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-06T21:19:13.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:19:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:13 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1904973535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:13 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1904973535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:13 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1904973535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:14 smithi031 ceph-mon[98539]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2023-10-06T21:19:15.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:14 smithi146 ceph-mon[102809]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2023-10-06T21:19:15.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:14 smithi142 ceph-mon[102951]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2023-10-06T21:19:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 -- 172.21.15.31:0/3436757620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 msgr2=0x7f3c7c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/3436757620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c101400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c680099c0 tx=0x7f3c68009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 -- 172.21.15.31:0/3436757620 shutdown_connections 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/3436757620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/3436757620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/3436757620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c101400 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 -- 172.21.15.31:0/3436757620 >> 172.21.15.31:0/3436757620 conn(0x7f3c7c0780a0 msgr2=0x7f3c7c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 -- 172.21.15.31:0/3436757620 shutdown_connections 2023-10-06T21:19:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.919+0000 7f3c82ab5700 1 -- 172.21.15.31:0/3436757620 wait complete. 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.920+0000 7f3c82ab5700 1 Processor -- start 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.920+0000 7f3c82ab5700 1 -- start start 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19bed0 con 0x7f3c7c100f80 2023-10-06T21:19:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19c040 con 0x7f3c7c106f90 2023-10-06T21:19:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c82ab5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19c200 con 0x7f3c7c101940 2023-10-06T21:19:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c80851700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c80851700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52822/0 (socket says 172.21.15.31:52822) 2023-10-06T21:19:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.921+0000 7f3c80851700 1 -- 172.21.15.31:0/1501245060 learned_addr learned my addr 172.21.15.31:0/1501245060 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:15.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 -- 172.21.15.31:0/1501245060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 msgr2=0x7f3c7c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:15.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c7bfff700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:15.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c81052700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:15.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 -- 172.21.15.31:0/1501245060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 msgr2=0x7f3c7c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 -- 172.21.15.31:0/1501245060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680095f0 con 0x7f3c7c100f80 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c7bfff700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c81052700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c80851700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000d950 tx=0x7f3c7000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70009b40 con 0x7f3c7c100f80 2023-10-06T21:19:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c70009ca0 con 0x7f3c7c100f80 2023-10-06T21:19:15.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c19c4e0 con 0x7f3c7c100f80 2023-10-06T21:19:15.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000f620 con 0x7f3c7c100f80 2023-10-06T21:19:15.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.922+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c1a3d40 con 0x7f3c7c100f80 2023-10-06T21:19:15.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.923+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c04ebb0 con 0x7f3c7c100f80 2023-10-06T21:19:15.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.925+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3c70010460 con 0x7f3c7c100f80 2023-10-06T21:19:15.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.927+0000 7f3c79ffb700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 0x7f3c6c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:15.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.927+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c700cd0e0 con 0x7f3c7c100f80 2023-10-06T21:19:15.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.927+0000 7f3c7bfff700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 0x7f3c6c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:15.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.928+0000 7f3c7bfff700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 0x7f3c6c0a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006010 tx=0x7f3c68000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:15.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:15.931+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3c70096690 con 0x7f3c7c100f80 2023-10-06T21:19:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.143+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c7c110e40 con 0x7f3c7c100f80 2023-10-06T21:19:16.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.144+0000 7f3c79ffb700 1 -- 172.21.15.31:0/1501245060 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22117 (secure 0 0 0) 0x7f3c70099cb0 con 0x7f3c7c100f80 2023-10-06T21:19:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:16.147 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:18:55.827472+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.146+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 msgr2=0x7f3c6c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.146+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 0x7f3c6c0a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006010 tx=0x7f3c68000bc0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 msgr2=0x7f3c7c196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000d950 tx=0x7f3c7000dd10 comp rx=0 tx=0).stop 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 shutdown_connections 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c6c0a1510 0x7f3c6c0a39d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c106f90 0x7f3c7c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c7c100f80 0x7f3c7c196f10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 --2- 172.21.15.31:0/1501245060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c7c101940 0x7f3c7c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 >> 172.21.15.31:0/1501245060 conn(0x7f3c7c0780a0 msgr2=0x7f3c7c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:16.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 shutdown_connections 2023-10-06T21:19:16.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:16.147+0000 7f3c82ab5700 1 -- 172.21.15.31:0/1501245060 wait complete. 2023-10-06T21:19:16.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-06T21:19:16.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-06T21:19:16.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-06T21:19:16.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-06T21:19:16.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-06T21:19:16.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-06T21:19:16.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-06T21:19:16.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-06T21:19:16.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-06T21:19:16.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-06T21:19:16.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-06T21:19:16.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-06T21:19:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:17 smithi031 ceph-mon[98539]: pgmap v311: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2023-10-06T21:19:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:17 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1501245060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:17 smithi146 ceph-mon[102809]: pgmap v311: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2023-10-06T21:19:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:17 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1501245060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:17.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:17 smithi142 ceph-mon[102951]: pgmap v311: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2023-10-06T21:19:17.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:17 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1501245060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:18.389 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:18.489 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:18.838 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:19 smithi146 ceph-mon[102809]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-10-06T21:19:19.303 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:19 smithi031 ceph-mon[98539]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-10-06T21:19:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:19 smithi142 ceph-mon[102951]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-10-06T21:19:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:21 smithi031 ceph-mon[98539]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-10-06T21:19:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:21 smithi146 ceph-mon[102809]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-10-06T21:19:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:21 smithi142 ceph-mon[102951]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-10-06T21:19:21.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.248+0000 7f905836e700 1 -- 172.21.15.31:0/4248439768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905010d930 msgr2=0x7f905010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.248+0000 7f905836e700 1 --2- 172.21.15.31:0/4248439768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905010d930 0x7f905010fd20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f904800b0a0 tx=0x7f904800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 -- 172.21.15.31:0/4248439768 shutdown_connections 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 --2- 172.21.15.31:0/4248439768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f905010d930 0x7f905010fd20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 --2- 172.21.15.31:0/4248439768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90500730e0 0x7f90500734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 --2- 172.21.15.31:0/4248439768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9050073a00 0x7f905010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 -- 172.21.15.31:0/4248439768 >> 172.21.15.31:0/4248439768 conn(0x7f90500780a0 msgr2=0x7f90500784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 -- 172.21.15.31:0/4248439768 shutdown_connections 2023-10-06T21:19:21.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.249+0000 7f905836e700 1 -- 172.21.15.31:0/4248439768 wait complete. 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 Processor -- start 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 -- start start 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90500730e0 0x7f9050199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 0x7f9050199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f905010d930 0x7f905019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e2d0 con 0x7f9050073a00 2023-10-06T21:19:21.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e440 con 0x7f90500730e0 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f905836e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e600 con 0x7f905010d930 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f9055909700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 0x7f9050199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.250+0000 7f9055909700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 0x7f9050199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52850/0 (socket says 172.21.15.31:52850) 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 -- 172.21.15.31:0/1675008186 learned_addr learned my addr 172.21.15.31:0/1675008186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f905690b700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f905010d930 0x7f905019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f905610a700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90500730e0 0x7f9050199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:21.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 -- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f905010d930 msgr2=0x7f905019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f905010d930 0x7f905019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 -- 172.21.15.31:0/1675008186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90500730e0 msgr2=0x7f9050199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90500730e0 0x7f9050199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048009d00 con 0x7f9050073a00 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f905610a700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90500730e0 0x7f9050199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f9055909700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 0x7f9050199850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f904000d950 tx=0x7f904000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:21.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040009940 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9040010460 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905019e8e0 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.251+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000f620 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.252+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90501a6140 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.252+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9050113190 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.255+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f90400105d0 con 0x7f9050073a00 2023-10-06T21:19:21.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.256+0000 7f90477fe700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f903c0a1590 0x7f903c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:21.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f90400ccfe0 con 0x7f9050073a00 2023-10-06T21:19:21.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f905610a700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f903c0a1590 0x7f903c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:21.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f90477fe700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] conn(0x7f903c0aad70 0x7f903c0ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:21.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f903c0ad840 con 0x7f903c0aad70 2023-10-06T21:19:21.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90400cd3f0 con 0x7f9050073a00 2023-10-06T21:19:21.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.257+0000 7f905610a700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f903c0a1590 0x7f903c0a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f904c005fd0 tx=0x7f904c005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.258+0000 7f905690b700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] conn(0x7f903c0aad70 0x7f903c0ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.258+0000 7f905690b700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] conn(0x7f903c0aad70 0x7f903c0ad190 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.259+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== osd.5 v2:172.21.15.146:6808/4108707603 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f903c0ad840 con 0x7f903c0aad70 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.276+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 --> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f905004ebb0 con 0x7f903c0aad70 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.277+0000 7f90477fe700 1 -- 172.21.15.31:0/1675008186 <== osd.5 v2:172.21.15.146:6808/4108707603 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f905004ebb0 con 0x7f903c0aad70 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.277+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] conn(0x7f903c0aad70 msgr2=0x7f903c0ad190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.277+0000 7f905836e700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] conn(0x7f903c0aad70 0x7f903c0ad190 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f903c0a1590 msgr2=0x7f903c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f903c0a1590 0x7f903c0a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f904c005fd0 tx=0x7f904c005ee0 comp rx=0 tx=0).stop 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 msgr2=0x7f9050199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 --2- 172.21.15.31:0/1675008186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9050073a00 0x7f9050199850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f904000d950 tx=0x7f904000dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905690b700 1 -- 172.21.15.31:0/1675008186 reap_dead start 2023-10-06T21:19:21.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 shutdown_connections 2023-10-06T21:19:21.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 >> 172.21.15.31:0/1675008186 conn(0x7f90500780a0 msgr2=0x7f90500fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:21.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 shutdown_connections 2023-10-06T21:19:21.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:21.278+0000 7f905836e700 1 -- 172.21.15.31:0/1675008186 wait complete. 2023-10-06T21:19:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:22.817 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:23.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:23 smithi146 ceph-mon[102809]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-10-06T21:19:23.299 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:23 smithi031 ceph-mon[98539]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-10-06T21:19:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:23 smithi142 ceph-mon[102951]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-10-06T21:19:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 -- 172.21.15.31:0/2444539461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 msgr2=0x7fbabc101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2444539461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc101380 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009990 tx=0x7fbab0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 -- 172.21.15.31:0/2444539461 shutdown_connections 2023-10-06T21:19:23.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2444539461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc106080 0x7fbabc10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2444539461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc101380 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2444539461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbabc101950 0x7fbabc1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.265+0000 7fbac1e18700 1 -- 172.21.15.31:0/2444539461 >> 172.21.15.31:0/2444539461 conn(0x7fbabc0fc8b0 msgr2=0x7fbabc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.266+0000 7fbac1e18700 1 -- 172.21.15.31:0/2444539461 shutdown_connections 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.266+0000 7fbac1e18700 1 -- 172.21.15.31:0/2444539461 wait complete. 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 Processor -- start 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 -- start start 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:23.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbabc101950 0x7fbabc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc106080 0x7fbabc073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a5200 con 0x7fbabc100fa0 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a5370 con 0x7fbabc106080 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac1e18700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a5530 con 0x7fbabc101950 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.267+0000 7fbac0e16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52278/0 (socket says 172.21.15.31:52278) 2023-10-06T21:19:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 -- 172.21.15.31:0/2827095035 learned_addr learned my addr 172.21.15.31:0/2827095035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbabbfff700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbabc101950 0x7fbabc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac1617700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc106080 0x7fbabc073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 -- 172.21.15.31:0/2827095035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbabc101950 msgr2=0x7fbabc0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbabc101950 0x7fbabc0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 -- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc106080 msgr2=0x7fbabc073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbabc106080 0x7fbabc073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00095f0 con 0x7fbabc100fa0 2023-10-06T21:19:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.268+0000 7fbac0e16700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc0731b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbab00036c0 tx=0x7fbab0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.270+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab001d070 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.270+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbab0003ec0 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.270+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0022b00 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.270+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc1a57b0 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.270+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc1a5b70 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.271+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbaa0000ff0 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.275+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbab0004030 con 0x7fbabc100fa0 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.278+0000 7fbab9ffb700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbaac0a1660 0x7fbaac0a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:23.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.278+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbab00ceec0 con 0x7fbabc100fa0 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.278+0000 7fbabbfff700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbaac0a1660 0x7fbaac0a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.279+0000 7fbab9ffb700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] conn(0x7fbaac0aae40 0x7fbaac0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.279+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbaac0ad910 con 0x7fbaac0aae40 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.279+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbab00cf230 con 0x7fbabc100fa0 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.279+0000 7fbac1617700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] conn(0x7fbaac0aae40 0x7fbaac0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.279+0000 7fbabbfff700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbaac0a1660 0x7fbaac0a3b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0749a0 tx=0x7fbaa80073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:23.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.280+0000 7fbac1617700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] conn(0x7fbaac0aae40 0x7fbaac0ad260 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.281+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== osd.1 v2:172.21.15.142:6800/2309964911 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbaac0ad910 con 0x7fbaac0aae40 2023-10-06T21:19:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.315+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 --> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbaa0002d70 con 0x7fbaac0aae40 2023-10-06T21:19:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.316+0000 7fbab9ffb700 1 -- 172.21.15.31:0/2827095035 <== osd.1 v2:172.21.15.142:6800/2309964911 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbaa0002d70 con 0x7fbaac0aae40 2023-10-06T21:19:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.316+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] conn(0x7fbaac0aae40 msgr2=0x7fbaac0ad260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.316+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] conn(0x7fbaac0aae40 0x7fbaac0ad260 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbaac0a1660 msgr2=0x7fbaac0a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbaac0a1660 0x7fbaac0a3b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0749a0 tx=0x7fbaa80073d0 comp rx=0 tx=0).stop 2023-10-06T21:19:23.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 msgr2=0x7fbabc0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 --2- 172.21.15.31:0/2827095035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbabc100fa0 0x7fbabc0731b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbab00036c0 tx=0x7fbab0005740 comp rx=0 tx=0).stop 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1617700 1 -- 172.21.15.31:0/2827095035 reap_dead start 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 shutdown_connections 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 >> 172.21.15.31:0/2827095035 conn(0x7fbabc0fc8b0 msgr2=0x7fbabc0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 shutdown_connections 2023-10-06T21:19:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:23.317+0000 7fbac1e18700 1 -- 172.21.15.31:0/2827095035 wait complete. 2023-10-06T21:19:23.749 INFO:teuthology.orchestra.run.smithi031.stdout:167503724595 2023-10-06T21:19:23.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-06T21:19:25.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:24 smithi031 ceph-mon[98539]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-06T21:19:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:24 smithi142 ceph-mon[102951]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-06T21:19:25.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:24 smithi146 ceph-mon[102809]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-06T21:19:25.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-06T21:19:25.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-06T21:19:25.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-10-06T21:19:25.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-10-06T21:19:25.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2023-10-06T21:19:26.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-10-06T21:19:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.160+0000 7f4c5be53700 1 -- 172.21.15.31:0/2558233058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540689a0 msgr2=0x7f4c54068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.160+0000 7f4c5be53700 1 --2- 172.21.15.31:0/2558233058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540689a0 0x7f4c54068d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009990 tx=0x7f4c50009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 -- 172.21.15.31:0/2558233058 shutdown_connections 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 --2- 172.21.15.31:0/2558233058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c5410da10 0x7f4c5410fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 --2- 172.21.15.31:0/2558233058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540689a0 0x7f4c54068d80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 --2- 172.21.15.31:0/2558233058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c540692c0 0x7f4c5410d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 -- 172.21.15.31:0/2558233058 >> 172.21.15.31:0/2558233058 conn(0x7f4c54075e00 msgr2=0x7f4c54076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.161+0000 7f4c5be53700 1 -- 172.21.15.31:0/2558233058 shutdown_connections 2023-10-06T21:19:26.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.162+0000 7f4c5be53700 1 -- 172.21.15.31:0/2558233058 wait complete. 2023-10-06T21:19:26.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.163+0000 7f4c5be53700 1 Processor -- start 2023-10-06T21:19:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.163+0000 7f4c5be53700 1 -- start start 2023-10-06T21:19:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.163+0000 7f4c5be53700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c540689a0 0x7f4c54199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.163+0000 7f4c5be53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 0x7f4c54199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.163+0000 7f4c5be53700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c5410da10 0x7f4c5419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c5be53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5419e0f0 con 0x7f4c540692c0 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c5be53700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5419e260 con 0x7f4c540689a0 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c5be53700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5419e420 con 0x7f4c5410da10 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 0x7f4c54199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 0x7f4c54199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52304/0 (socket says 172.21.15.31:52304) 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c59bef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c540689a0 0x7f4c54199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 -- 172.21.15.31:0/4242648391 learned_addr learned my addr 172.21.15.31:0/4242648391 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c5a3f0700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c5410da10 0x7f4c5419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 -- 172.21.15.31:0/4242648391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c5410da10 msgr2=0x7f4c5419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c5410da10 0x7f4c5419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 -- 172.21.15.31:0/4242648391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c540689a0 msgr2=0x7f4c54199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c540689a0 0x7f4c54199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c593ee700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500095f0 con 0x7f4c540692c0 2023-10-06T21:19:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.164+0000 7f4c59bef700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c540689a0 0x7f4c54199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.165+0000 7f4c593ee700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 0x7f4c54199670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400d950 tx=0x7f4c4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.166+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44009940 con 0x7f4c540692c0 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.166+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c44010460 con 0x7f4c540692c0 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.166+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400f620 con 0x7f4c540692c0 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.166+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5419e700 con 0x7f4c540692c0 2023-10-06T21:19:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.166+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c541a5f60 con 0x7f4c540692c0 2023-10-06T21:19:26.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.167+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c54113260 con 0x7f4c540692c0 2023-10-06T21:19:26.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.171+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4c440105d0 con 0x7f4c540692c0 2023-10-06T21:19:26.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.172+0000 7f4c4affd700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1590 0x7f4c400a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.172+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c440cd1e0 con 0x7f4c540692c0 2023-10-06T21:19:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.172+0000 7f4c59bef700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1590 0x7f4c400a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.173+0000 7f4c4affd700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] conn(0x7f4c400ab1d0 0x7f4c400ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.173+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c400add40 con 0x7f4c400ab1d0 2023-10-06T21:19:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.173+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c440cd550 con 0x7f4c540692c0 2023-10-06T21:19:26.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.173+0000 7f4c5a3f0700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] conn(0x7f4c400ab1d0 0x7f4c400ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.174+0000 7f4c59bef700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1590 0x7f4c400a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006010 tx=0x7f4c5000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.174+0000 7f4c5a3f0700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] conn(0x7f4c400ab1d0 0x7f4c400ad670 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.174+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== osd.0 v2:172.21.15.31:6802/811549482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4c400add40 con 0x7f4c400ab1d0 2023-10-06T21:19:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.194+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 --> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c5404ebb0 con 0x7f4c400ab1d0 2023-10-06T21:19:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c4affd700 1 -- 172.21.15.31:0/4242648391 <== osd.0 v2:172.21.15.31:6802/811549482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c5404ebb0 con 0x7f4c400ab1d0 2023-10-06T21:19:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] conn(0x7f4c400ab1d0 msgr2=0x7f4c400ad670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] conn(0x7f4c400ab1d0 0x7f4c400ad670 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1590 msgr2=0x7f4c400a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1590 0x7f4c400a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006010 tx=0x7f4c5000b260 comp rx=0 tx=0).stop 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 msgr2=0x7f4c54199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5be53700 1 --2- 172.21.15.31:0/4242648391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c540692c0 0x7f4c54199670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400d950 tx=0x7f4c4400dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.195+0000 7f4c5a3f0700 1 -- 172.21.15.31:0/4242648391 reap_dead start 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.196+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 shutdown_connections 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.196+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 >> 172.21.15.31:0/4242648391 conn(0x7f4c54075e00 msgr2=0x7f4c5410ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.196+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 shutdown_connections 2023-10-06T21:19:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.196+0000 7f4c5be53700 1 -- 172.21.15.31:0/4242648391 wait complete. 2023-10-06T21:19:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 -- 172.21.15.31:0/958800464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 msgr2=0x7f626410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 --2- 172.21.15.31:0/958800464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f626410aec0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6250009990 tx=0x7f6250009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 -- 172.21.15.31:0/958800464 shutdown_connections 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 --2- 172.21.15.31:0/958800464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626410b4e0 0x7f626410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 --2- 172.21.15.31:0/958800464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62641020b0 0x7f6264102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 --2- 172.21.15.31:0/958800464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f626410aec0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.613+0000 7f626b0a6700 1 -- 172.21.15.31:0/958800464 >> 172.21.15.31:0/958800464 conn(0x7f62640fb920 msgr2=0x7f62640fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.614+0000 7f626b0a6700 1 -- 172.21.15.31:0/958800464 shutdown_connections 2023-10-06T21:19:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.614+0000 7f626b0a6700 1 -- 172.21.15.31:0/958800464 wait complete. 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 Processor -- start 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 -- start start 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62641020b0 0x7f62640ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f6264100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626410b4e0 0x7f62641007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a5440 con 0x7f62641029d0 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a55b0 con 0x7f626410b4e0 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f626b0a6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a5770 con 0x7f62641020b0 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f6263fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f6264100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.615+0000 7f6263fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f6264100250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52292/0 (socket says 172.21.15.31:52292) 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 -- 172.21.15.31:0/4220694730 learned_addr learned my addr 172.21.15.31:0/4220694730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6269643700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626410b4e0 0x7f62641007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6268e42700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62641020b0 0x7f62640ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 -- 172.21.15.31:0/4220694730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62641020b0 msgr2=0x7f62640ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62641020b0 0x7f62640ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 -- 172.21.15.31:0/4220694730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626410b4e0 msgr2=0x7f62641007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626410b4e0 0x7f62641007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500095f0 con 0x7f62641029d0 2023-10-06T21:19:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6263fff700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f6264100250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6250005910 tx=0x7f62500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625001d070 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6250022470 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62641a59f0 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.616+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250017650 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.617+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641a5e90 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.618+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6264111000 con 0x7f62641029d0 2023-10-06T21:19:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.620+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f62500225e0 con 0x7f62641029d0 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.621+0000 7f6261ffb700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f62540a1510 0x7f62540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.621+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f62500ced20 con 0x7f62641029d0 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.621+0000 7f6268e42700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f62540a1510 0x7f62540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6261ffb700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] conn(0x7f62540aacf0 0x7f62540ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62540ad7c0 con 0x7f62540aacf0 2023-10-06T21:19:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62500cf090 con 0x7f62641029d0 2023-10-06T21:19:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6269643700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] conn(0x7f62540aacf0 0x7f62540ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6268e42700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f62540a1510 0x7f62540a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6258009bf0 tx=0x7f6258009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.622+0000 7f6269643700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] conn(0x7f62540aacf0 0x7f62540ad110 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.623+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== osd.6 v2:172.21.15.31:6818/1906076219 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f62540ad7c0 con 0x7f62540aacf0 2023-10-06T21:19:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.641+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 --> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f626404ebb0 con 0x7f62540aacf0 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.641+0000 7f6261ffb700 1 -- 172.21.15.31:0/4220694730 <== osd.6 v2:172.21.15.31:6818/1906076219 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f626404ebb0 con 0x7f62540aacf0 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.641+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] conn(0x7f62540aacf0 msgr2=0x7f62540ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.641+0000 7f626b0a6700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] conn(0x7f62540aacf0 0x7f62540ad110 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f62540a1510 msgr2=0x7f62540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f62540a1510 0x7f62540a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6258009bf0 tx=0x7f6258009260 comp rx=0 tx=0).stop 2023-10-06T21:19:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 msgr2=0x7f6264100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 --2- 172.21.15.31:0/4220694730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62641029d0 0x7f6264100250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6250005910 tx=0x7f62500057f0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f6269643700 1 -- 172.21.15.31:0/4220694730 reap_dead start 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 shutdown_connections 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 >> 172.21.15.31:0/4220694730 conn(0x7f62640fb920 msgr2=0x7f62640fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 shutdown_connections 2023-10-06T21:19:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:25.642+0000 7f626b0a6700 1 -- 172.21.15.31:0/4220694730 wait complete. 2023-10-06T21:19:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-06T21:19:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-06T21:19:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-10-06T21:19:26.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-10-06T21:19:26.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2023-10-06T21:19:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 -- 172.21.15.31:0/1664839839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 msgr2=0x7fd1e01059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 --2- 172.21.15.31:0/1664839839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01059a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009990 tx=0x7fd1d0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 -- 172.21.15.31:0/1664839839 shutdown_connections 2023-10-06T21:19:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 --2- 172.21.15.31:0/1664839839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e0106080 0x7fd1e010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 --2- 172.21.15.31:0/1664839839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 0x7fd1e0101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 --2- 172.21.15.31:0/1664839839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01059a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 -- 172.21.15.31:0/1664839839 >> 172.21.15.31:0/1664839839 conn(0x7fd1e00fc8b0 msgr2=0x7fd1e00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 -- 172.21.15.31:0/1664839839 shutdown_connections 2023-10-06T21:19:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.558+0000 7fd1e7410700 1 -- 172.21.15.31:0/1664839839 wait complete. 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 Processor -- start 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 -- start start 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 0x7fd1e0196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e0106080 0x7fd1e019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019be60 con 0x7fd1e0101950 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019bfd0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.559+0000 7fd1e7410700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019c190 con 0x7fd1e0106080 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e5c0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e5c0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52324/0 (socket says 172.21.15.31:52324) 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e5c0d700 1 -- 172.21.15.31:0/943843985 learned_addr learned my addr 172.21.15.31:0/943843985 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 0x7fd1e0196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e6c0f700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e0106080 0x7fd1e019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 -- 172.21.15.31:0/943843985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e0106080 msgr2=0x7fd1e019b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e0106080 0x7fd1e019b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 -- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 msgr2=0x7fd1e01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e640e700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00095f0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.560+0000 7fd1e5c0d700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101950 0x7fd1e01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:26.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1e640e700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 0x7fd1e0196ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d950 tx=0x7fd1dc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00f890 con 0x7fd1e0100fa0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e019c470 con 0x7fd1e0100fa0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1dc00fe80 con 0x7fd1e0100fa0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00e3b0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.561+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e01a3cd0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.562+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1e0110e90 con 0x7fd1e0100fa0 2023-10-06T21:19:26.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.566+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd1dc00f9f0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.567+0000 7fd1d77fe700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1cc0a1510 0x7fd1cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd1dc0cd0c0 con 0x7fd1e0100fa0 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1e5c0d700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1cc0a1510 0x7fd1cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1d77fe700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] conn(0x7fd1cc0ab150 0x7fd1cc0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1cc0adcc0 con 0x7fd1cc0ab150 2023-10-06T21:19:26.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1dc0cd430 con 0x7fd1e0100fa0 2023-10-06T21:19:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.568+0000 7fd1e5c0d700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1cc0a1510 0x7fd1cc0a39d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1e01984c0 tx=0x7fd1d0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.569+0000 7fd1e6c0f700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] conn(0x7fd1cc0ab150 0x7fd1cc0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.569+0000 7fd1e6c0f700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] conn(0x7fd1cc0ab150 0x7fd1cc0ad5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:26.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.570+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== osd.7 v2:172.21.15.142:6816/722083176 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd1cc0adcc0 con 0x7fd1cc0ab150 2023-10-06T21:19:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.588+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 --> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1e004f400 con 0x7fd1cc0ab150 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.588+0000 7fd1d77fe700 1 -- 172.21.15.31:0/943843985 <== osd.7 v2:172.21.15.142:6816/722083176 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd1e004f400 con 0x7fd1cc0ab150 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] conn(0x7fd1cc0ab150 msgr2=0x7fd1cc0ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] conn(0x7fd1cc0ab150 0x7fd1cc0ad5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1cc0a1510 msgr2=0x7fd1cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1cc0a1510 0x7fd1cc0a39d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1e01984c0 tx=0x7fd1d0005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 msgr2=0x7fd1e0196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 --2- 172.21.15.31:0/943843985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0100fa0 0x7fd1e0196ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d950 tx=0x7fd1dc00dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e6c0f700 1 -- 172.21.15.31:0/943843985 reap_dead start 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 shutdown_connections 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 >> 172.21.15.31:0/943843985 conn(0x7fd1e00fc8b0 msgr2=0x7fd1e00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 shutdown_connections 2023-10-06T21:19:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:26.589+0000 7fd1e7410700 1 -- 172.21.15.31:0/943843985 wait complete. 2023-10-06T21:19:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-06T21:19:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-06T21:19:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-06T21:19:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-10-06T21:19:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2023-10-06T21:19:27.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:26 smithi031 ceph-mon[98539]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:19:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-10-06T21:19:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2023-10-06T21:19:27.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:26 smithi142 ceph-mon[102951]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:19:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-06T21:19:27.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-10-06T21:19:27.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-10-06T21:19:27.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2023-10-06T21:19:27.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:26 smithi146 ceph-mon[102809]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:19:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:34359738446 2023-10-06T21:19:27.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-06T21:19:27.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:27 smithi031 ceph-mon[98539]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:19:28.009 INFO:teuthology.orchestra.run.smithi031.stdout:55834574920 2023-10-06T21:19:28.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-06T21:19:28.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:27 smithi142 ceph-mon[102951]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:19:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:27 smithi146 ceph-mon[102809]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 -- 172.21.15.31:0/1283578489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c40068c00 msgr2=0x7f1c40068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 --2- 172.21.15.31:0/1283578489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c40068c00 0x7f1c40068fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009990 tx=0x7f1c30009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 -- 172.21.15.31:0/1283578489 shutdown_connections 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 --2- 172.21.15.31:0/1283578489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c401062b0 0x7f1c4010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 --2- 172.21.15.31:0/1283578489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c40068c00 0x7f1c40068fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 --2- 172.21.15.31:0/1283578489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c400695b0 0x7f1c40105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 -- 172.21.15.31:0/1283578489 >> 172.21.15.31:0/1283578489 conn(0x7f1c40075e00 msgr2=0x7f1c40076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 -- 172.21.15.31:0/1283578489 shutdown_connections 2023-10-06T21:19:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.374+0000 7f1c4683c700 1 -- 172.21.15.31:0/1283578489 wait complete. 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.375+0000 7f1c4683c700 1 Processor -- start 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 -- start start 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c40068c00 0x7f1c40196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 0x7f1c40197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c401062b0 0x7f1c4019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019bf90 con 0x7f1c400695b0 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019c100 con 0x7f1c401062b0 2023-10-06T21:19:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c4683c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019c2c0 con 0x7f1c40068c00 2023-10-06T21:19:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c3f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 0x7f1c40197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c3f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 0x7f1c40197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52342/0 (socket says 172.21.15.31:52342) 2023-10-06T21:19:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.376+0000 7f1c3f7fe700 1 -- 172.21.15.31:0/2073250096 learned_addr learned my addr 172.21.15.31:0/2073250096 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c44dd9700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c401062b0 0x7f1c4019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3ffff700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c40068c00 0x7f1c40196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 -- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c40068c00 msgr2=0x7f1c40196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c40068c00 0x7f1c40196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 -- 172.21.15.31:0/2073250096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c401062b0 msgr2=0x7f1c4019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c401062b0 0x7f1c4019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300095f0 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3f7fe700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 0x7f1c40197510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400d950 tx=0x7f1c3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.377+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f890 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.378+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c3400fe80 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.378+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4019c5a0 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.378+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400e3b0 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.378+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c401a3e00 con 0x7f1c400695b0 2023-10-06T21:19:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.379+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c4004ebb0 con 0x7f1c400695b0 2023-10-06T21:19:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.381+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1c3400f9f0 con 0x7f1c400695b0 2023-10-06T21:19:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3d7fa700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1c280a1510 0x7f1c280a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1c340ccdc0 con 0x7f1c400695b0 2023-10-06T21:19:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3ffff700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1c280a1510 0x7f1c280a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3d7fa700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] conn(0x7f1c280ab150 0x7f1c280ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c280adcc0 con 0x7f1c280ab150 2023-10-06T21:19:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.383+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c340cd130 con 0x7f1c400695b0 2023-10-06T21:19:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.384+0000 7f1c44dd9700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] conn(0x7f1c280ab150 0x7f1c280ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.384+0000 7f1c3ffff700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1c280a1510 0x7f1c280a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b5c0 tx=0x7f1c30005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.384+0000 7f1c44dd9700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] conn(0x7f1c280ab150 0x7f1c280ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.385+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== osd.2 v2:172.21.15.146:6800/1261175101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1c280adcc0 con 0x7f1c280ab150 2023-10-06T21:19:28.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.404+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 --> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c40111070 con 0x7f1c280ab150 2023-10-06T21:19:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c3d7fa700 1 -- 172.21.15.31:0/2073250096 <== osd.2 v2:172.21.15.146:6800/1261175101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c38010040 con 0x7f1c280ab150 2023-10-06T21:19:28.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] conn(0x7f1c280ab150 msgr2=0x7f1c280ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] conn(0x7f1c280ab150 0x7f1c280ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1c280a1510 msgr2=0x7f1c280a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f1c280a1510 0x7f1c280a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b5c0 tx=0x7f1c30005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 msgr2=0x7f1c40197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 --2- 172.21.15.31:0/2073250096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c400695b0 0x7f1c40197510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400d950 tx=0x7f1c3400dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c44dd9700 1 -- 172.21.15.31:0/2073250096 reap_dead start 2023-10-06T21:19:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 shutdown_connections 2023-10-06T21:19:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 >> 172.21.15.31:0/2073250096 conn(0x7f1c40075e00 msgr2=0x7f1c400fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.405+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 shutdown_connections 2023-10-06T21:19:28.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.406+0000 7f1c4683c700 1 -- 172.21.15.31:0/2073250096 wait complete. 2023-10-06T21:19:28.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 -- 172.21.15.31:0/2939338509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 msgr2=0x7f90a0068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/2939338509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0068d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9090009990 tx=0x7f9090009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 -- 172.21.15.31:0/2939338509 shutdown_connections 2023-10-06T21:19:28.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/2939338509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a010da10 0x7f90a010fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/2939338509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0068d80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/2939338509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a00692c0 0x7f90a010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 -- 172.21.15.31:0/2939338509 >> 172.21.15.31:0/2939338509 conn(0x7f90a0075e00 msgr2=0x7f90a0076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.430+0000 7f90a5c5b700 1 -- 172.21.15.31:0/2939338509 shutdown_connections 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.431+0000 7f90a5c5b700 1 -- 172.21.15.31:0/2939338509 wait complete. 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.431+0000 7f90a5c5b700 1 Processor -- start 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 -- start start 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a00692c0 0x7f90a0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a010da10 0x7f90a019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019e0f0 con 0x7f90a00689a0 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019e260 con 0x7f90a010da10 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f90a5c5b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019e420 con 0x7f90a00692c0 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f909f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f909f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52358/0 (socket says 172.21.15.31:52358) 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.432+0000 7f909f7fe700 1 -- 172.21.15.31:0/1239442419 learned_addr learned my addr 172.21.15.31:0/1239442419 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909effd700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a00692c0 0x7f90a0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909ffff700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a010da10 0x7f90a019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 -- 172.21.15.31:0/1239442419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a00692c0 msgr2=0x7f90a0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a00692c0 0x7f90a0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 -- 172.21.15.31:0/1239442419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a010da10 msgr2=0x7f90a019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a010da10 0x7f90a019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900095f0 con 0x7f90a00689a0 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909f7fe700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0199130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9090000c00 tx=0x7f9090009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909001d070 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9090022470 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a019e6a0 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.433+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090017650 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.434+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a01a5f60 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.434+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90a0113220 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.437+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f909000fb00 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909cff9700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90880a1590 0x7f90880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f90900ced90 con 0x7f90a00689a0 2023-10-06T21:19:28.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909effd700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90880a1590 0x7f90880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909cff9700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] conn(0x7f90880ab1d0 0x7f90880ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90880add40 con 0x7f90880ab1d0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.438+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90900cf100 con 0x7f90a00689a0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.439+0000 7f909ffff700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] conn(0x7f90880ab1d0 0x7f90880ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.439+0000 7f909effd700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90880a1590 0x7f90880a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90a019a750 tx=0x7f9094009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.439+0000 7f909ffff700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] conn(0x7f90880ab1d0 0x7f90880ad670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.440+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== osd.3 v2:172.21.15.31:6810/3470883528 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f90880add40 con 0x7f90880ab1d0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.457+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 --> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90a004ebb0 con 0x7f90880ab1d0 2023-10-06T21:19:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.457+0000 7f909cff9700 1 -- 172.21.15.31:0/1239442419 <== osd.3 v2:172.21.15.31:6810/3470883528 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f90a004ebb0 con 0x7f90880ab1d0 2023-10-06T21:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] conn(0x7f90880ab1d0 msgr2=0x7f90880ad670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] conn(0x7f90880ab1d0 0x7f90880ad670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90880a1590 msgr2=0x7f90880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90880a1590 0x7f90880a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90a019a750 tx=0x7f9094009260 comp rx=0 tx=0).stop 2023-10-06T21:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 msgr2=0x7f90a0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 --2- 172.21.15.31:0/1239442419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a00689a0 0x7f90a0199130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9090000c00 tx=0x7f9090009570 comp rx=0 tx=0).stop 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f909ffff700 1 -- 172.21.15.31:0/1239442419 reap_dead start 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 shutdown_connections 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 >> 172.21.15.31:0/1239442419 conn(0x7f90a0075e00 msgr2=0x7f90a010ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 shutdown_connections 2023-10-06T21:19:28.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:28.458+0000 7f90a5c5b700 1 -- 172.21.15.31:0/1239442419 wait complete. 2023-10-06T21:19:28.643 INFO:teuthology.orchestra.run.smithi031.stdout:223338299432 2023-10-06T21:19:28.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-06T21:19:28.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:28 smithi031 ceph-mon[98539]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-06T21:19:29.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:28 smithi142 ceph-mon[102951]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-06T21:19:29.263 INFO:teuthology.orchestra.run.smithi031.stdout:201863462958 2023-10-06T21:19:29.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-06T21:19:29.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:28 smithi146 ceph-mon[102809]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 -- 172.21.15.31:0/3885172920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c0695b0 msgr2=0x7fda8c105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 --2- 172.21.15.31:0/3885172920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c0695b0 0x7fda8c105b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda80009990 tx=0x7fda80009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 -- 172.21.15.31:0/3885172920 shutdown_connections 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 --2- 172.21.15.31:0/3885172920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 0x7fda8c10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 --2- 172.21.15.31:0/3885172920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda8c068c00 0x7fda8c068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 --2- 172.21.15.31:0/3885172920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c0695b0 0x7fda8c105b80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 -- 172.21.15.31:0/3885172920 >> 172.21.15.31:0/3885172920 conn(0x7fda8c075e00 msgr2=0x7fda8c076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 -- 172.21.15.31:0/3885172920 shutdown_connections 2023-10-06T21:19:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.341+0000 7fda919e1700 1 -- 172.21.15.31:0/3885172920 wait complete. 2023-10-06T21:19:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.343+0000 7fda919e1700 1 Processor -- start 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.343+0000 7fda919e1700 1 -- start start 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 0x7fda8c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda8c0695b0 0x7fda8c100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 0x7fda8c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a5460 con 0x7fda8c1062b0 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a55d0 con 0x7fda8c068c00 2023-10-06T21:19:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda919e1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a5790 con 0x7fda8c0695b0 2023-10-06T21:19:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda8affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 0x7fda8c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda8b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 0x7fda8c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda8affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 0x7fda8c0fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56656/0 (socket says 172.21.15.31:56656) 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda8affd700 1 -- 172.21.15.31:0/2051223723 learned_addr learned my addr 172.21.15.31:0/2051223723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.344+0000 7fda8a7fc700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda8c0695b0 0x7fda8c100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 -- 172.21.15.31:0/2051223723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda8c0695b0 msgr2=0x7fda8c100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda8c0695b0 0x7fda8c100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 -- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 msgr2=0x7fda8c0fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 0x7fda8c0fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda84009c40 con 0x7fda8c1062b0 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8affd700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda8c068c00 0x7fda8c0fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda8b7fe700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 0x7fda8c1009f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fda8400efb0 tx=0x7fda8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84003a00 con 0x7fda8c1062b0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda800095f0 con 0x7fda8c1062b0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda8400f040 con 0x7fda8c1062b0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.345+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda840044d0 con 0x7fda8c1062b0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.346+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c1a5db0 con 0x7fda8c1062b0 2023-10-06T21:19:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.348+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda8c04ebb0 con 0x7fda8c1062b0 2023-10-06T21:19:29.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.349+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fda84004630 con 0x7fda8c1062b0 2023-10-06T21:19:29.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.350+0000 7fda909df700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fda740a1510 0x7fda740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:29.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda840d53a0 con 0x7fda8c1062b0 2023-10-06T21:19:29.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda8affd700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fda740a1510 0x7fda740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:29.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda909df700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] conn(0x7fda740ab150 0x7fda740ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:29.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda740adcc0 con 0x7fda740ab150 2023-10-06T21:19:29.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda840d5760 con 0x7fda8c1062b0 2023-10-06T21:19:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda8affd700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fda740a1510 0x7fda740a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda7c011550 tx=0x7fda7c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.351+0000 7fda8a7fc700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] conn(0x7fda740ab150 0x7fda740ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.352+0000 7fda8a7fc700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] conn(0x7fda740ab150 0x7fda740ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.353+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== osd.4 v2:172.21.15.142:6808/3887215066 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fda740adcc0 con 0x7fda740ab150 2023-10-06T21:19:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.371+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 --> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda8c111050 con 0x7fda740ab150 2023-10-06T21:19:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.372+0000 7fda909df700 1 -- 172.21.15.31:0/2051223723 <== osd.4 v2:172.21.15.142:6808/3887215066 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fda8001e040 con 0x7fda740ab150 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.372+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] conn(0x7fda740ab150 msgr2=0x7fda740ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.372+0000 7fda919e1700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] conn(0x7fda740ab150 0x7fda740ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fda740a1510 msgr2=0x7fda740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fda740a1510 0x7fda740a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda7c011550 tx=0x7fda7c009260 comp rx=0 tx=0).stop 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 msgr2=0x7fda8c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 --2- 172.21.15.31:0/2051223723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8c1062b0 0x7fda8c1009f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fda8400efb0 tx=0x7fda8400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda8b7fe700 1 -- 172.21.15.31:0/2051223723 reap_dead start 2023-10-06T21:19:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 shutdown_connections 2023-10-06T21:19:29.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 >> 172.21.15.31:0/2051223723 conn(0x7fda8c075e00 msgr2=0x7fda8c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:29.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 shutdown_connections 2023-10-06T21:19:29.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:29.373+0000 7fda919e1700 1 -- 172.21.15.31:0/2051223723 wait complete. 2023-10-06T21:19:30.492 INFO:teuthology.orchestra.run.smithi031.stdout:77309411396 2023-10-06T21:19:30.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-06T21:19:30.556 INFO:teuthology.orchestra.run.smithi031.stdout:146028888120 2023-10-06T21:19:30.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-06T21:19:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.871+0000 7f6232d23700 1 -- 172.21.15.31:0/3004807063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 msgr2=0x7f622c1163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.871+0000 7f6232d23700 1 --2- 172.21.15.31:0/3004807063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 0x7f622c1163f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6224013890 tx=0x7f6224013ba0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.874+0000 7f6232d23700 1 -- 172.21.15.31:0/3004807063 shutdown_connections 2023-10-06T21:19:30.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.874+0000 7f6232d23700 1 --2- 172.21.15.31:0/3004807063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 0x7f622c1163f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.874+0000 7f6232d23700 1 --2- 172.21.15.31:0/3004807063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622c1080e0 0x7f622c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.874+0000 7f6232d23700 1 --2- 172.21.15.31:0/3004807063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c108a00 0x7f622c113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.874+0000 7f6232d23700 1 -- 172.21.15.31:0/3004807063 >> 172.21.15.31:0/3004807063 conn(0x7f622c06d0e0 msgr2=0x7f622c06d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.883+0000 7f6232d23700 1 -- 172.21.15.31:0/3004807063 shutdown_connections 2023-10-06T21:19:30.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.884+0000 7f6232d23700 1 -- 172.21.15.31:0/3004807063 wait complete. 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.887+0000 7f6232d23700 1 Processor -- start 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.887+0000 7f6232d23700 1 -- start start 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 0x7f622c1ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622c108a00 0x7f622c1aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 0x7f622c1b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1b3540 con 0x7f622c1080e0 2023-10-06T21:19:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1b36b0 con 0x7f622c114000 2023-10-06T21:19:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.888+0000 7f6232d23700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1b3870 con 0x7f622c108a00 2023-10-06T21:19:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.890+0000 7f6230abf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 0x7f622c1ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.890+0000 7f6230abf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 0x7f622c1ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52402/0 (socket says 172.21.15.31:52402) 2023-10-06T21:19:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.890+0000 7f6230abf700 1 -- 172.21.15.31:0/3565613059 learned_addr learned my addr 172.21.15.31:0/3565613059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.891+0000 7f6230abf700 1 -- 172.21.15.31:0/3565613059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622c108a00 msgr2=0x7f622c1aeb10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.891+0000 7f6230abf700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622c108a00 0x7f622c1aeb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.891+0000 7f6230abf700 1 -- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 msgr2=0x7f622c1b2f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.891+0000 7f6230abf700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f622c114000 0x7f622c1b2f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.891+0000 7f6230abf700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224009c40 con 0x7f622c1080e0 2023-10-06T21:19:30.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.892+0000 7f6230abf700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 0x7f622c1ae5d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f622000db20 tx=0x7f622000dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.893+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000ca90 con 0x7f622c1080e0 2023-10-06T21:19:30.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.893+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f622000cbf0 con 0x7f622c1080e0 2023-10-06T21:19:30.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.893+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220014610 con 0x7f622c1080e0 2023-10-06T21:19:30.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.893+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c1b3b50 con 0x7f622c1080e0 2023-10-06T21:19:30.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.893+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c1bb3b0 con 0x7f622c1080e0 2023-10-06T21:19:30.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.895+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6210000fc0 con 0x7f622c1080e0 2023-10-06T21:19:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.900+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f622000cd60 con 0x7f622c1080e0 2023-10-06T21:19:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.903+0000 7f6229ffb700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f621c0a1710 0x7f621c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.904+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f62200ccde0 con 0x7f622c1080e0 2023-10-06T21:19:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.904+0000 7f6229ffb700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] conn(0x7f621c0ab350 0x7f621c0ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.904+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f621c0adec0 con 0x7f621c0ab350 2023-10-06T21:19:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.904+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62200cd190 con 0x7f622c1080e0 2023-10-06T21:19:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.905+0000 7f62312c0700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] conn(0x7f621c0ab350 0x7f621c0ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.906+0000 7f622bfff700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f621c0a1710 0x7f621c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.907+0000 7f622bfff700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f621c0a1710 0x7f621c0a3bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f622c1afba0 tx=0x7f6218006cb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.908+0000 7f62312c0700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] conn(0x7f621c0ab350 0x7f621c0ad7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.910+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== osd.10 v2:172.21.15.142:6824/1169171647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f621c0adec0 con 0x7f621c0ab350 2023-10-06T21:19:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.945+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 --> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6210002d40 con 0x7f621c0ab350 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.946+0000 7f6229ffb700 1 -- 172.21.15.31:0/3565613059 <== osd.10 v2:172.21.15.142:6824/1169171647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6210002d40 con 0x7f621c0ab350 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.947+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] conn(0x7f621c0ab350 msgr2=0x7f621c0ad7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.947+0000 7f6232d23700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] conn(0x7f621c0ab350 0x7f621c0ad7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f621c0a1710 msgr2=0x7f621c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f621c0a1710 0x7f621c0a3bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f622c1afba0 tx=0x7f6218006cb0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 msgr2=0x7f622c1ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 --2- 172.21.15.31:0/3565613059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622c1080e0 0x7f622c1ae5d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f622000db20 tx=0x7f622000dee0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f62312c0700 1 -- 172.21.15.31:0/3565613059 reap_dead start 2023-10-06T21:19:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 shutdown_connections 2023-10-06T21:19:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.948+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 >> 172.21.15.31:0/3565613059 conn(0x7f622c06d0e0 msgr2=0x7f622c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.949+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 shutdown_connections 2023-10-06T21:19:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.949+0000 7f6232d23700 1 -- 172.21.15.31:0/3565613059 wait complete. 2023-10-06T21:19:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.950+0000 7f8e7062b700 1 -- 172.21.15.31:0/2003619134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680ffe60 msgr2=0x7f8e681002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.950+0000 7f8e7062b700 1 --2- 172.21.15.31:0/2003619134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680ffe60 0x7f8e681002e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009990 tx=0x7f8e64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 -- 172.21.15.31:0/2003619134 shutdown_connections 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 --2- 172.21.15.31:0/2003619134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e68100820 0x7f8e6810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 --2- 172.21.15.31:0/2003619134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 0x7f8e68106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 --2- 172.21.15.31:0/2003619134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680ffe60 0x7f8e681002e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 -- 172.21.15.31:0/2003619134 >> 172.21.15.31:0/2003619134 conn(0x7f8e68075e00 msgr2=0x7f8e68076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 -- 172.21.15.31:0/2003619134 shutdown_connections 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.951+0000 7f8e7062b700 1 -- 172.21.15.31:0/2003619134 wait complete. 2023-10-06T21:19:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.953+0000 7f8e7062b700 1 Processor -- start 2023-10-06T21:19:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.953+0000 7f8e7062b700 1 -- start start 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.953+0000 7f8e7062b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 0x7f8e68196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.953+0000 7f8e7062b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68100820 0x7f8e68197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e7062b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 0x7f8e6819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e7062b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819bec0 con 0x7f8e68105e70 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e7062b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819c030 con 0x7f8e68100820 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e7062b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819c1f0 con 0x7f8e680ffe60 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 0x7f8e68196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 0x7f8e68196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57678/0 (socket says 172.21.15.31:57678) 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 -- 172.21.15.31:0/1698644134 learned_addr learned my addr 172.21.15.31:0/1698644134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6dbc6700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68100820 0x7f8e68197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6ebc8700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 0x7f8e6819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 -- 172.21.15.31:0/1698644134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68100820 msgr2=0x7f8e68197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68100820 0x7f8e68197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 -- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 msgr2=0x7f8e6819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 0x7f8e6819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6e3c7700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640095f0 con 0x7f8e680ffe60 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.954+0000 7f8e6dbc6700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68100820 0x7f8e68197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.955+0000 7f8e6ebc8700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e68105e70 0x7f8e6819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.955+0000 7f8e6e3c7700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 0x7f8e68196f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef10 tx=0x7f8e6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.957+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000ca70 con 0x7f8e680ffe60 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.957+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e6000f040 con 0x7f8e680ffe60 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.957+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60014480 con 0x7f8e680ffe60 2023-10-06T21:19:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.957+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6819c470 con 0x7f8e680ffe60 2023-10-06T21:19:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.957+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6810d620 con 0x7f8e680ffe60 2023-10-06T21:19:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.958+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e6819c750 con 0x7f8e680ffe60 2023-10-06T21:19:30.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.961+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8e60004ad0 con 0x7f8e680ffe60 2023-10-06T21:19:30.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e5f7fe700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e540a1690 0x7f8e540a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e600cd260 con 0x7f8e680ffe60 2023-10-06T21:19:30.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e5f7fe700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] conn(0x7f8e540ab2d0 0x7f8e540ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e540ade40 con 0x7f8e540ab2d0 2023-10-06T21:19:30.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e600cd5d0 con 0x7f8e680ffe60 2023-10-06T21:19:30.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.963+0000 7f8e6dbc6700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e540a1690 0x7f8e540a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.964+0000 7f8e6ebc8700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] conn(0x7f8e540ab2d0 0x7f8e540ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.964+0000 7f8e6dbc6700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e540a1690 0x7f8e540a3b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e68198520 tx=0x7f8e64005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.964+0000 7f8e6ebc8700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] conn(0x7f8e540ab2d0 0x7f8e540ad770 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.966+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== osd.8 v2:172.21.15.146:6816/1953163251 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8e540ade40 con 0x7f8e540ab2d0 2023-10-06T21:19:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.987+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 --> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e681110c0 con 0x7f8e540ab2d0 2023-10-06T21:19:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.987+0000 7f8e5f7fe700 1 -- 172.21.15.31:0/1698644134 <== osd.8 v2:172.21.15.146:6816/1953163251 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8e5800b800 con 0x7f8e540ab2d0 2023-10-06T21:19:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] conn(0x7f8e540ab2d0 msgr2=0x7f8e540ad770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] conn(0x7f8e540ab2d0 0x7f8e540ad770 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e540a1690 msgr2=0x7f8e540a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e540a1690 0x7f8e540a3b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e68198520 tx=0x7f8e64005fd0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 msgr2=0x7f8e68196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 --2- 172.21.15.31:0/1698644134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e680ffe60 0x7f8e68196f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef10 tx=0x7f8e6000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e6ebc8700 1 -- 172.21.15.31:0/1698644134 reap_dead start 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 shutdown_connections 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 >> 172.21.15.31:0/1698644134 conn(0x7f8e68075e00 msgr2=0x7f8e680fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 shutdown_connections 2023-10-06T21:19:30.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:30.988+0000 7f8e7062b700 1 -- 172.21.15.31:0/1698644134 wait complete. 2023-10-06T21:19:31.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:30 smithi031 ceph-mon[98539]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:31.148 INFO:teuthology.orchestra.run.smithi031.stdout:111669149758 2023-10-06T21:19:31.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-06T21:19:31.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:30 smithi142 ceph-mon[102951]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:31.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1239937570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 msgr2=0x7f8e64068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1239937570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64068fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009990 tx=0x7f8e60009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1239937570 shutdown_connections 2023-10-06T21:19:31.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1239937570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e641062b0 0x7f8e6410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1239937570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64068fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1239937570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e640695b0 0x7f8e64105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.187+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1239937570 >> 172.21.15.31:0/1239937570 conn(0x7f8e64075e00 msgr2=0x7f8e64076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:31.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.188+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1239937570 shutdown_connections 2023-10-06T21:19:31.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.188+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1239937570 wait complete. 2023-10-06T21:19:31.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 Processor -- start 2023-10-06T21:19:31.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 -- start start 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 0x7f8e64197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e641062b0 0x7f8e6419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6419bff0 con 0x7f8e64068c00 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6419c160 con 0x7f8e641062b0 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.189+0000 7f8e6baeb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6419c320 con 0x7f8e640695b0 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69887700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 0x7f8e64197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 0x7f8e64197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57700/0 (socket says 172.21.15.31:57700) 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 -- 172.21.15.31:0/1749685078 learned_addr learned my addr 172.21.15.31:0/1749685078 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e6a088700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e641062b0 0x7f8e6419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 -- 172.21.15.31:0/1749685078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e641062b0 msgr2=0x7f8e6419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e641062b0 0x7f8e6419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 -- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 msgr2=0x7f8e64197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600095f0 con 0x7f8e640695b0 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69887700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e64068c00 0x7f8e64197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:31.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.190+0000 7f8e69086700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 0x7f8e64197570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d950 tx=0x7f8e5400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.191+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004220 con 0x7f8e640695b0 2023-10-06T21:19:31.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.191+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6419c600 con 0x7f8e640695b0 2023-10-06T21:19:31.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.191+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e54004cc0 con 0x7f8e640695b0 2023-10-06T21:19:31.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.191+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54005600 con 0x7f8e640695b0 2023-10-06T21:19:31.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.191+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e64100030 con 0x7f8e640695b0 2023-10-06T21:19:31.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.192+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e64111030 con 0x7f8e640695b0 2023-10-06T21:19:31.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.194+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8e54004380 con 0x7f8e640695b0 2023-10-06T21:19:31.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e5affd700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e500a1510 0x7f8e500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e540cd3b0 con 0x7f8e640695b0 2023-10-06T21:19:31.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e69887700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e500a1510 0x7f8e500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e5affd700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] conn(0x7f8e500ab150 0x7f8e500ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e500adcc0 con 0x7f8e500ab150 2023-10-06T21:19:31.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.196+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e540cd720 con 0x7f8e640695b0 2023-10-06T21:19:31.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.197+0000 7f8e6a088700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] conn(0x7f8e500ab150 0x7f8e500ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.197+0000 7f8e69887700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e500a1510 0x7f8e500a39d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b5c0 tx=0x7f8e60005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.197+0000 7f8e6a088700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] conn(0x7f8e500ab150 0x7f8e500ad5f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.198+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== osd.11 v2:172.21.15.146:6824/740640181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8e500adcc0 con 0x7f8e500ab150 2023-10-06T21:19:31.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.222+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 --> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e6404ebb0 con 0x7f8e500ab150 2023-10-06T21:19:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e5affd700 1 -- 172.21.15.31:0/1749685078 <== osd.11 v2:172.21.15.146:6824/740640181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8e6404ebb0 con 0x7f8e500ab150 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] conn(0x7f8e500ab150 msgr2=0x7f8e500ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] conn(0x7f8e500ab150 0x7f8e500ad5f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e500a1510 msgr2=0x7f8e500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e500a1510 0x7f8e500a39d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b5c0 tx=0x7f8e60005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 msgr2=0x7f8e64197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6baeb700 1 --2- 172.21.15.31:0/1749685078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e640695b0 0x7f8e64197570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d950 tx=0x7f8e5400dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.223+0000 7f8e6a088700 1 -- 172.21.15.31:0/1749685078 reap_dead start 2023-10-06T21:19:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.224+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 shutdown_connections 2023-10-06T21:19:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.224+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 >> 172.21.15.31:0/1749685078 conn(0x7f8e64075e00 msgr2=0x7f8e640ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.224+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 shutdown_connections 2023-10-06T21:19:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.224+0000 7f8e6baeb700 1 -- 172.21.15.31:0/1749685078 wait complete. 2023-10-06T21:19:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:30 smithi146 ceph-mon[102809]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 -- 172.21.15.31:0/4099626451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c073a00 msgr2=0x7fd19c10d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 --2- 172.21.15.31:0/4099626451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c073a00 0x7fd19c10d3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd190009990 tx=0x7fd190009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 -- 172.21.15.31:0/4099626451 shutdown_connections 2023-10-06T21:19:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 --2- 172.21.15.31:0/4099626451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd19c10d930 0x7fd19c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 --2- 172.21.15.31:0/4099626451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c0730e0 0x7fd19c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 --2- 172.21.15.31:0/4099626451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c073a00 0x7fd19c10d3f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 -- 172.21.15.31:0/4099626451 >> 172.21.15.31:0/4099626451 conn(0x7fd19c0780a0 msgr2=0x7fd19c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 -- 172.21.15.31:0/4099626451 shutdown_connections 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.584+0000 7fd1a172f700 1 -- 172.21.15.31:0/4099626451 wait complete. 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.585+0000 7fd1a172f700 1 Processor -- start 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.585+0000 7fd1a172f700 1 -- start start 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 0x7fd19c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd19c073a00 0x7fd19c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c10d930 0x7fd19c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19e170 con 0x7fd19c0730e0 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19e2e0 con 0x7fd19c073a00 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd1a172f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19e4a0 con 0x7fd19c10d930 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 0x7fd19c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd19c073a00 0x7fd19c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 0x7fd19c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52474/0 (socket says 172.21.15.31:52474) 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 -- 172.21.15.31:0/162085754 learned_addr learned my addr 172.21.15.31:0/162085754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19b7fe700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c10d930 0x7fd19c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 -- 172.21.15.31:0/162085754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c10d930 msgr2=0x7fd19c19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c10d930 0x7fd19c19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 -- 172.21.15.31:0/162085754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd19c073a00 msgr2=0x7fd19c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd19c073a00 0x7fd19c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19affd700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900095f0 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.586+0000 7fd19b7fe700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd19c10d930 0x7fd19c19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd19affd700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 0x7fd19c1991b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9a0 tx=0x7fd18c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c009790 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd18c00cbd0 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c018680 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c19e780 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.587+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c1a5fe0 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.588+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd19c1131a0 con 0x7fd19c0730e0 2023-10-06T21:19:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.590+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd18c00cd40 con 0x7fd19c0730e0 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.591+0000 7fd183fff700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1840a1510 0x7fd1840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.591+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd18c014070 con 0x7fd19c0730e0 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd19a7fc700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1840a1510 0x7fd1840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd183fff700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] conn(0x7fd1840ab150 0x7fd1840ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1840adcc0 con 0x7fd1840ab150 2023-10-06T21:19:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18c0cd930 con 0x7fd19c0730e0 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd19b7fe700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] conn(0x7fd1840ab150 0x7fd1840ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.592+0000 7fd19a7fc700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1840a1510 0x7fd1840a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd19c19a7d0 tx=0x7fd19000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.593+0000 7fd19b7fe700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] conn(0x7fd1840ab150 0x7fd1840ad5f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.593+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== osd.9 v2:172.21.15.31:6826/3073626060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd1840adcc0 con 0x7fd1840ab150 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.610+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 --> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd19c04ebb0 con 0x7fd1840ab150 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd183fff700 1 -- 172.21.15.31:0/162085754 <== osd.9 v2:172.21.15.31:6826/3073626060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd19c04ebb0 con 0x7fd1840ab150 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] conn(0x7fd1840ab150 msgr2=0x7fd1840ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] conn(0x7fd1840ab150 0x7fd1840ad5f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1840a1510 msgr2=0x7fd1840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd1840a1510 0x7fd1840a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd19c19a7d0 tx=0x7fd19000b260 comp rx=0 tx=0).stop 2023-10-06T21:19:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 msgr2=0x7fd19c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 --2- 172.21.15.31:0/162085754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0730e0 0x7fd19c1991b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9a0 tx=0x7fd18c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd19b7fe700 1 -- 172.21.15.31:0/162085754 reap_dead start 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 shutdown_connections 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 >> 172.21.15.31:0/162085754 conn(0x7fd19c0780a0 msgr2=0x7fd19c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 shutdown_connections 2023-10-06T21:19:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:31.611+0000 7fd1a172f700 1 -- 172.21.15.31:0/162085754 wait complete. 2023-10-06T21:19:32.069 INFO:teuthology.orchestra.run.smithi031.stdout:347892350996 2023-10-06T21:19:32.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-06T21:19:32.158 INFO:teuthology.orchestra.run.smithi031.stdout:313532612634 2023-10-06T21:19:32.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-06T21:19:32.820 INFO:teuthology.orchestra.run.smithi031.stdout:292057776158 2023-10-06T21:19:32.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-06T21:19:32.841 INFO:teuthology.orchestra.run.smithi031.stdout:257698037796 2023-10-06T21:19:32.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-06T21:19:32.864 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:32.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:32 smithi031 ceph-mon[98539]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:33.076 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.138 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:32 smithi142 ceph-mon[102951]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:33.220 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:32 smithi146 ceph-mon[102809]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.485 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.486 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:33.487 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:35.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:34 smithi031 ceph-mon[98539]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:35.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:34 smithi142 ceph-mon[102951]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:34 smithi146 ceph-mon[102809]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:37.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:36 smithi031 ceph-mon[98539]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s; 11 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:37.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:36 smithi142 ceph-mon[102951]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s; 11 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:36 smithi146 ceph-mon[102809]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s; 11 B/s, 1 keys/s, 2 objects/s recovering 2023-10-06T21:19:37.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3344976803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58106f90 msgr2=0x7f2b58107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3344976803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58106f90 0x7f2b58107370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2b480099c0 tx=0x7f2b48009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3344976803 shutdown_connections 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3344976803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 0x7f2b5810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3344976803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58106f90 0x7f2b58107370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3344976803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58100f80 0x7f2b58101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3344976803 >> 172.21.15.31:0/3344976803 conn(0x7f2b580780a0 msgr2=0x7f2b580784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3344976803 shutdown_connections 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.717+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3344976803 wait complete. 2023-10-06T21:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.718+0000 7f2b5c9a8700 1 Processor -- start 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 -- start start 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 0x7f2b580736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 0x7f2b58073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581a52e0 con 0x7f2b58100f80 2023-10-06T21:19:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581a5450 con 0x7f2b58101940 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5c9a8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581a5610 con 0x7f2b58106f90 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43532/0 (socket says 172.21.15.31:43532) 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b5659c700 1 -- 172.21.15.31:0/3123550493 learned_addr learned my addr 172.21.15.31:0/3123550493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.719+0000 7f2b55d9b700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 0x7f2b580736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b56d9d700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 0x7f2b58073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 -- 172.21.15.31:0/3123550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 msgr2=0x7f2b58073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 0x7f2b58073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 -- 172.21.15.31:0/3123550493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 msgr2=0x7f2b580736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 0x7f2b580736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 -- 172.21.15.31:0/3123550493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480095f0 con 0x7f2b58100f80 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b56d9d700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 0x7f2b58073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5659c700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009f50 tx=0x7f2b48004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801d070 con 0x7f2b58100f80 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b48003fc0 con 0x7f2b58100f80 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b581a5890 con 0x7f2b58100f80 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.720+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48022c30 con 0x7f2b58100f80 2023-10-06T21:19:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.721+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b581a5d80 con 0x7f2b58100f80 2023-10-06T21:19:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.722+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5804ebb0 con 0x7f2b58100f80 2023-10-06T21:19:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.723+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2b48022470 con 0x7f2b58100f80 2023-10-06T21:19:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.725+0000 7f2b477fe700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 0x7f2b400a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.725+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2b480ce2d0 con 0x7f2b58100f80 2023-10-06T21:19:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.725+0000 7f2b55d9b700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 0x7f2b400a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.726+0000 7f2b55d9b700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 0x7f2b400a3a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b580749a0 tx=0x7f2b4c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.729+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b480977e0 con 0x7f2b58100f80 2023-10-06T21:19:37.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.966+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b58110de0 con 0x7f2b58100f80 2023-10-06T21:19:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.967+0000 7f2b477fe700 1 -- 172.21.15.31:0/3123550493 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b4809ae00 con 0x7f2b58100f80 2023-10-06T21:19:37.970 INFO:teuthology.orchestra.run.smithi031.stdout:55834574923 2023-10-06T21:19:37.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 msgr2=0x7f2b400a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 0x7f2b400a3a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b580749a0 tx=0x7f2b4c00b410 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 msgr2=0x7f2b580731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009f50 tx=0x7f2b48004710 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 shutdown_connections 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2b400a1590 0x7f2b400a3a50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b58106f90 0x7f2b58073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b58100f80 0x7f2b580731b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 --2- 172.21.15.31:0/3123550493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b58101940 0x7f2b580736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.969+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 >> 172.21.15.31:0/3123550493 conn(0x7f2b580780a0 msgr2=0x7f2b58104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.970+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 shutdown_connections 2023-10-06T21:19:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:37.970+0000 7f2b5c9a8700 1 -- 172.21.15.31:0/3123550493 wait complete. 2023-10-06T21:19:38.968 INFO:teuthology.orchestra.run.smithi031.stdout:77309411397 2023-10-06T21:19:38.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.558+0000 7f04875b5700 1 -- 172.21.15.31:0/3503539100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480106f90 msgr2=0x7f0480107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.558+0000 7f04875b5700 1 --2- 172.21.15.31:0/3503539100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480106f90 0x7f0480107370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f047c0099c0 tx=0x7f047c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 -- 172.21.15.31:0/3503539100 shutdown_connections 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 --2- 172.21.15.31:0/3503539100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480101940 0x7f048010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 --2- 172.21.15.31:0/3503539100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480106f90 0x7f0480107370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 --2- 172.21.15.31:0/3503539100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 0x7f0480101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 -- 172.21.15.31:0/3503539100 >> 172.21.15.31:0/3503539100 conn(0x7f04800780a0 msgr2=0x7f04800784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 -- 172.21.15.31:0/3503539100 shutdown_connections 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.559+0000 7f04875b5700 1 -- 172.21.15.31:0/3503539100 wait complete. 2023-10-06T21:19:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 Processor -- start 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 -- start start 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 0x7f0480199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480106f90 0x7f048019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048019e120 con 0x7f0480101940 2023-10-06T21:19:38.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048019e290 con 0x7f0480100f80 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.560+0000 7f04875b5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048019e450 con 0x7f0480106f90 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43554/0 (socket says 172.21.15.31:43554) 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 -- 172.21.15.31:0/2740005015 learned_addr learned my addr 172.21.15.31:0/2740005015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0485b52700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480106f90 0x7f048019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0485351700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 0x7f0480199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 -- 172.21.15.31:0/2740005015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480106f90 msgr2=0x7f048019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480106f90 0x7f048019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 -- 172.21.15.31:0/2740005015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 msgr2=0x7f0480199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 0x7f0480199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 -- 172.21.15.31:0/2740005015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0095f0 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.561+0000 7f0484b50700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f047000b530 tx=0x7f047000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.562+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470011830 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.562+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0470011e20 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.562+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048019e730 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.562+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000f2e0 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.562+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04801a5f90 con 0x7f0480101940 2023-10-06T21:19:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.563+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048004ebb0 con 0x7f0480101940 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.565+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0470011990 con 0x7f0480101940 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.566+0000 7f04767fc700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 0x7f046c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.566+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f04700cc5e0 con 0x7f0480101940 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.567+0000 7f0485351700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 0x7f046c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.567+0000 7f0485351700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 0x7f046c0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f047c0036c0 tx=0x7f047c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.571+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0470095b70 con 0x7f0480101940 2023-10-06T21:19:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.793+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f048019a4c0 con 0x7f0480101940 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.794+0000 7f04767fc700 1 -- 172.21.15.31:0/2740005015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0470099190 con 0x7f0480101940 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 msgr2=0x7f046c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 0x7f046c0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f047c0036c0 tx=0x7f047c005fd0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 msgr2=0x7f04801996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f047000b530 tx=0x7f047000b840 comp rx=0 tx=0).stop 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 shutdown_connections 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f046c0a1510 0x7f046c0a39d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0480106f90 0x7f048019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0480100f80 0x7f0480199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 --2- 172.21.15.31:0/2740005015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0480101940 0x7f04801996a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 >> 172.21.15.31:0/2740005015 conn(0x7f04800780a0 msgr2=0x7f048010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 shutdown_connections 2023-10-06T21:19:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:38.796+0000 7f04875b5700 1 -- 172.21.15.31:0/2740005015 wait complete. 2023-10-06T21:19:39.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:38 smithi031 ceph-mon[98539]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 279 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T21:19:39.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:38 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3123550493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:19:39.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:38 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2740005015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:19:39.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:38 smithi142 ceph-mon[102951]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 279 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T21:19:39.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:38 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3123550493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:19:39.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:38 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2740005015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:19:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:38 smithi146 ceph-mon[102809]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 279 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T21:19:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:38 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3123550493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:19:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:38 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2740005015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:19:39.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 -- 172.21.15.31:0/4263789749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72541062b0 msgr2=0x7f725410de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 --2- 172.21.15.31:0/4263789749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72541062b0 0x7f725410de30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f724c00b0a0 tx=0x7f724c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 -- 172.21.15.31:0/4263789749 shutdown_connections 2023-10-06T21:19:39.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 --2- 172.21.15.31:0/4263789749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72541062b0 0x7f725410de30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 --2- 172.21.15.31:0/4263789749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7254068c00 0x7f7254068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 --2- 172.21.15.31:0/4263789749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72540695b0 0x7f7254105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 -- 172.21.15.31:0/4263789749 >> 172.21.15.31:0/4263789749 conn(0x7f7254075e00 msgr2=0x7f7254076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 -- 172.21.15.31:0/4263789749 shutdown_connections 2023-10-06T21:19:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.478+0000 7f725a509700 1 -- 172.21.15.31:0/4263789749 wait complete. 2023-10-06T21:19:39.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.479+0000 7f725a509700 1 Processor -- start 2023-10-06T21:19:39.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 -- start start 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72540695b0 0x7f7254197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 0x7f725419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c020 con 0x7f7254068c00 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c190 con 0x7f72540695b0 2023-10-06T21:19:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f725a509700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c350 con 0x7f72541062b0 2023-10-06T21:19:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f7253fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f7253fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43580/0 (socket says 172.21.15.31:43580) 2023-10-06T21:19:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.480+0000 7f7253fff700 1 -- 172.21.15.31:0/2142675417 learned_addr learned my addr 172.21.15.31:0/2142675417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7258aa6700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 0x7f725419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 -- 172.21.15.31:0/2142675417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 msgr2=0x7f725419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 0x7f725419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 -- 172.21.15.31:0/2142675417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72540695b0 msgr2=0x7f7254197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72540695b0 0x7f7254197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 -- 172.21.15.31:0/2142675417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009d00 con 0x7f7254068c00 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7258aa6700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 0x7f725419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f7253fff700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f724400e9a0 tx=0x7f724400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244009790 con 0x7f7254068c00 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f724400cbd0 con 0x7f7254068c00 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244018680 con 0x7f7254068c00 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.481+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725419c630 con 0x7f7254068c00 2023-10-06T21:19:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.482+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72541a3e90 con 0x7f7254068c00 2023-10-06T21:19:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.483+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725419c850 con 0x7f7254068c00 2023-10-06T21:19:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.485+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f724400cd40 con 0x7f7254068c00 2023-10-06T21:19:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.486+0000 7f72517fa700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 0x7f723c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.486+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7244014070 con 0x7f7254068c00 2023-10-06T21:19:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.486+0000 7f72537fe700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 0x7f723c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.487+0000 7f72537fe700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 0x7f723c0a3a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7248005fd0 tx=0x7f7248005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.490+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7244096630 con 0x7f7254068c00 2023-10-06T21:19:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.707+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7254110e60 con 0x7f7254068c00 2023-10-06T21:19:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.708+0000 7f72517fa700 1 -- 172.21.15.31:0/2142675417 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7244099c50 con 0x7f7254068c00 2023-10-06T21:19:39.712 INFO:teuthology.orchestra.run.smithi031.stdout:167503724598 2023-10-06T21:19:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 msgr2=0x7f723c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 0x7f723c0a3a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7248005fd0 tx=0x7f7248005ee0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 msgr2=0x7f7254197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f724400e9a0 tx=0x7f724400ed60 comp rx=0 tx=0).stop 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 shutdown_connections 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f723c0a1590 0x7f723c0a3a50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72541062b0 0x7f725419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7254068c00 0x7f7254197100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.710+0000 7f725a509700 1 --2- 172.21.15.31:0/2142675417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72540695b0 0x7f7254197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.711+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 >> 172.21.15.31:0/2142675417 conn(0x7f7254075e00 msgr2=0x7f72540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.711+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 shutdown_connections 2023-10-06T21:19:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:39.711+0000 7f725a509700 1 -- 172.21.15.31:0/2142675417 wait complete. 2023-10-06T21:19:39.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2023-10-06T21:19:39.892 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:39.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:39 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2142675417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:19:40.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:39 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2142675417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 -- 172.21.15.31:0/3831252287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54106f90 msgr2=0x7fce54107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 --2- 172.21.15.31:0/3831252287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54106f90 0x7fce54107370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fce500099c0 tx=0x7fce50009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 -- 172.21.15.31:0/3831252287 shutdown_connections 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 --2- 172.21.15.31:0/3831252287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce54101940 0x7fce5410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 --2- 172.21.15.31:0/3831252287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54106f90 0x7fce54107370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 --2- 172.21.15.31:0/3831252287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54100f80 0x7fce54101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.245+0000 7fce5befc700 1 -- 172.21.15.31:0/3831252287 >> 172.21.15.31:0/3831252287 conn(0x7fce540780a0 msgr2=0x7fce540784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.246+0000 7fce5befc700 1 -- 172.21.15.31:0/3831252287 shutdown_connections 2023-10-06T21:19:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.246+0000 7fce5befc700 1 -- 172.21.15.31:0/3831252287 wait complete. 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.247+0000 7fce5befc700 1 Processor -- start 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.247+0000 7fce5befc700 1 -- start start 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54101940 0x7fce54197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce54106f90 0x7fce5419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5419be80 con 0x7fce54100f80 2023-10-06T21:19:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5419bff0 con 0x7fce54101940 2023-10-06T21:19:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce5befc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5419c1b0 con 0x7fce54106f90 2023-10-06T21:19:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce59c98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce59c98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43596/0 (socket says 172.21.15.31:43596) 2023-10-06T21:19:40.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce59c98700 1 -- 172.21.15.31:0/1099312753 learned_addr learned my addr 172.21.15.31:0/1099312753 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.248+0000 7fce59497700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54101940 0x7fce54197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 -- 172.21.15.31:0/1099312753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce54106f90 msgr2=0x7fce5419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce54106f90 0x7fce5419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 -- 172.21.15.31:0/1099312753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54101940 msgr2=0x7fce54197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54101940 0x7fce54197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 -- 172.21.15.31:0/1099312753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500095f0 con 0x7fce54100f80 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce59c98700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce50009f50 tx=0x7fce500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5001d070 con 0x7fce54100f80 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5419c430 con 0x7fce54100f80 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce541a3cf0 con 0x7fce54100f80 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce50004040 con 0x7fce54100f80 2023-10-06T21:19:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.249+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50022c80 con 0x7fce54100f80 2023-10-06T21:19:40.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.251+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38005320 con 0x7fce54100f80 2023-10-06T21:19:40.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.253+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fce50022470 con 0x7fce54100f80 2023-10-06T21:19:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.255+0000 7fce4affd700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 0x7fce400a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.255+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fce500ce740 con 0x7fce54100f80 2023-10-06T21:19:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.257+0000 7fce59497700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 0x7fce400a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.257+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fce50097c50 con 0x7fce54100f80 2023-10-06T21:19:40.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.258+0000 7fce59497700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 0x7fce400a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce44005950 tx=0x7fce440058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:39 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2142675417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 -- 172.21.15.31:0/3948390343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40101940 msgr2=0x7f3f4010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 --2- 172.21.15.31:0/3948390343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40101940 0x7f3f4010ddb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b0d0 tx=0x7f3f3800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 -- 172.21.15.31:0/3948390343 shutdown_connections 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 --2- 172.21.15.31:0/3948390343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40101940 0x7f3f4010ddb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 --2- 172.21.15.31:0/3948390343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40106f90 0x7f3f40107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 --2- 172.21.15.31:0/3948390343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 0x7f3f40101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 -- 172.21.15.31:0/3948390343 >> 172.21.15.31:0/3948390343 conn(0x7f3f400780a0 msgr2=0x7f3f400784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.328+0000 7f3f452d2700 1 -- 172.21.15.31:0/3948390343 shutdown_connections 2023-10-06T21:19:40.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.329+0000 7f3f452d2700 1 -- 172.21.15.31:0/3948390343 wait complete. 2023-10-06T21:19:40.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.329+0000 7f3f452d2700 1 Processor -- start 2023-10-06T21:19:40.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 -- start start 2023-10-06T21:19:40.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 0x7f3f40197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40101940 0x7f3f40197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4019bfe0 con 0x7f3f40106f90 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4019c150 con 0x7f3f40101940 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.330+0000 7f3f452d2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4019c310 con 0x7f3f40100f80 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43606/0 (socket says 172.21.15.31:43606) 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 -- 172.21.15.31:0/1542364715 learned_addr learned my addr 172.21.15.31:0/1542364715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3effd700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 0x7f3f40197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3e7fc700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40101940 0x7f3f40197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 -- 172.21.15.31:0/1542364715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 msgr2=0x7f3f40197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 0x7f3f40197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 -- 172.21.15.31:0/1542364715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40101940 msgr2=0x7f3f40197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40101940 0x7f3f40197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 -- 172.21.15.31:0/1542364715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38009d00 con 0x7f3f40106f90 2023-10-06T21:19:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.331+0000 7f3f3f7fe700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3f38008e50 tx=0x7f3f38008e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.332+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800e040 con 0x7f3f40106f90 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.332+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f38003e90 con 0x7f3f40106f90 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.332+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3801b460 con 0x7f3f40106f90 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.332+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4019c590 con 0x7f3f40106f90 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.332+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f401a3c80 con 0x7f3f40106f90 2023-10-06T21:19:40.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.333+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4004ebb0 con 0x7f3f40106f90 2023-10-06T21:19:40.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.336+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3f38003a00 con 0x7f3f40106f90 2023-10-06T21:19:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.338+0000 7f3f27fff700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 0x7f3f280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.338+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f380cdec0 con 0x7f3f40106f90 2023-10-06T21:19:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.338+0000 7f3f3effd700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 0x7f3f280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.339+0000 7f3f3effd700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 0x7f3f280a3a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009be0 tx=0x7f3f30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.342+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f380973d0 con 0x7f3f40106f90 2023-10-06T21:19:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.486+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fce38005190 con 0x7fce54100f80 2023-10-06T21:19:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.487+0000 7fce4affd700 1 -- 172.21.15.31:0/1099312753 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fce50027020 con 0x7fce54100f80 2023-10-06T21:19:40.490 INFO:teuthology.orchestra.run.smithi031.stdout:347892350998 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 msgr2=0x7fce400a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 0x7fce400a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce44005950 tx=0x7fce440058e0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 msgr2=0x7fce54196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce50009f50 tx=0x7fce500047c0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 shutdown_connections 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fce400a1590 0x7fce400a3a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce54106f90 0x7fce5419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce54100f80 0x7fce54196ec0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 --2- 172.21.15.31:0/1099312753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce54101940 0x7fce54197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.489+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 >> 172.21.15.31:0/1099312753 conn(0x7fce540780a0 msgr2=0x7fce540ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.490+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 shutdown_connections 2023-10-06T21:19:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.490+0000 7fce5befc700 1 -- 172.21.15.31:0/1099312753 wait complete. 2023-10-06T21:19:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.574+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3f40110e90 con 0x7f3f40106f90 2023-10-06T21:19:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.575+0000 7f3f27fff700 1 -- 172.21.15.31:0/1542364715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f3809a9f0 con 0x7f3f40106f90 2023-10-06T21:19:40.578 INFO:teuthology.orchestra.run.smithi031.stdout:34359738449 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 msgr2=0x7f3f280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 0x7f3f280a3a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009be0 tx=0x7f3f30009260 comp rx=0 tx=0).stop 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 msgr2=0x7f3f4019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3f38008e50 tx=0x7f3f38008e80 comp rx=0 tx=0).stop 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 shutdown_connections 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3f280a1590 0x7f3f280a3a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40106f90 0x7f3f4019b9d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f40100f80 0x7f3f40197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 --2- 172.21.15.31:0/1542364715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f40101940 0x7f3f40197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 >> 172.21.15.31:0/1542364715 conn(0x7f3f400780a0 msgr2=0x7f3f400ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 shutdown_connections 2023-10-06T21:19:40.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.578+0000 7f3f452d2700 1 -- 172.21.15.31:0/1542364715 wait complete. 2023-10-06T21:19:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 -- 172.21.15.31:0/3772805703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 msgr2=0x7f8e08102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3772805703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e08102490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009990 tx=0x7f8e00009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 -- 172.21.15.31:0/3772805703 shutdown_connections 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3772805703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0810b4e0 0x7f8e0810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3772805703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e08102490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3772805703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e081029d0 0x7f8e0810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 -- 172.21.15.31:0/3772805703 >> 172.21.15.31:0/3772805703 conn(0x7f8e080fb920 msgr2=0x7f8e080fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 -- 172.21.15.31:0/3772805703 shutdown_connections 2023-10-06T21:19:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.627+0000 7f8e107b6700 1 -- 172.21.15.31:0/3772805703 wait complete. 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.628+0000 7f8e107b6700 1 Processor -- start 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.628+0000 7f8e107b6700 1 -- start start 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e081991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 0x7f8e0819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0819e180 con 0x7f8e081029d0 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0819e2f0 con 0x7f8e081020b0 2023-10-06T21:19:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e107b6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0819e4b0 con 0x7f8e0810b4e0 2023-10-06T21:19:40.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e0dd51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e0dd51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43618/0 (socket says 172.21.15.31:43618) 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e0dd51700 1 -- 172.21.15.31:0/3661220577 learned_addr learned my addr 172.21.15.31:0/3661220577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.629+0000 7f8e0e552700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e081991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0ed53700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 0x7f8e0819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 -- 172.21.15.31:0/3661220577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 msgr2=0x7f8e0819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 0x7f8e0819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 -- 172.21.15.31:0/3661220577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 msgr2=0x7f8e081991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e081991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 -- 172.21.15.31:0/3661220577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000095f0 con 0x7f8e081029d0 2023-10-06T21:19:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0ed53700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 0x7f8e0819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e0dd51700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00efc0 tx=0x7f8dfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00c9b0 con 0x7f8e081029d0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dfc00f040 con 0x7f8e081029d0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.630+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0819e730 con 0x7f8e081029d0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.631+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc007ea0 con 0x7f8e081029d0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.631+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e081a5ff0 con 0x7f8e081029d0 2023-10-06T21:19:40.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.632+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0804ebb0 con 0x7f8e081029d0 2023-10-06T21:19:40.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.634+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8dfc007600 con 0x7f8e081029d0 2023-10-06T21:19:40.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.636+0000 7f8e077fe700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 0x7f8df00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:40.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.636+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8dfc0ccc70 con 0x7f8e081029d0 2023-10-06T21:19:40.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.636+0000 7f8e0e552700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 0x7f8df00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:40.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.637+0000 7f8e0e552700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 0x7f8df00a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006010 tx=0x7f8e0000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:40.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.640+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8dfc096150 con 0x7f8e081029d0 2023-10-06T21:19:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.861+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8e0819a520 con 0x7f8e081029d0 2023-10-06T21:19:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.861+0000 7f8e077fe700 1 -- 172.21.15.31:0/3661220577 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8dfc099770 con 0x7f8e081029d0 2023-10-06T21:19:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:146028888122 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.863+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 msgr2=0x7f8df00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.863+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 0x7f8df00a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006010 tx=0x7f8e0000b260 comp rx=0 tx=0).stop 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.863+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 msgr2=0x7f8e08199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.863+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00efc0 tx=0x7f8dfc00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 shutdown_connections 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8df00a1590 0x7f8df00a3a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e0810b4e0 0x7f8e0819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e081020b0 0x7f8e081991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 --2- 172.21.15.31:0/3661220577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e081029d0 0x7f8e08199700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 >> 172.21.15.31:0/3661220577 conn(0x7f8e080fb920 msgr2=0x7f8e0810d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 shutdown_connections 2023-10-06T21:19:40.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:40.864+0000 7f8e107b6700 1 -- 172.21.15.31:0/3661220577 wait complete. 2023-10-06T21:19:41.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:40 smithi031 ceph-mon[98539]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-06T21:19:41.112 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1099312753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:19:41.112 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1542364715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:19:41.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:40 smithi142 ceph-mon[102951]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-06T21:19:41.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:40 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1099312753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:19:41.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:40 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1542364715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:19:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 -- 172.21.15.31:0/2727553762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b01062b0 msgr2=0x7f45b010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2727553762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b01062b0 0x7f45b010de30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45a800b0a0 tx=0x7f45a800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 -- 172.21.15.31:0/2727553762 shutdown_connections 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2727553762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b01062b0 0x7f45b010de30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2727553762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b0068c00 0x7f45b0068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2727553762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b00695b0 0x7f45b0105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 -- 172.21.15.31:0/2727553762 >> 172.21.15.31:0/2727553762 conn(0x7f45b0075e00 msgr2=0x7f45b0076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.268+0000 7f45b55ab700 1 -- 172.21.15.31:0/2727553762 shutdown_connections 2023-10-06T21:19:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.269+0000 7f45b55ab700 1 -- 172.21.15.31:0/2727553762 wait complete. 2023-10-06T21:19:41.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.270+0000 7f45b55ab700 1 Processor -- start 2023-10-06T21:19:41.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.270+0000 7f45b55ab700 1 -- start start 2023-10-06T21:19:41.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:41.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b01062b0 0x7f45b01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a54b0 con 0x7f45b00695b0 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a5620 con 0x7f45b0068c00 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45b55ab700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a57e0 con 0x7f45b01062b0 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43840/0 (socket says 172.21.15.31:43840) 2023-10-06T21:19:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43638/0 (socket says 172.21.15.31:43638) 2023-10-06T21:19:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45aeffd700 1 -- 172.21.15.31:0/2028381290 learned_addr learned my addr 172.21.15.31:0/2028381290 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.271+0000 7f45af7fe700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b01062b0 0x7f45b01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 -- 172.21.15.31:0/2028381290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b01062b0 msgr2=0x7f45b01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b01062b0 0x7f45b01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 -- 172.21.15.31:0/2028381290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 msgr2=0x7f45b00fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 -- 172.21.15.31:0/2028381290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009d00 con 0x7f45b00695b0 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45aeffd700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f45ae7fc700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45b006a6b0 tx=0x7f45a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:41.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.272+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400f890 con 0x7f45b00695b0 2023-10-06T21:19:41.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.273+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45a400fe80 con 0x7f45b00695b0 2023-10-06T21:19:41.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.273+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b01a5ac0 con 0x7f45b00695b0 2023-10-06T21:19:41.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.273+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4019360 con 0x7f45b00695b0 2023-10-06T21:19:41.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.273+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b01a6010 con 0x7f45b00695b0 2023-10-06T21:19:41.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411397 for osd.2 2023-10-06T21:19:41.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.274+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b004ebb0 con 0x7f45b00695b0 2023-10-06T21:19:41.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.276+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f45a400f9f0 con 0x7f45b00695b0 2023-10-06T21:19:41.280 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:41.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.278+0000 7f4597fff700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 0x7f45980a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:41.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.278+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f45a40ccaa0 con 0x7f45b00695b0 2023-10-06T21:19:41.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.278+0000 7f45aeffd700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 0x7f45980a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:41.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.279+0000 7f45aeffd700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 0x7f45980a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005950 tx=0x7f45a000b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.282+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f45a4095f80 con 0x7f45b00695b0 2023-10-06T21:19:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:40 smithi146 ceph-mon[102809]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-06T21:19:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:40 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1099312753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:19:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:40 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1542364715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:19:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.504+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f45b0110e50 con 0x7f45b00695b0 2023-10-06T21:19:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.504+0000 7f4597fff700 1 -- 172.21.15.31:0/2028381290 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45a4016090 con 0x7f45b00695b0 2023-10-06T21:19:41.507 INFO:teuthology.orchestra.run.smithi031.stdout:111669149761 2023-10-06T21:19:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 msgr2=0x7f45980a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 0x7f45980a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005950 tx=0x7f45a000b500 comp rx=0 tx=0).stop 2023-10-06T21:19:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 msgr2=0x7f45b0100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45b006a6b0 tx=0x7f45a400dc30 comp rx=0 tx=0).stop 2023-10-06T21:19:41.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 shutdown_connections 2023-10-06T21:19:41.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f45980a1590 0x7f45980a3a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45b01062b0 0x7f45b01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45b0068c00 0x7f45b00fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 --2- 172.21.15.31:0/2028381290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b00695b0 0x7f45b0100470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:41.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 >> 172.21.15.31:0/2028381290 conn(0x7f45b0075e00 msgr2=0x7f45b00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:41.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 shutdown_connections 2023-10-06T21:19:41.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:41.507+0000 7f45b55ab700 1 -- 172.21.15.31:0/2028381290 wait complete. 2023-10-06T21:19:41.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:41 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3661220577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:19:41.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:41 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2028381290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:19:41.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724595 got 167503724598 for osd.5 2023-10-06T21:19:41.885 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:41.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888122 for osd.4 2023-10-06T21:19:41.896 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:42.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:41 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3661220577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:19:42.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:41 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2028381290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 -- 172.21.15.31:0/4121253736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 msgr2=0x7f08f410d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 --2- 172.21.15.31:0/4121253736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f410d3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009990 tx=0x7f08e8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 -- 172.21.15.31:0/4121253736 shutdown_connections 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 --2- 172.21.15.31:0/4121253736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 0x7f08f410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 --2- 172.21.15.31:0/4121253736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 0x7f08f40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 --2- 172.21.15.31:0/4121253736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f410d3f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 -- 172.21.15.31:0/4121253736 >> 172.21.15.31:0/4121253736 conn(0x7f08f40780a0 msgr2=0x7f08f40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:42.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 -- 172.21.15.31:0/4121253736 shutdown_connections 2023-10-06T21:19:42.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.206+0000 7f08f8f64700 1 -- 172.21.15.31:0/4121253736 wait complete. 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 Processor -- start 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 -- start start 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 0x7f08f4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 0x7f08f419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f419e0e0 con 0x7f08f4073a00 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f419e250 con 0x7f08f410d930 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f8f64700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f419e410 con 0x7f08f40730e0 2023-10-06T21:19:42.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43650/0 (socket says 172.21.15.31:43650) 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.208+0000 7f08f1d9b700 1 -- 172.21.15.31:0/1511151121 learned_addr learned my addr 172.21.15.31:0/1511151121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f259c700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 0x7f08f4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f2d9d700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 0x7f08f419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 -- 172.21.15.31:0/1511151121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 msgr2=0x7f08f4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 0x7f08f4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 -- 172.21.15.31:0/1511151121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 msgr2=0x7f08f419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 0x7f08f419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 -- 172.21.15.31:0/1511151121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80095f0 con 0x7f08f4073a00 2023-10-06T21:19:42.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08f1d9b700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005910 tx=0x7f08e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.209+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e801d070 con 0x7f08f4073a00 2023-10-06T21:19:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.210+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f419e690 con 0x7f08f4073a00 2023-10-06T21:19:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.210+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08e800fbb0 con 0x7f08f4073a00 2023-10-06T21:19:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.210+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8017650 con 0x7f08f4073a00 2023-10-06T21:19:42.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.210+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f41a5f50 con 0x7f08f4073a00 2023-10-06T21:19:42.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.211+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f4108250 con 0x7f08f4073a00 2023-10-06T21:19:42.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.213+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f08e8021410 con 0x7f08f4073a00 2023-10-06T21:19:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.215+0000 7f08e37fe700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 0x7f08dc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.215+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f08e80ce080 con 0x7f08f4073a00 2023-10-06T21:19:42.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.215+0000 7f08f259c700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 0x7f08dc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:42.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.216+0000 7f08f259c700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 0x7f08dc0a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009bf0 tx=0x7f08e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:42.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.219+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08e8097590 con 0x7f08f4073a00 2023-10-06T21:19:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:41 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3661220577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:19:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:41 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2028381290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:19:42.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.436+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f08f404ebb0 con 0x7f08f4073a00 2023-10-06T21:19:42.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.436+0000 7f08e37fe700 1 -- 172.21.15.31:0/1511151121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08e8021e60 con 0x7f08f4073a00 2023-10-06T21:19:42.442 INFO:teuthology.orchestra.run.smithi031.stdout:223338299435 2023-10-06T21:19:42.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 msgr2=0x7f08dc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:42.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 0x7f08dc0a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009bf0 tx=0x7f08e4009260 comp rx=0 tx=0).stop 2023-10-06T21:19:42.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 msgr2=0x7f08f4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005910 tx=0x7f08e80047b0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 shutdown_connections 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08dc0a1510 0x7f08dc0a39d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08f410d930 0x7f08f419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f40730e0 0x7f08f4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 --2- 172.21.15.31:0/1511151121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f4073a00 0x7f08f4199660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.438+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 >> 172.21.15.31:0/1511151121 conn(0x7f08f40780a0 msgr2=0x7f08f4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.439+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 shutdown_connections 2023-10-06T21:19:42.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:42.439+0000 7f08f8f64700 1 -- 172.21.15.31:0/1511151121 wait complete. 2023-10-06T21:19:42.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2023-10-06T21:19:42.865 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:43.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892350998 for osd.11 2023-10-06T21:19:43.009 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:43.106 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:42 smithi031 ceph-mon[98539]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-06T21:19:43.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1511151121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:19:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:42 smithi142 ceph-mon[102951]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-06T21:19:43.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1511151121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:19:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:42 smithi146 ceph-mon[102809]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-06T21:19:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1511151121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:19:43.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 -- 172.21.15.31:0/555617702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c073af0 msgr2=0x7fe78c10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 --2- 172.21.15.31:0/555617702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c073af0 0x7fe78c10d2a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe780009990 tx=0x7fe780009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 -- 172.21.15.31:0/555617702 shutdown_connections 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 --2- 172.21.15.31:0/555617702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c10d7e0 0x7fe78c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 --2- 172.21.15.31:0/555617702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c0731d0 0x7fe78c0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 --2- 172.21.15.31:0/555617702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c073af0 0x7fe78c10d2a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 -- 172.21.15.31:0/555617702 >> 172.21.15.31:0/555617702 conn(0x7fe78c0fc9d0 msgr2=0x7fe78c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 -- 172.21.15.31:0/555617702 shutdown_connections 2023-10-06T21:19:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.484+0000 7fe792083700 1 -- 172.21.15.31:0/555617702 wait complete. 2023-10-06T21:19:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.485+0000 7fe792083700 1 Processor -- start 2023-10-06T21:19:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.485+0000 7fe792083700 1 -- start start 2023-10-06T21:19:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c073af0 0x7fe78c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c10d7e0 0x7fe78c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c19e1d0 con 0x7fe78c0731d0 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c19e340 con 0x7fe78c10d7e0 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe792083700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c19e500 con 0x7fe78c073af0 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42048/0 (socket says 172.21.15.31:42048) 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe78b7fe700 1 -- 172.21.15.31:0/4254956754 learned_addr learned my addr 172.21.15.31:0/4254956754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe78bfff700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c10d7e0 0x7fe78c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.486+0000 7fe78affd700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c073af0 0x7fe78c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 -- 172.21.15.31:0/4254956754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c073af0 msgr2=0x7fe78c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c073af0 0x7fe78c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 -- 172.21.15.31:0/4254956754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c10d7e0 msgr2=0x7fe78c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c10d7e0 0x7fe78c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 -- 172.21.15.31:0/4254956754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800095f0 con 0x7fe78c0731d0 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe78b7fe700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00d950 tx=0x7fe77c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c009940 con 0x7fe78c0731d0 2023-10-06T21:19:43.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe77c010460 con 0x7fe78c0731d0 2023-10-06T21:19:43.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00f620 con 0x7fe78c0731d0 2023-10-06T21:19:43.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c19e7e0 con 0x7fe78c0731d0 2023-10-06T21:19:43.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.487+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c1a6040 con 0x7fe78c0731d0 2023-10-06T21:19:43.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.491+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c04ebb0 con 0x7fe78c0731d0 2023-10-06T21:19:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.491+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe77c0105d0 con 0x7fe78c0731d0 2023-10-06T21:19:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.493+0000 7fe788ff9700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 0x7fe7740a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.493+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe77c0cccf0 con 0x7fe78c0731d0 2023-10-06T21:19:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.493+0000 7fe78affd700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 0x7fe7740a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.494+0000 7fe78affd700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 0x7fe7740a3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b5c0 tx=0x7fe780005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.498+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe77c096170 con 0x7fe78c0731d0 2023-10-06T21:19:43.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299432 got 223338299435 for osd.7 2023-10-06T21:19:43.628 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 -- 172.21.15.31:0/255332454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4068ee0 msgr2=0x7f75d4069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 --2- 172.21.15.31:0/255332454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4068ee0 0x7f75d4069360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f75c80098f0 tx=0x7f75c8009c00 comp rx=0 tx=0).stop 2023-10-06T21:19:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 -- 172.21.15.31:0/255332454 shutdown_connections 2023-10-06T21:19:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 --2- 172.21.15.31:0/255332454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4103960 0x7f75d4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 --2- 172.21.15.31:0/255332454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4105ef0 0x7f75d40689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 --2- 172.21.15.31:0/255332454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4068ee0 0x7f75d4069360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 -- 172.21.15.31:0/255332454 >> 172.21.15.31:0/255332454 conn(0x7f75d4075e00 msgr2=0x7f75d4076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 -- 172.21.15.31:0/255332454 shutdown_connections 2023-10-06T21:19:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.741+0000 7f75da629700 1 -- 172.21.15.31:0/255332454 wait complete. 2023-10-06T21:19:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.742+0000 7f75da629700 1 Processor -- start 2023-10-06T21:19:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.743+0000 7f75da629700 1 -- start start 2023-10-06T21:19:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.743+0000 7f75da629700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 0x7f75d419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.743+0000 7f75da629700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.743+0000 7f75da629700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4105ef0 0x7f75d41a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75da629700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41a2420 con 0x7f75d4105ef0 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75da629700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41a2590 con 0x7f75d4103960 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75da629700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41a2750 con 0x7f75d4068ee0 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d8bc6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4105ef0 0x7f75d41a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57496/0 (socket says 172.21.15.31:57496) 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 -- 172.21.15.31:0/2944351696 learned_addr learned my addr 172.21.15.31:0/2944351696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d3fff700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 0x7f75d419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 -- 172.21.15.31:0/2944351696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 msgr2=0x7f75d419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 0x7f75d419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 -- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4105ef0 msgr2=0x7f75d41a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4105ef0 0x7f75d41a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d37fe700 1 -- 172.21.15.31:0/2944351696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009520 con 0x7f75d4103960 2023-10-06T21:19:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.744+0000 7f75d3fff700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 0x7f75d419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75d37fe700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009230 tx=0x7f75c800bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c801c070 con 0x7f75d4103960 2023-10-06T21:19:43.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75c80058e0 con 0x7f75d4103960 2023-10-06T21:19:43.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c80038e0 con 0x7f75d4103960 2023-10-06T21:19:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d41a29d0 con 0x7f75d4103960 2023-10-06T21:19:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.745+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d41aa290 con 0x7f75d4103960 2023-10-06T21:19:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.747+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d404ebb0 con 0x7f75d4103960 2023-10-06T21:19:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.748+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f75c8020030 con 0x7f75d4103960 2023-10-06T21:19:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.750+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe78c112f70 con 0x7fe78c0731d0 2023-10-06T21:19:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.750+0000 7fe788ff9700 1 -- 172.21.15.31:0/4254956754 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe77c016090 con 0x7fe78c0731d0 2023-10-06T21:19:43.755 INFO:teuthology.orchestra.run.smithi031.stdout:313532612637 2023-10-06T21:19:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.751+0000 7f75d17fa700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 0x7f75bc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.751+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f75c80cdcf0 con 0x7f75d4103960 2023-10-06T21:19:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.751+0000 7f75d3fff700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 0x7f75bc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.752+0000 7f75d3fff700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 0x7f75bc0a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005fd0 tx=0x7f75c4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 msgr2=0x7fe7740a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 0x7fe7740a3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b5c0 tx=0x7fe780005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 msgr2=0x7fe78c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00d950 tx=0x7fe77c00dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 shutdown_connections 2023-10-06T21:19:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe7740a1610 0x7fe7740a3ad0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c10d7e0 0x7fe78c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe78c0731d0 0x7fe78c199210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 --2- 172.21.15.31:0/4254956754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe78c073af0 0x7fe78c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 >> 172.21.15.31:0/4254956754 conn(0x7fe78c0fc9d0 msgr2=0x7fe78c10f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 shutdown_connections 2023-10-06T21:19:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.753+0000 7fe792083700 1 -- 172.21.15.31:0/4254956754 wait complete. 2023-10-06T21:19:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.755+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f75c8097200 con 0x7f75d4103960 2023-10-06T21:19:43.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.995+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f75d419e7c0 con 0x7f75d4103960 2023-10-06T21:19:43.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.996+0000 7f75d17fa700 1 -- 172.21.15.31:0/2944351696 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75c809a820 con 0x7f75d4103960 2023-10-06T21:19:43.999 INFO:teuthology.orchestra.run.smithi031.stdout:201863462961 2023-10-06T21:19:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 msgr2=0x7f75bc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 0x7f75bc0a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005fd0 tx=0x7f75c4009310 comp rx=0 tx=0).stop 2023-10-06T21:19:44.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 msgr2=0x7f75d419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:44.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009230 tx=0x7f75c800bf00 comp rx=0 tx=0).stop 2023-10-06T21:19:44.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 shutdown_connections 2023-10-06T21:19:44.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75bc0a1590 0x7f75bc0a3a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:44.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d4105ef0 0x7f75d41a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:44.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d4068ee0 0x7f75d419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:44.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 --2- 172.21.15.31:0/2944351696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4103960 0x7f75d419d9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:44.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.998+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 >> 172.21.15.31:0/2944351696 conn(0x7f75d4075e00 msgr2=0x7f75d40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:44.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.999+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 shutdown_connections 2023-10-06T21:19:44.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:43.999+0000 7f75da629700 1 -- 172.21.15.31:0/2944351696 wait complete. 2023-10-06T21:19:44.025 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:43 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4254956754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:19:44.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:43 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4254956754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:19:44.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149761 for osd.3 2023-10-06T21:19:44.162 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:43 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4254956754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:19:44.916 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:44 smithi031 ceph-mon[98539]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:19:44.916 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:44 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2944351696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:19:45.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462958 got 201863462961 for osd.6 2023-10-06T21:19:45.079 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:45.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:44 smithi142 ceph-mon[102951]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:19:45.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:44 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2944351696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:19:45.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612634 got 313532612637 for osd.10 2023-10-06T21:19:45.209 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:45.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:44 smithi146 ceph-mon[102809]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:19:45.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:44 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2944351696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:19:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 -- 172.21.15.31:0/3684225301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c073620 msgr2=0x7f641c073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 --2- 172.21.15.31:0/3684225301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c073620 0x7f641c073a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6408009990 tx=0x7f6408009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 -- 172.21.15.31:0/3684225301 shutdown_connections 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 --2- 172.21.15.31:0/3684225301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c111bc0 0x7f641c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 --2- 172.21.15.31:0/3684225301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c074ef0 0x7f641c0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 --2- 172.21.15.31:0/3684225301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c073620 0x7f641c073a80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 -- 172.21.15.31:0/3684225301 >> 172.21.15.31:0/3684225301 conn(0x7f641c0780a0 msgr2=0x7f641c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 -- 172.21.15.31:0/3684225301 shutdown_connections 2023-10-06T21:19:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.309+0000 7f6422fb3700 1 -- 172.21.15.31:0/3684225301 wait complete. 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.311+0000 7f6422fb3700 1 Processor -- start 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.311+0000 7f6422fb3700 1 -- start start 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.311+0000 7f6422fb3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c073620 0x7f641c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.311+0000 7f6422fb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6422fb3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c111bc0 0x7f641c1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6422fb3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c1a25e0 con 0x7f641c074ef0 2023-10-06T21:19:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6422fb3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c1a2750 con 0x7f641c073620 2023-10-06T21:19:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6422fb3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c1a2910 con 0x7f641c111bc0 2023-10-06T21:19:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42076/0 (socket says 172.21.15.31:42076) 2023-10-06T21:19:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 -- 172.21.15.31:0/1194764757 learned_addr learned my addr 172.21.15.31:0/1194764757 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6420d4f700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c073620 0x7f641c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f6421550700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c111bc0 0x7f641c1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 -- 172.21.15.31:0/1194764757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c111bc0 msgr2=0x7f641c1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c111bc0 0x7f641c1a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 -- 172.21.15.31:0/1194764757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c073620 msgr2=0x7f641c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c073620 0x7f641c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.312+0000 7f641bfff700 1 -- 172.21.15.31:0/1194764757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080095f0 con 0x7f641c074ef0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f641bfff700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f64080036c0 tx=0x7f6408005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640801d070 con 0x7f641c074ef0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6408003ec0 con 0x7f641c074ef0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408022b90 con 0x7f641c074ef0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c1a2b90 con 0x7f641c074ef0 2023-10-06T21:19:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.313+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c1aa280 con 0x7f641c074ef0 2023-10-06T21:19:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.314+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c04ebb0 con 0x7f641c074ef0 2023-10-06T21:19:46.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.317+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6408004030 con 0x7f641c074ef0 2023-10-06T21:19:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.318+0000 7f6419ffb700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 0x7f640c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.319+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f64080ce430 con 0x7f641c074ef0 2023-10-06T21:19:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.319+0000 7f6420d4f700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 0x7f640c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.319+0000 7f6420d4f700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 0x7f640c0a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6410007be0 tx=0x7f64100073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.323+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6408097940 con 0x7f641c074ef0 2023-10-06T21:19:46.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.546+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f641c19e980 con 0x7f641c074ef0 2023-10-06T21:19:46.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.547+0000 7f6419ffb700 1 -- 172.21.15.31:0/1194764757 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f640809af60 con 0x7f641c074ef0 2023-10-06T21:19:46.550 INFO:teuthology.orchestra.run.smithi031.stdout:257698037799 2023-10-06T21:19:46.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 msgr2=0x7f640c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 0x7f640c0a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6410007be0 tx=0x7f64100073d0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 msgr2=0x7f641c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f64080036c0 tx=0x7f6408005740 comp rx=0 tx=0).stop 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 shutdown_connections 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f640c0a1590 0x7f640c0a3a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f641c111bc0 0x7f641c1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f641c073620 0x7f641c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 --2- 172.21.15.31:0/1194764757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f641c074ef0 0x7f641c19db60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.549+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 >> 172.21.15.31:0/1194764757 conn(0x7f641c0780a0 msgr2=0x7f641c110760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.550+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 shutdown_connections 2023-10-06T21:19:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.550+0000 7f6422fb3700 1 -- 172.21.15.31:0/1194764757 wait complete. 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 -- 172.21.15.31:0/2438291685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4111bc0 msgr2=0x7fe4e4113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 --2- 172.21.15.31:0/2438291685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4111bc0 0x7fe4e4113fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b0a0 tx=0x7fe4dc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 -- 172.21.15.31:0/2438291685 shutdown_connections 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 --2- 172.21.15.31:0/2438291685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4111bc0 0x7fe4e4113fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 --2- 172.21.15.31:0/2438291685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4074ef0 0x7fe4e40730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 --2- 172.21.15.31:0/2438291685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4073620 0x7fe4e4073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 -- 172.21.15.31:0/2438291685 >> 172.21.15.31:0/2438291685 conn(0x7fe4e40780a0 msgr2=0x7fe4e40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:46.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 -- 172.21.15.31:0/2438291685 shutdown_connections 2023-10-06T21:19:46.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.599+0000 7fe4eb323700 1 -- 172.21.15.31:0/2438291685 wait complete. 2023-10-06T21:19:46.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.600+0000 7fe4eb323700 1 Processor -- start 2023-10-06T21:19:46.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.600+0000 7fe4eb323700 1 -- start start 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 0x7fe4e419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4111bc0 0x7fe4e41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a15e0 con 0x7fe4e4074ef0 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a1750 con 0x7fe4e4073620 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4eb323700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a1910 con 0x7fe4e4111bc0 2023-10-06T21:19:46.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4e88be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4e88be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42088/0 (socket says 172.21.15.31:42088) 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4e88be700 1 -- 172.21.15.31:0/3589349177 learned_addr learned my addr 172.21.15.31:0/3589349177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4e90bf700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 0x7fe4e419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.601+0000 7fe4e98c0700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4111bc0 0x7fe4e41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 -- 172.21.15.31:0/3589349177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4111bc0 msgr2=0x7fe4e41a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4111bc0 0x7fe4e41a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 -- 172.21.15.31:0/3589349177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 msgr2=0x7fe4e419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 0x7fe4e419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 -- 172.21.15.31:0/3589349177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0009520 con 0x7fe4e4074ef0 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e90bf700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 0x7fe4e419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:46.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4e88be700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d950 tx=0x7fe4d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:46.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400f890 con 0x7fe4e4074ef0 2023-10-06T21:19:46.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc009d00 con 0x7fe4e4074ef0 2023-10-06T21:19:46.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d400fe80 con 0x7fe4e4074ef0 2023-10-06T21:19:46.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400e3b0 con 0x7fe4e4074ef0 2023-10-06T21:19:46.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.602+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e41aa370 con 0x7fe4e4074ef0 2023-10-06T21:19:46.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.603+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e404ebb0 con 0x7fe4e4074ef0 2023-10-06T21:19:46.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.606+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe4d4010460 con 0x7fe4e4074ef0 2023-10-06T21:19:46.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.607+0000 7fe4da7fc700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 0x7fe4d00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:46.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.607+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe4d40ccbb0 con 0x7fe4e4074ef0 2023-10-06T21:19:46.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.607+0000 7fe4e90bf700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 0x7fe4d00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:46.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.608+0000 7fe4e90bf700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 0x7fe4d00a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0011630 tx=0x7fe4e0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.610+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4d4096090 con 0x7fe4e4074ef0 2023-10-06T21:19:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.826+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe4e419ea40 con 0x7fe4e4074ef0 2023-10-06T21:19:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.826+0000 7fe4da7fc700 1 -- 172.21.15.31:0/3589349177 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4d4020020 con 0x7fe4e4074ef0 2023-10-06T21:19:46.829 INFO:teuthology.orchestra.run.smithi031.stdout:292057776161 2023-10-06T21:19:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 msgr2=0x7fe4d00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 0x7fe4d00a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0011630 tx=0x7fe4e0009260 comp rx=0 tx=0).stop 2023-10-06T21:19:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 msgr2=0x7fe4e419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d950 tx=0x7fe4d400dc60 comp rx=0 tx=0).stop 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 shutdown_connections 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fe4d00a1590 0x7fe4d00a3a50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e4111bc0 0x7fe4e41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4e4073620 0x7fe4e419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 --2- 172.21.15.31:0/3589349177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4074ef0 0x7fe4e419dc20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 >> 172.21.15.31:0/3589349177 conn(0x7fe4e40780a0 msgr2=0x7fe4e4110760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 shutdown_connections 2023-10-06T21:19:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:46.829+0000 7fe4eb323700 1 -- 172.21.15.31:0/3589349177 wait complete. 2023-10-06T21:19:46.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:46 smithi031 ceph-mon[98539]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:46.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:46 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1194764757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:19:46.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:46 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3589349177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:19:47.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037799 for osd.8 2023-10-06T21:19:47.049 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:46 smithi142 ceph-mon[102951]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:46 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1194764757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:19:47.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:46 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3589349177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:19:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:46 smithi146 ceph-mon[102809]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:46 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1194764757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:19:47.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:46 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3589349177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:19:47.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776158 got 292057776161 for osd.9 2023-10-06T21:19:47.344 DEBUG:teuthology.parallel:result is None 2023-10-06T21:19:47.344 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-06T21:19:47.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T21:19:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:49.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:48 smithi031 ceph-mon[98539]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-06T21:19:49.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:48 smithi142 ceph-mon[102951]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-06T21:19:49.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:48 smithi146 ceph-mon[102809]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-06T21:19:49.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 -- 172.21.15.31:0/3841904871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 msgr2=0x7fd848107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/3841904871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848107370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0099c0 tx=0x7fd83c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 -- 172.21.15.31:0/3841904871 shutdown_connections 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/3841904871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd84810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/3841904871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848107370 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/3841904871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 0x7fd848101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 -- 172.21.15.31:0/3841904871 >> 172.21.15.31:0/3841904871 conn(0x7fd8480780a0 msgr2=0x7fd8480784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 -- 172.21.15.31:0/3841904871 shutdown_connections 2023-10-06T21:19:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.589+0000 7fd84ecaf700 1 -- 172.21.15.31:0/3841904871 wait complete. 2023-10-06T21:19:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.590+0000 7fd84ecaf700 1 Processor -- start 2023-10-06T21:19:49.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.590+0000 7fd84ecaf700 1 -- start start 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 0x7fd8480731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd8480736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a5360 con 0x7fd848106f90 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a54d0 con 0x7fd848100f80 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ecaf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a5690 con 0x7fd848101940 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84d24c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84d24c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42106/0 (socket says 172.21.15.31:42106) 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84d24c700 1 -- 172.21.15.31:0/1422894228 learned_addr learned my addr 172.21.15.31:0/1422894228 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd84ca4b700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 0x7fd8480731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:49.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.591+0000 7fd847fff700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd8480736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 -- 172.21.15.31:0/1422894228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 msgr2=0x7fd8480736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd8480736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 -- 172.21.15.31:0/1422894228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 msgr2=0x7fd8480731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 0x7fd8480731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 -- 172.21.15.31:0/1422894228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0095f0 con 0x7fd848106f90 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd847fff700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd8480736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84d24c700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd840007f10 tx=0x7fd84000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840017070 con 0x7fd848106f90 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd84000f040 con 0x7fd848106f90 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840005710 con 0x7fd848106f90 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8481a5970 con 0x7fd848106f90 2023-10-06T21:19:49.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.592+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8481a5e20 con 0x7fd848106f90 2023-10-06T21:19:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.593+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84804ebb0 con 0x7fd848106f90 2023-10-06T21:19:49.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.595+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd840005870 con 0x7fd848106f90 2023-10-06T21:19:49.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.596+0000 7fd845ffb700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 0x7fd8380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:49.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.597+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd8400cc600 con 0x7fd848106f90 2023-10-06T21:19:49.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.597+0000 7fd84ca4b700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 0x7fd8380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:49.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.597+0000 7fd84ca4b700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 0x7fd8380a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00b580 tx=0x7fd83c009f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:49.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.601+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd840095a80 con 0x7fd848106f90 2023-10-06T21:19:49.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.806+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd848110f50 con 0x7fd8380a1510 2023-10-06T21:19:49.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.818+0000 7fd845ffb700 1 -- 172.21.15.31:0/1422894228 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340131 (secure 0 0 0) 0x7fd848110f50 con 0x7fd8380a1510 2023-10-06T21:19:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:49.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 msgr2=0x7fd8380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:49.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 0x7fd8380a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00b580 tx=0x7fd83c009f10 comp rx=0 tx=0).stop 2023-10-06T21:19:49.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 msgr2=0x7fd848073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:49.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd840007f10 tx=0x7fd84000d3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 shutdown_connections 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd8380a1510 0x7fd8380a39d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd848106f90 0x7fd848073c70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848100f80 0x7fd8480731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 --2- 172.21.15.31:0/1422894228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd848101940 0x7fd8480736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 >> 172.21.15.31:0/1422894228 conn(0x7fd8480780a0 msgr2=0x7fd8480ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 shutdown_connections 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:49.823+0000 7fd84ecaf700 1 -- 172.21.15.31:0/1422894228 wait complete. 2023-10-06T21:19:49.832 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:19:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-10-06T21:19:47.823680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656407,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":629,"num_read_kb":440,"num_write":417,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336052,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605772,"statfs":{"total":1151940427776,"available":1151596310528,"internally_reserved":0,"allocated":12062720,"data_stored":7475259,"data_compressed":182031,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":69,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841866+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:19:25.841866+0000","last_peered":"2023-10-06T21:19:25.841866+0000","last_clean":"2023-10-06T21:19:25.841866+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:19:25.841866+0000","last_undegraded":"2023-10-06T21:19:25.841866+0000","last_fullsized":"2023-10-06T21:19:25.841866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832672+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:19:25.832672+0000","last_peered":"2023-10-06T21:19:25.832672+0000","last_clean":"2023-10-06T21:19:25.832672+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:19:25.832672+0000","last_undegraded":"2023-10-06T21:19:25.832672+0000","last_fullsized":"2023-10-06T21:19:25.832672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832409+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:19:25.832409+0000","last_peered":"2023-10-06T21:19:25.832409+0000","last_clean":"2023-10-06T21:19:25.832409+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:19:25.832409+0000","last_undegraded":"2023-10-06T21:19:25.832409+0000","last_fullsized":"2023-10-06T21:19:25.832409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835657+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:19:25.835657+0000","last_peered":"2023-10-06T21:19:25.835657+0000","last_clean":"2023-10-06T21:19:25.835657+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:19:25.835657+0000","last_undegraded":"2023-10-06T21:19:25.835657+0000","last_fullsized":"2023-10-06T21:19:25.835657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838062+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:19:25.838062+0000","last_peered":"2023-10-06T21:19:25.838062+0000","last_clean":"2023-10-06T21:19:25.838062+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:19:25.838062+0000","last_undegraded":"2023-10-06T21:19:25.838062+0000","last_fullsized":"2023-10-06T21:19:25.838062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841748+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:19:25.841748+0000","last_peered":"2023-10-06T21:19:25.841748+0000","last_clean":"2023-10-06T21:19:25.841748+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:19:25.841748+0000","last_undegraded":"2023-10-06T21:19:25.841748+0000","last_fullsized":"2023-10-06T21:19:25.841748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832621+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:19:25.832621+0000","last_peered":"2023-10-06T21:19:25.832621+0000","last_clean":"2023-10-06T21:19:25.832621+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:19:25.832621+0000","last_undegraded":"2023-10-06T21:19:25.832621+0000","last_fullsized":"2023-10-06T21:19:25.832621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835282+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:19:25.835282+0000","last_peered":"2023-10-06T21:19:25.835282+0000","last_clean":"2023-10-06T21:19:25.835282+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:19:25.835282+0000","last_undegraded":"2023-10-06T21:19:25.835282+0000","last_fullsized":"2023-10-06T21:19:25.835282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843808+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:19:25.843808+0000","last_peered":"2023-10-06T21:19:25.843808+0000","last_clean":"2023-10-06T21:19:25.843808+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:19:25.843808+0000","last_undegraded":"2023-10-06T21:19:25.843808+0000","last_fullsized":"2023-10-06T21:19:25.843808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841122+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:19:25.841122+0000","last_peered":"2023-10-06T21:19:25.841122+0000","last_clean":"2023-10-06T21:19:25.841122+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:19:25.841122+0000","last_undegraded":"2023-10-06T21:19:25.841122+0000","last_fullsized":"2023-10-06T21:19:25.841122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833138+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:19:25.833138+0000","last_peered":"2023-10-06T21:19:25.833138+0000","last_clean":"2023-10-06T21:19:25.833138+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:19:25.833138+0000","last_undegraded":"2023-10-06T21:19:25.833138+0000","last_fullsized":"2023-10-06T21:19:25.833138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835449+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:19:25.835449+0000","last_peered":"2023-10-06T21:19:25.835449+0000","last_clean":"2023-10-06T21:19:25.835449+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:19:25.835449+0000","last_undegraded":"2023-10-06T21:19:25.835449+0000","last_fullsized":"2023-10-06T21:19:25.835449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838913+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:19:25.838913+0000","last_peered":"2023-10-06T21:19:25.838913+0000","last_clean":"2023-10-06T21:19:25.838913+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:19:25.838913+0000","last_undegraded":"2023-10-06T21:19:25.838913+0000","last_fullsized":"2023-10-06T21:19:25.838913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832111+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:19:25.832111+0000","last_peered":"2023-10-06T21:19:25.832111+0000","last_clean":"2023-10-06T21:19:25.832111+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:19:25.832111+0000","last_undegraded":"2023-10-06T21:19:25.832111+0000","last_fullsized":"2023-10-06T21:19:25.832111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832065+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:19:25.832065+0000","last_peered":"2023-10-06T21:19:25.832065+0000","last_clean":"2023-10-06T21:19:25.832065+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:19:25.832065+0000","last_undegraded":"2023-10-06T21:19:25.832065+0000","last_fullsized":"2023-10-06T21:19:25.832065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843154+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:19:25.843154+0000","last_peered":"2023-10-06T21:19:25.843154+0000","last_clean":"2023-10-06T21:19:25.843154+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:19:25.843154+0000","last_undegraded":"2023-10-06T21:19:25.843154+0000","last_fullsized":"2023-10-06T21:19:25.843154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.842101+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:19:25.842101+0000","last_peered":"2023-10-06T21:19:25.842101+0000","last_clean":"2023-10-06T21:19:25.842101+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:19:25.842101+0000","last_undegraded":"2023-10-06T21:19:25.842101+0000","last_fullsized":"2023-10-06T21:19:25.842101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839761+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:19:25.839761+0000","last_peered":"2023-10-06T21:19:25.839761+0000","last_clean":"2023-10-06T21:19:25.839761+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:19:25.839761+0000","last_undegraded":"2023-10-06T21:19:25.839761+0000","last_fullsized":"2023-10-06T21:19:25.839761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840155+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:19:25.840155+0000","last_peered":"2023-10-06T21:19:25.840155+0000","last_clean":"2023-10-06T21:19:25.840155+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:19:25.840155+0000","last_undegraded":"2023-10-06T21:19:25.840155+0000","last_fullsized":"2023-10-06T21:19:25.840155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:45.932702+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:19:45.932702+0000","last_peered":"2023-10-06T21:19:45.932702+0000","last_clean":"2023-10-06T21:19:45.932702+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:19:45.932702+0000","last_undegraded":"2023-10-06T21:19:45.932702+0000","last_fullsized":"2023-10-06T21:19:45.932702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840953+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:19:25.840953+0000","last_peered":"2023-10-06T21:19:25.840953+0000","last_clean":"2023-10-06T21:19:25.840953+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:19:25.840953+0000","last_undegraded":"2023-10-06T21:19:25.840953+0000","last_fullsized":"2023-10-06T21:19:25.840953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841280+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:19:25.841280+0000","last_peered":"2023-10-06T21:19:25.841280+0000","last_clean":"2023-10-06T21:19:25.841280+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:19:25.841280+0000","last_undegraded":"2023-10-06T21:19:25.841280+0000","last_fullsized":"2023-10-06T21:19:25.841280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833139+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:19:25.833139+0000","last_peered":"2023-10-06T21:19:25.833139+0000","last_clean":"2023-10-06T21:19:25.833139+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:19:25.833139+0000","last_undegraded":"2023-10-06T21:19:25.833139+0000","last_fullsized":"2023-10-06T21:19:25.833139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.863512+0000","last_change":"2023-10-06T21:19:26.863512+0000","last_active":"2023-10-06T21:19:26.863512+0000","last_peered":"2023-10-06T21:19:26.863512+0000","last_clean":"2023-10-06T21:19:26.863512+0000","last_became_active":"2023-10-06T21:19:26.845274+0000","last_became_peered":"2023-10-06T21:19:26.845274+0000","last_unstale":"2023-10-06T21:19:26.863512+0000","last_undegraded":"2023-10-06T21:19:26.863512+0000","last_fullsized":"2023-10-06T21:19:26.863512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:16.751914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843466+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:19:25.843466+0000","last_peered":"2023-10-06T21:19:25.843466+0000","last_clean":"2023-10-06T21:19:25.843466+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:19:25.843466+0000","last_undegraded":"2023-10-06T21:19:25.843466+0000","last_fullsized":"2023-10-06T21:19:25.843466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838422+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:19:25.838422+0000","last_peered":"2023-10-06T21:19:25.838422+0000","last_clean":"2023-10-06T21:19:25.838422+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:19:25.838422+0000","last_undegraded":"2023-10-06T21:19:25.838422+0000","last_fullsized":"2023-10-06T21:19:25.838422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.991142+0000","last_change":"2023-10-06T21:19:26.991142+0000","last_active":"2023-10-06T21:19:26.991142+0000","last_peered":"2023-10-06T21:19:26.991142+0000","last_clean":"2023-10-06T21:19:26.991142+0000","last_became_active":"2023-10-06T21:19:26.990765+0000","last_became_peered":"2023-10-06T21:19:26.990765+0000","last_unstale":"2023-10-06T21:19:26.991142+0000","last_undegraded":"2023-10-06T21:19:26.991142+0000","last_fullsized":"2023-10-06T21:19:26.991142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:35:44.692174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:44.895284+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:19:44.895284+0000","last_peered":"2023-10-06T21:19:44.895284+0000","last_clean":"2023-10-06T21:19:44.895284+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:19:44.895284+0000","last_undegraded":"2023-10-06T21:19:44.895284+0000","last_fullsized":"2023-10-06T21:19:44.895284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837859+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:19:25.837859+0000","last_peered":"2023-10-06T21:19:25.837859+0000","last_clean":"2023-10-06T21:19:25.837859+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:19:25.837859+0000","last_undegraded":"2023-10-06T21:19:25.837859+0000","last_fullsized":"2023-10-06T21:19:25.837859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.998407+0000","last_change":"2023-10-06T21:19:26.998407+0000","last_active":"2023-10-06T21:19:26.998407+0000","last_peered":"2023-10-06T21:19:26.998407+0000","last_clean":"2023-10-06T21:19:26.998407+0000","last_became_active":"2023-10-06T21:19:26.991918+0000","last_became_peered":"2023-10-06T21:19:26.991918+0000","last_unstale":"2023-10-06T21:19:26.998407+0000","last_undegraded":"2023-10-06T21:19:26.998407+0000","last_fullsized":"2023-10-06T21:19:26.998407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:05.903796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.846902+0000","last_change":"2023-10-06T21:19:26.846902+0000","last_active":"2023-10-06T21:19:26.846902+0000","last_peered":"2023-10-06T21:19:26.846902+0000","last_clean":"2023-10-06T21:19:26.846902+0000","last_became_active":"2023-10-06T21:19:26.846491+0000","last_became_peered":"2023-10-06T21:19:26.846491+0000","last_unstale":"2023-10-06T21:19:26.846902+0000","last_undegraded":"2023-10-06T21:19:26.846902+0000","last_fullsized":"2023-10-06T21:19:26.846902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:18:06.115026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838901+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:19:25.838901+0000","last_peered":"2023-10-06T21:19:25.838901+0000","last_clean":"2023-10-06T21:19:25.838901+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:19:25.838901+0000","last_undegraded":"2023-10-06T21:19:25.838901+0000","last_fullsized":"2023-10-06T21:19:25.838901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835625+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:19:25.835625+0000","last_peered":"2023-10-06T21:19:25.835625+0000","last_clean":"2023-10-06T21:19:25.835625+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:19:25.835625+0000","last_undegraded":"2023-10-06T21:19:25.835625+0000","last_fullsized":"2023-10-06T21:19:25.835625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839531+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:19:25.839531+0000","last_peered":"2023-10-06T21:19:25.839531+0000","last_clean":"2023-10-06T21:19:25.839531+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:19:25.839531+0000","last_undegraded":"2023-10-06T21:19:25.839531+0000","last_fullsized":"2023-10-06T21:19:25.839531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839103+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:19:25.839103+0000","last_peered":"2023-10-06T21:19:25.839103+0000","last_clean":"2023-10-06T21:19:25.839103+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:19:25.839103+0000","last_undegraded":"2023-10-06T21:19:25.839103+0000","last_fullsized":"2023-10-06T21:19:25.839103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":112,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:45.896106+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:19:45.896106+0000","last_peered":"2023-10-06T21:19:45.896106+0000","last_clean":"2023-10-06T21:19:45.896106+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:19:45.896106+0000","last_undegraded":"2023-10-06T21:19:45.896106+0000","last_fullsized":"2023-10-06T21:19:45.896106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843810+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:19:25.843810+0000","last_peered":"2023-10-06T21:19:25.843810+0000","last_clean":"2023-10-06T21:19:25.843810+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:19:25.843810+0000","last_undegraded":"2023-10-06T21:19:25.843810+0000","last_fullsized":"2023-10-06T21:19:25.843810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833096+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:19:25.833096+0000","last_peered":"2023-10-06T21:19:25.833096+0000","last_clean":"2023-10-06T21:19:25.833096+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:19:25.833096+0000","last_undegraded":"2023-10-06T21:19:25.833096+0000","last_fullsized":"2023-10-06T21:19:25.833096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833053+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:19:25.833053+0000","last_peered":"2023-10-06T21:19:25.833053+0000","last_clean":"2023-10-06T21:19:25.833053+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:19:25.833053+0000","last_undegraded":"2023-10-06T21:19:25.833053+0000","last_fullsized":"2023-10-06T21:19:25.833053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843136+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:19:25.843136+0000","last_peered":"2023-10-06T21:19:25.843136+0000","last_clean":"2023-10-06T21:19:25.843136+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:19:25.843136+0000","last_undegraded":"2023-10-06T21:19:25.843136+0000","last_fullsized":"2023-10-06T21:19:25.843136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843582+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:19:25.843582+0000","last_peered":"2023-10-06T21:19:25.843582+0000","last_clean":"2023-10-06T21:19:25.843582+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:19:25.843582+0000","last_undegraded":"2023-10-06T21:19:25.843582+0000","last_fullsized":"2023-10-06T21:19:25.843582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:45.896099+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:19:45.896099+0000","last_peered":"2023-10-06T21:19:45.896099+0000","last_clean":"2023-10-06T21:19:45.896099+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:19:45.896099+0000","last_undegraded":"2023-10-06T21:19:45.896099+0000","last_fullsized":"2023-10-06T21:19:45.896099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58981,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":111,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837984+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:19:25.837984+0000","last_peered":"2023-10-06T21:19:25.837984+0000","last_clean":"2023-10-06T21:19:25.837984+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:19:25.837984+0000","last_undegraded":"2023-10-06T21:19:25.837984+0000","last_fullsized":"2023-10-06T21:19:25.837984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839795+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:19:25.839795+0000","last_peered":"2023-10-06T21:19:25.839795+0000","last_clean":"2023-10-06T21:19:25.839795+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:19:25.839795+0000","last_undegraded":"2023-10-06T21:19:25.839795+0000","last_fullsized":"2023-10-06T21:19:25.839795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838622+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:19:25.838622+0000","last_peered":"2023-10-06T21:19:25.838622+0000","last_clean":"2023-10-06T21:19:25.838622+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:19:25.838622+0000","last_undegraded":"2023-10-06T21:19:25.838622+0000","last_fullsized":"2023-10-06T21:19:25.838622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.842403+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:19:25.842403+0000","last_peered":"2023-10-06T21:19:25.842403+0000","last_clean":"2023-10-06T21:19:25.842403+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:19:25.842403+0000","last_undegraded":"2023-10-06T21:19:25.842403+0000","last_fullsized":"2023-10-06T21:19:25.842403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833007+0000","last_change":"2023-10-06T21:18:54.860485+0000","last_active":"2023-10-06T21:19:25.833007+0000","last_peered":"2023-10-06T21:19:25.833007+0000","last_clean":"2023-10-06T21:19:25.833007+0000","last_became_active":"2023-10-06T21:18:54.860171+0000","last_became_peered":"2023-10-06T21:18:54.860171+0000","last_unstale":"2023-10-06T21:19:25.833007+0000","last_undegraded":"2023-10-06T21:19:25.833007+0000","last_fullsized":"2023-10-06T21:19:25.833007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:46:50.890861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:44.930847+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:19:44.930847+0000","last_peered":"2023-10-06T21:19:44.930847+0000","last_clean":"2023-10-06T21:19:44.930847+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:19:44.930847+0000","last_undegraded":"2023-10-06T21:19:44.930847+0000","last_fullsized":"2023-10-06T21:19:44.930847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832537+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:19:25.832537+0000","last_peered":"2023-10-06T21:19:25.832537+0000","last_clean":"2023-10-06T21:19:25.832537+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:19:25.832537+0000","last_undegraded":"2023-10-06T21:19:25.832537+0000","last_fullsized":"2023-10-06T21:19:25.832537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.862636+0000","last_change":"2023-10-06T21:19:26.862156+0000","last_active":"2023-10-06T21:19:26.862636+0000","last_peered":"2023-10-06T21:19:26.862636+0000","last_clean":"2023-10-06T21:19:26.862636+0000","last_became_active":"2023-10-06T21:19:26.844987+0000","last_became_peered":"2023-10-06T21:19:26.844987+0000","last_unstale":"2023-10-06T21:19:26.862636+0000","last_undegraded":"2023-10-06T21:19:26.862636+0000","last_fullsized":"2023-10-06T21:19:26.862636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:12:56.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840419+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:19:25.840419+0000","last_peered":"2023-10-06T21:19:25.840419+0000","last_clean":"2023-10-06T21:19:25.840419+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:19:25.840419+0000","last_undegraded":"2023-10-06T21:19:25.840419+0000","last_fullsized":"2023-10-06T21:19:25.840419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838617+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:19:25.838617+0000","last_peered":"2023-10-06T21:19:25.838617+0000","last_clean":"2023-10-06T21:19:25.838617+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:19:25.838617+0000","last_undegraded":"2023-10-06T21:19:25.838617+0000","last_fullsized":"2023-10-06T21:19:25.838617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841625+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:19:25.841625+0000","last_peered":"2023-10-06T21:19:25.841625+0000","last_clean":"2023-10-06T21:19:25.841625+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:19:25.841625+0000","last_undegraded":"2023-10-06T21:19:25.841625+0000","last_fullsized":"2023-10-06T21:19:25.841625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839426+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:19:25.839426+0000","last_peered":"2023-10-06T21:19:25.839426+0000","last_clean":"2023-10-06T21:19:25.839426+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:19:25.839426+0000","last_undegraded":"2023-10-06T21:19:25.839426+0000","last_fullsized":"2023-10-06T21:19:25.839426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841000+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:19:25.841000+0000","last_peered":"2023-10-06T21:19:25.841000+0000","last_clean":"2023-10-06T21:19:25.841000+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:19:25.841000+0000","last_undegraded":"2023-10-06T21:19:25.841000+0000","last_fullsized":"2023-10-06T21:19:25.841000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:41.927740+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:19:41.927740+0000","last_peered":"2023-10-06T21:19:41.927740+0000","last_clean":"2023-10-06T21:19:41.927740+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:19:41.927740+0000","last_undegraded":"2023-10-06T21:19:41.927740+0000","last_fullsized":"2023-10-06T21:19:41.927740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840826+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:19:25.840826+0000","last_peered":"2023-10-06T21:19:25.840826+0000","last_clean":"2023-10-06T21:19:25.840826+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:19:25.840826+0000","last_undegraded":"2023-10-06T21:19:25.840826+0000","last_fullsized":"2023-10-06T21:19:25.840826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840309+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:19:25.840309+0000","last_peered":"2023-10-06T21:19:25.840309+0000","last_clean":"2023-10-06T21:19:25.840309+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:19:25.840309+0000","last_undegraded":"2023-10-06T21:19:25.840309+0000","last_fullsized":"2023-10-06T21:19:25.840309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:41.924655+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:19:41.924655+0000","last_peered":"2023-10-06T21:19:41.924655+0000","last_clean":"2023-10-06T21:19:41.924655+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:19:41.924655+0000","last_undegraded":"2023-10-06T21:19:41.924655+0000","last_fullsized":"2023-10-06T21:19:41.924655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.836095+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:19:25.836095+0000","last_peered":"2023-10-06T21:19:25.836095+0000","last_clean":"2023-10-06T21:19:25.836095+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:19:25.836095+0000","last_undegraded":"2023-10-06T21:19:25.836095+0000","last_fullsized":"2023-10-06T21:19:25.836095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835492+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:19:25.835492+0000","last_peered":"2023-10-06T21:19:25.835492+0000","last_clean":"2023-10-06T21:19:25.835492+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:19:25.835492+0000","last_undegraded":"2023-10-06T21:19:25.835492+0000","last_fullsized":"2023-10-06T21:19:25.835492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":320,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832687+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:19:25.832687+0000","last_peered":"2023-10-06T21:19:25.832687+0000","last_clean":"2023-10-06T21:19:25.832687+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:19:25.832687+0000","last_undegraded":"2023-10-06T21:19:25.832687+0000","last_fullsized":"2023-10-06T21:19:25.832687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837150+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:19:25.837150+0000","last_peered":"2023-10-06T21:19:25.837150+0000","last_clean":"2023-10-06T21:19:25.837150+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:19:25.837150+0000","last_undegraded":"2023-10-06T21:19:25.837150+0000","last_fullsized":"2023-10-06T21:19:25.837150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.865423+0000","last_change":"2023-10-06T21:19:26.864852+0000","last_active":"2023-10-06T21:19:26.865423+0000","last_peered":"2023-10-06T21:19:26.865423+0000","last_clean":"2023-10-06T21:19:26.865423+0000","last_became_active":"2023-10-06T21:19:26.846795+0000","last_became_peered":"2023-10-06T21:19:26.846795+0000","last_unstale":"2023-10-06T21:19:26.865423+0000","last_undegraded":"2023-10-06T21:19:26.865423+0000","last_fullsized":"2023-10-06T21:19:26.865423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:15:22.977641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838087+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:19:25.838087+0000","last_peered":"2023-10-06T21:19:25.838087+0000","last_clean":"2023-10-06T21:19:25.838087+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:19:25.838087+0000","last_undegraded":"2023-10-06T21:19:25.838087+0000","last_fullsized":"2023-10-06T21:19:25.838087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839228+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:19:25.839228+0000","last_peered":"2023-10-06T21:19:25.839228+0000","last_clean":"2023-10-06T21:19:25.839228+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:19:25.839228+0000","last_undegraded":"2023-10-06T21:19:25.839228+0000","last_fullsized":"2023-10-06T21:19:25.839228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839860+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:19:25.839860+0000","last_peered":"2023-10-06T21:19:25.839860+0000","last_clean":"2023-10-06T21:19:25.839860+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:19:25.839860+0000","last_undegraded":"2023-10-06T21:19:25.839860+0000","last_fullsized":"2023-10-06T21:19:25.839860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841073+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:19:25.841073+0000","last_peered":"2023-10-06T21:19:25.841073+0000","last_clean":"2023-10-06T21:19:25.841073+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:19:25.841073+0000","last_undegraded":"2023-10-06T21:19:25.841073+0000","last_fullsized":"2023-10-06T21:19:25.841073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835559+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:19:25.835559+0000","last_peered":"2023-10-06T21:19:25.835559+0000","last_clean":"2023-10-06T21:19:25.835559+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:19:25.835559+0000","last_undegraded":"2023-10-06T21:19:25.835559+0000","last_fullsized":"2023-10-06T21:19:25.835559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.841631+0000","last_change":"2023-10-06T21:19:26.841631+0000","last_active":"2023-10-06T21:19:26.841631+0000","last_peered":"2023-10-06T21:19:26.841631+0000","last_clean":"2023-10-06T21:19:26.841631+0000","last_became_active":"2023-10-06T21:19:26.841352+0000","last_became_peered":"2023-10-06T21:19:26.841352+0000","last_unstale":"2023-10-06T21:19:26.841631+0000","last_undegraded":"2023-10-06T21:19:26.841631+0000","last_fullsized":"2023-10-06T21:19:26.841631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:26:07.482368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843282+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:19:25.843282+0000","last_peered":"2023-10-06T21:19:25.843282+0000","last_clean":"2023-10-06T21:19:25.843282+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:19:25.843282+0000","last_undegraded":"2023-10-06T21:19:25.843282+0000","last_fullsized":"2023-10-06T21:19:25.843282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837413+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:19:25.837413+0000","last_peered":"2023-10-06T21:19:25.837413+0000","last_clean":"2023-10-06T21:19:25.837413+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:19:25.837413+0000","last_undegraded":"2023-10-06T21:19:25.837413+0000","last_fullsized":"2023-10-06T21:19:25.837413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835460+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:19:25.835460+0000","last_peered":"2023-10-06T21:19:25.835460+0000","last_clean":"2023-10-06T21:19:25.835460+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:19:25.835460+0000","last_undegraded":"2023-10-06T21:19:25.835460+0000","last_fullsized":"2023-10-06T21:19:25.835460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843427+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:19:25.843427+0000","last_peered":"2023-10-06T21:19:25.843427+0000","last_clean":"2023-10-06T21:19:25.843427+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:19:25.843427+0000","last_undegraded":"2023-10-06T21:19:25.843427+0000","last_fullsized":"2023-10-06T21:19:25.843427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839449+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:19:25.839449+0000","last_peered":"2023-10-06T21:19:25.839449+0000","last_clean":"2023-10-06T21:19:25.839449+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:19:25.839449+0000","last_undegraded":"2023-10-06T21:19:25.839449+0000","last_fullsized":"2023-10-06T21:19:25.839449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835663+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:19:25.835663+0000","last_peered":"2023-10-06T21:19:25.835663+0000","last_clean":"2023-10-06T21:19:25.835663+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:19:25.835663+0000","last_undegraded":"2023-10-06T21:19:25.835663+0000","last_fullsized":"2023-10-06T21:19:25.835663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838171+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:19:25.838171+0000","last_peered":"2023-10-06T21:19:25.838171+0000","last_clean":"2023-10-06T21:19:25.838171+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:19:25.838171+0000","last_undegraded":"2023-10-06T21:19:25.838171+0000","last_fullsized":"2023-10-06T21:19:25.838171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832919+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:19:25.832919+0000","last_peered":"2023-10-06T21:19:25.832919+0000","last_clean":"2023-10-06T21:19:25.832919+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:19:25.832919+0000","last_undegraded":"2023-10-06T21:19:25.832919+0000","last_fullsized":"2023-10-06T21:19:25.832919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:41.925133+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:19:41.925133+0000","last_peered":"2023-10-06T21:19:41.925133+0000","last_clean":"2023-10-06T21:19:41.925133+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:19:41.925133+0000","last_undegraded":"2023-10-06T21:19:41.925133+0000","last_fullsized":"2023-10-06T21:19:41.925133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833245+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:19:25.833245+0000","last_peered":"2023-10-06T21:19:25.833245+0000","last_clean":"2023-10-06T21:19:25.833245+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:19:25.833245+0000","last_undegraded":"2023-10-06T21:19:25.833245+0000","last_fullsized":"2023-10-06T21:19:25.833245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843021+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:19:25.843021+0000","last_peered":"2023-10-06T21:19:25.843021+0000","last_clean":"2023-10-06T21:19:25.843021+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:19:25.843021+0000","last_undegraded":"2023-10-06T21:19:25.843021+0000","last_fullsized":"2023-10-06T21:19:25.843021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833228+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:19:25.833228+0000","last_peered":"2023-10-06T21:19:25.833228+0000","last_clean":"2023-10-06T21:19:25.833228+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:19:25.833228+0000","last_undegraded":"2023-10-06T21:19:25.833228+0000","last_fullsized":"2023-10-06T21:19:25.833228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.851061+0000","last_change":"2023-10-06T21:19:26.851061+0000","last_active":"2023-10-06T21:19:26.851061+0000","last_peered":"2023-10-06T21:19:26.851061+0000","last_clean":"2023-10-06T21:19:26.851061+0000","last_became_active":"2023-10-06T21:19:26.845797+0000","last_became_peered":"2023-10-06T21:19:26.845797+0000","last_unstale":"2023-10-06T21:19:26.851061+0000","last_undegraded":"2023-10-06T21:19:26.851061+0000","last_fullsized":"2023-10-06T21:19:26.851061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:48.212793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833210+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:19:25.833210+0000","last_peered":"2023-10-06T21:19:25.833210+0000","last_clean":"2023-10-06T21:19:25.833210+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:19:25.833210+0000","last_undegraded":"2023-10-06T21:19:25.833210+0000","last_fullsized":"2023-10-06T21:19:25.833210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843522+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:19:25.843522+0000","last_peered":"2023-10-06T21:19:25.843522+0000","last_clean":"2023-10-06T21:19:25.843522+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:19:25.843522+0000","last_undegraded":"2023-10-06T21:19:25.843522+0000","last_fullsized":"2023-10-06T21:19:25.843522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.844504+0000","last_change":"2023-10-06T21:19:26.844504+0000","last_active":"2023-10-06T21:19:26.844504+0000","last_peered":"2023-10-06T21:19:26.844504+0000","last_clean":"2023-10-06T21:19:26.844504+0000","last_became_active":"2023-10-06T21:19:26.844105+0000","last_became_peered":"2023-10-06T21:19:26.844105+0000","last_unstale":"2023-10-06T21:19:26.844504+0000","last_undegraded":"2023-10-06T21:19:26.844504+0000","last_fullsized":"2023-10-06T21:19:26.844504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:08.655868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839315+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:19:25.839315+0000","last_peered":"2023-10-06T21:19:25.839315+0000","last_clean":"2023-10-06T21:19:25.839315+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:19:25.839315+0000","last_undegraded":"2023-10-06T21:19:25.839315+0000","last_fullsized":"2023-10-06T21:19:25.839315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832281+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:19:25.832281+0000","last_peered":"2023-10-06T21:19:25.832281+0000","last_clean":"2023-10-06T21:19:25.832281+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:19:25.832281+0000","last_undegraded":"2023-10-06T21:19:25.832281+0000","last_fullsized":"2023-10-06T21:19:25.832281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839150+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:19:25.839150+0000","last_peered":"2023-10-06T21:19:25.839150+0000","last_clean":"2023-10-06T21:19:25.839150+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:19:25.839150+0000","last_undegraded":"2023-10-06T21:19:25.839150+0000","last_fullsized":"2023-10-06T21:19:25.839150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841475+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:19:25.841475+0000","last_peered":"2023-10-06T21:19:25.841475+0000","last_clean":"2023-10-06T21:19:25.841475+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:19:25.841475+0000","last_undegraded":"2023-10-06T21:19:25.841475+0000","last_fullsized":"2023-10-06T21:19:25.841475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840949+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:19:25.840949+0000","last_peered":"2023-10-06T21:19:25.840949+0000","last_clean":"2023-10-06T21:19:25.840949+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:19:25.840949+0000","last_undegraded":"2023-10-06T21:19:25.840949+0000","last_fullsized":"2023-10-06T21:19:25.840949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840977+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:19:25.840977+0000","last_peered":"2023-10-06T21:19:25.840977+0000","last_clean":"2023-10-06T21:19:25.840977+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:19:25.840977+0000","last_undegraded":"2023-10-06T21:19:25.840977+0000","last_fullsized":"2023-10-06T21:19:25.840977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.847714+0000","last_change":"2023-10-06T21:19:26.847714+0000","last_active":"2023-10-06T21:19:26.847714+0000","last_peered":"2023-10-06T21:19:26.847714+0000","last_clean":"2023-10-06T21:19:26.847714+0000","last_became_active":"2023-10-06T21:19:26.847302+0000","last_became_peered":"2023-10-06T21:19:26.847302+0000","last_unstale":"2023-10-06T21:19:26.847714+0000","last_undegraded":"2023-10-06T21:19:26.847714+0000","last_fullsized":"2023-10-06T21:19:26.847714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:10:24.079203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835340+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:19:25.835340+0000","last_peered":"2023-10-06T21:19:25.835340+0000","last_clean":"2023-10-06T21:19:25.835340+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:19:25.835340+0000","last_undegraded":"2023-10-06T21:19:25.835340+0000","last_fullsized":"2023-10-06T21:19:25.835340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843439+0000","last_change":"2023-10-06T21:18:54.859301+0000","last_active":"2023-10-06T21:19:25.843439+0000","last_peered":"2023-10-06T21:19:25.843439+0000","last_clean":"2023-10-06T21:19:25.843439+0000","last_became_active":"2023-10-06T21:18:54.858936+0000","last_became_peered":"2023-10-06T21:18:54.858936+0000","last_unstale":"2023-10-06T21:19:25.843439+0000","last_undegraded":"2023-10-06T21:19:25.843439+0000","last_fullsized":"2023-10-06T21:19:25.843439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:29:35.239801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838994+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:19:25.838994+0000","last_peered":"2023-10-06T21:19:25.838994+0000","last_clean":"2023-10-06T21:19:25.838994+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:19:25.838994+0000","last_undegraded":"2023-10-06T21:19:25.838994+0000","last_fullsized":"2023-10-06T21:19:25.838994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841157+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:19:25.841157+0000","last_peered":"2023-10-06T21:19:25.841157+0000","last_clean":"2023-10-06T21:19:25.841157+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:19:25.841157+0000","last_undegraded":"2023-10-06T21:19:25.841157+0000","last_fullsized":"2023-10-06T21:19:25.841157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65793,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":493,"num_read_kb":321,"num_write":234,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197379,"data_compressed":5511,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460246,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.749},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.89600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.996}]},{"osd":10,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.92700000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1110209,"data_compressed":43346,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":1,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.754},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.0369999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.81000000000000005,"15min":0.79100000000000004},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.96599999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.054,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80800000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.86599999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1000000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.111}]},{"osd":6,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.069,"5min":1.532,"15min":1.532},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.052,"5min":0.98799999999999999,"15min":0.97699999999999998},"min":{"1min":0.60299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.5300000000000002,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2070000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.077},{"interface":"front","average":{"1min":0.878,"5min":0.96899999999999997,"15min":0.98399999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1140000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0009999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.151,"15min":1.1020000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.827999999999999,"5min":10.827999999999999,"15min":10.827999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.1020000000000001,"15min":1.0589999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":10.808999999999999,"5min":10.808999999999999,"15min":10.808999999999999},"last":0.94099999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8479999999999999,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.143}]}]},{"osd":9,"up_from":68,"seq":292057776162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":460064,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73499999999999999,"5min":0.749,"15min":0.749},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":1.131,"15min":1.131},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93899999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85099999999999998,"5min":1.097,"15min":1.097},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":1.21,"15min":1.21},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2030000000000001,"5min":1.242,"15min":1.242},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.90600000000000003},"min":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94799999999999995,"5min":1.222,"15min":1.222},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.98499999999999999,"5min":1.288,"15min":1.288},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99099999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.90300000000000002,"15min":0.93300000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0840000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.433},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.93000000000000005,"15min":0.95599999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.036,"15min":1.0600000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0880000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.016,"15min":1.04},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.621}]},{"osd":8,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.03,"15min":1.0580000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.248,"5min":1.379,"15min":1.379},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.028,"15min":1.056},"min":{"1min":0.69599999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1020000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1779999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.0369999999999999,"5min":1.575,"15min":1.575},"last":0.754}]}]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459999,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.63},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.70399999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.749,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.196,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.60199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.167,"5min":0.92100000000000004,"15min":0.89000000000000001},"min":{"1min":0.58399999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.63100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2450000000000001,"5min":1.468,"15min":1.468},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.89800000000000002,"15min":0.88900000000000001},"min":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.266,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.306,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.67300000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.94999999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85899999999999999,"15min":0.84399999999999997},"min":{"1min":0.62,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.048,"5min":0.92300000000000004,"15min":0.90200000000000002},"min":{"1min":0.73299999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.97999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1052634,"data_compressed":41509,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.45900000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":1.044,"15min":1.044},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94499999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.376,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91300000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.073,"15min":1.073},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81399999999999995,"15min":0.77200000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.05,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82299999999999995,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.018,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67500000000000004}]},{"osd":8,"last update":"Fri Oct 6 21:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":1.0269999999999999,"15min":1.0649999999999999},"min":{"1min":0.53200000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.996,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.111,"5min":1.119,"15min":1.119},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.88700000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.149,"5min":1.198,"15min":1.198},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.85299999999999998,"15min":0.85199999999999998},"min":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.98499999999999999,"5min":1.034,"15min":1.034},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.119,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.99199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459787,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0720000000000001,"5min":1.121,"15min":1.121},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.81200000000000006,"5min":1.887,"15min":1.887},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.441},"max":{"1min":1.0469999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.92800000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.034,"5min":1.097,"15min":1.097},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.66400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0509999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.88600000000000001,"15min":0.91800000000000004},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97399999999999998,"5min":1.827,"15min":1.827},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.56699999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.871,"15min":0.85999999999999999},"min":{"1min":0.68100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3280000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.399,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.997,"5min":1.905,"15min":1.905},"last":0.83999999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.89500000000000002,"15min":0.90000000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.361,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.01,"5min":1.165,"15min":1.165},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.073,"5min":1.232,"15min":1.232},"last":0.624},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0009999999999999,"15min":1.0049999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":519675,"data_compressed":7614,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.016,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.01},{"interface":"front","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.69199999999999995,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.4390000000000001,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.48199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.91100000000000003}]},{"osd":3,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":6.3419999999999996,"5min":6.3419999999999996,"15min":6.3419999999999996},"last":1.026},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.0780000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.60999999999999999,"15min":0.56000000000000005},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93700000000000006,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.59099999999999997,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.72299999999999998,"15min":0.64700000000000002},"min":{"1min":0.57199999999999995,"5min":0.433,"15min":0.433},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.73699999999999999,"15min":0.66200000000000003},"min":{"1min":0.66500000000000004,"5min":0.435,"15min":0.435},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":0.81799999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.79000000000000004,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.76100000000000001},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.98699999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":0.76100000000000001,"15min":0.68799999999999994},"min":{"1min":0.48799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":8.7620000000000005,"5min":8.7620000000000005,"15min":8.7620000000000005},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":0.78100000000000003,"15min":0.69099999999999995},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.4410000000000007,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":0.84299999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0940000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.58899999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0409999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.96899999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":0.877,"15min":0.82999999999999996},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":7.1760000000000002,"5min":7.1760000000000002,"15min":7.1760000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.038,"5min":0.89400000000000002,"15min":0.87},"min":{"1min":0.627,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":1.0669999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":0.93799999999999994,"15min":0.89400000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":6.0419999999999998,"5min":6.0419999999999998,"15min":6.0419999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.016,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.099}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461120,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1499999999999999,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.034,"5min":1.738,"15min":1.738},"last":0.82799999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.52900000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0580000000000001,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.012,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.45100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.99299999999999999,"5min":1.048,"15min":1.048},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.65700000000000003},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.091,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.67500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.85799999999999998,"15min":0.83499999999999996},"min":{"1min":0.65500000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.159,"5min":1.228,"15min":1.228},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.62},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.91200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.63900000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94099999999999995,"15min":0.95699999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.286,"15min":1.286},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.999,"5min":1,"15min":1},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.94199999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:18:59 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.66900000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":519622,"data_compressed":7614,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63400000000000001,"15min":0.57099999999999995},"min":{"1min":0.48799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.92200000000000004,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.70899999999999996,"15min":0.61499999999999999},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.65400000000000003,"15min":0.60799999999999998},"min":{"1min":0.48999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.044,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.878}]},{"osd":5,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.56899999999999995,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.056,"15min":1.056},"last":0.67000000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.123,"5min":1.298,"15min":1.298},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1220000000000001,"5min":1.266,"15min":1.266},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.88600000000000001,"15min":0.878},"min":{"1min":0.69799999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.198,"5min":1.266,"15min":1.266},"last":1},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.85699999999999998,"15min":0.85599999999999998},"min":{"1min":0.68799999999999994,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.135,"5min":1.194,"15min":1.194},"last":1.024}]},{"osd":10,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.93500000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2490000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0880000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459618,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45000000000000001,"15min":0.40999999999999998},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71299999999999997,"15min":0.65400000000000003},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":1.0009999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.75,"15min":0.67200000000000004},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.1799999999999997,"5min":5.1799999999999997,"15min":5.1799999999999997},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.67300000000000004},"min":{"1min":0.56000000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.054,"5min":1.206,"15min":1.206},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.67100000000000004},"min":{"1min":0.54800000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Oct 6 21:19:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.746,"15min":0.745},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.57799999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95099999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.26,"15min":1.26},"last":0.442},{"interface":"front","average":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.077,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1759999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.008,"5min":1.196,"15min":1.196},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.93400000000000005,"15min":0.94099999999999995},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.113,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.75700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":460032,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.36399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76300000000000001,"5min":0.874,"15min":0.874},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.42299999999999999},"min":{"1min":0.34699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":1.601,"15min":1.601},"last":0.97699999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65500000000000003,"15min":0.629},"min":{"1min":0.52600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84499999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70299999999999996,"15min":0.65000000000000002},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86499999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.77600000000000002,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.028,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.758,"15min":1.758},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.46800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80900000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0629999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86799999999999999,"15min":0.84999999999999998},"min":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.90300000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92400000000000004},"min":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.113,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.89900000000000002,"15min":0.89700000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.0169999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1052253,"data_compressed":41509,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.625,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.68700000000000006,"15min":0.63},"min":{"1min":0.56699999999999995,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0820000000000001,"5min":1.089,"15min":1.089},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51100000000000001,"15min":0.499},"min":{"1min":0.38700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74399999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.88600000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68000000000000005,"15min":0.66200000000000003},"min":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.623,"5min":0.442,"15min":0.442},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74099999999999999,"15min":0.70399999999999996},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.157,"5min":1.196,"15min":1.196},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89400000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.84299999999999997,"15min":0.83499999999999996},"min":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.78400000000000003,"15min":0.76600000000000001},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.69099999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.93700000000000006}]},{"osd":9,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0369999999999999,"5min":1.339,"15min":1.339},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.871,"15min":0.876},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1040000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.874,"15min":0.85799999999999998},"min":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.083,"5min":1.262,"15min":1.262},"last":0.82399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60733,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60953,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61107,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:19:50.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T21:19:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:50 smithi142 ceph-mon[102951]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:51.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:50 smithi142 ceph-mon[102951]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:50 smithi031 ceph-mon[98539]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:51.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:50 smithi031 ceph-mon[98539]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:50 smithi146 ceph-mon[102809]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:51.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:50 smithi146 ceph-mon[102809]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 -- 172.21.15.31:0/3738794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c106f90 msgr2=0x7f8f8c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 --2- 172.21.15.31:0/3738794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c106f90 0x7f8f8c107370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f800099c0 tx=0x7f8f80009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 -- 172.21.15.31:0/3738794083 shutdown_connections 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 --2- 172.21.15.31:0/3738794083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c101940 0x7f8f8c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 --2- 172.21.15.31:0/3738794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c106f90 0x7f8f8c107370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 --2- 172.21.15.31:0/3738794083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c100f80 0x7f8f8c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 -- 172.21.15.31:0/3738794083 >> 172.21.15.31:0/3738794083 conn(0x7f8f8c0780a0 msgr2=0x7f8f8c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 -- 172.21.15.31:0/3738794083 shutdown_connections 2023-10-06T21:19:52.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.536+0000 7f8f92de1700 1 -- 172.21.15.31:0/3738794083 wait complete. 2023-10-06T21:19:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.537+0000 7f8f92de1700 1 Processor -- start 2023-10-06T21:19:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.537+0000 7f8f92de1700 1 -- start start 2023-10-06T21:19:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 0x7f8f8c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c106f90 0x7f8f8c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c19e0e0 con 0x7f8f8c100f80 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c19e250 con 0x7f8f8c101940 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f92de1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c19e410 con 0x7f8f8c106f90 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f90b7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f90b7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42128/0 (socket says 172.21.15.31:42128) 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f90b7d700 1 -- 172.21.15.31:0/376220233 learned_addr learned my addr 172.21.15.31:0/376220233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f8bfff700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 0x7f8f8c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.538+0000 7f8f9137e700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c106f90 0x7f8f8c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 -- 172.21.15.31:0/376220233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c106f90 msgr2=0x7f8f8c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c106f90 0x7f8f8c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 -- 172.21.15.31:0/376220233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 msgr2=0x7f8f8c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 0x7f8f8c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 -- 172.21.15.31:0/376220233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800095f0 con 0x7f8f8c100f80 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f8bfff700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 0x7f8f8c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f90b7d700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f80004730 tx=0x7f8f80005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8001d070 con 0x7f8f8c100f80 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f800040e0 con 0x7f8f8c100f80 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c19e690 con 0x7f8f8c100f80 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80005290 con 0x7f8f8c100f80 2023-10-06T21:19:52.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.539+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1a5f50 con 0x7f8f8c100f80 2023-10-06T21:19:52.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.540+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c04ebb0 con 0x7f8f8c100f80 2023-10-06T21:19:52.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.542+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8f800049a0 con 0x7f8f8c100f80 2023-10-06T21:19:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.543+0000 7f8f89ffb700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 0x7f8f7c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.544+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8f800cdd80 con 0x7f8f8c100f80 2023-10-06T21:19:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.544+0000 7f8f8bfff700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 0x7f8f7c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.544+0000 7f8f8bfff700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 0x7f8f7c0a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005950 tx=0x7f8f780058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:52.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.547+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f80097290 con 0x7f8f8c100f80 2023-10-06T21:19:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.752+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f8c110e40 con 0x7f8f7c0a1590 2023-10-06T21:19:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.763+0000 7f8f89ffb700 1 -- 172.21.15.31:0/376220233 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340001 (secure 0 0 0) 0x7f8f8c110e40 con 0x7f8f7c0a1590 2023-10-06T21:19:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 msgr2=0x7f8f7c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 0x7f8f7c0a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005950 tx=0x7f8f780058e0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 msgr2=0x7f8f8c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f80004730 tx=0x7f8f80005e80 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 shutdown_connections 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8f7c0a1590 0x7f8f7c0a3a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f8c106f90 0x7f8f8c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f8c100f80 0x7f8f8c199120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 --2- 172.21.15.31:0/376220233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c101940 0x7f8f8c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 >> 172.21.15.31:0/376220233 conn(0x7f8f8c0780a0 msgr2=0x7f8f8c10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 shutdown_connections 2023-10-06T21:19:52.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:52.768+0000 7f8f92de1700 1 -- 172.21.15.31:0/376220233 wait complete. 2023-10-06T21:19:52.776 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:19:53.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:52 smithi142 ceph-mon[102951]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2023-10-06T21:19:53.194 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:52 smithi031 ceph-mon[98539]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2023-10-06T21:19:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:52 smithi146 ceph-mon[102809]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2023-10-06T21:19:53.302 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2023-10-06T21:19:51.824720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656407,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":674,"num_read_kb":460,"num_write":417,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336052,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605772,"statfs":{"total":1151940427776,"available":1151596310528,"internally_reserved":0,"allocated":12062720,"data_stored":7475259,"data_compressed":182031,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":64,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003118"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841866+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:19:25.841866+0000","last_peered":"2023-10-06T21:19:25.841866+0000","last_clean":"2023-10-06T21:19:25.841866+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:19:25.841866+0000","last_undegraded":"2023-10-06T21:19:25.841866+0000","last_fullsized":"2023-10-06T21:19:25.841866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832672+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:19:25.832672+0000","last_peered":"2023-10-06T21:19:25.832672+0000","last_clean":"2023-10-06T21:19:25.832672+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:19:25.832672+0000","last_undegraded":"2023-10-06T21:19:25.832672+0000","last_fullsized":"2023-10-06T21:19:25.832672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832409+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:19:25.832409+0000","last_peered":"2023-10-06T21:19:25.832409+0000","last_clean":"2023-10-06T21:19:25.832409+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:19:25.832409+0000","last_undegraded":"2023-10-06T21:19:25.832409+0000","last_fullsized":"2023-10-06T21:19:25.832409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835657+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:19:25.835657+0000","last_peered":"2023-10-06T21:19:25.835657+0000","last_clean":"2023-10-06T21:19:25.835657+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:19:25.835657+0000","last_undegraded":"2023-10-06T21:19:25.835657+0000","last_fullsized":"2023-10-06T21:19:25.835657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838062+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:19:25.838062+0000","last_peered":"2023-10-06T21:19:25.838062+0000","last_clean":"2023-10-06T21:19:25.838062+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:19:25.838062+0000","last_undegraded":"2023-10-06T21:19:25.838062+0000","last_fullsized":"2023-10-06T21:19:25.838062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841748+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:19:25.841748+0000","last_peered":"2023-10-06T21:19:25.841748+0000","last_clean":"2023-10-06T21:19:25.841748+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:19:25.841748+0000","last_undegraded":"2023-10-06T21:19:25.841748+0000","last_fullsized":"2023-10-06T21:19:25.841748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832621+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:19:25.832621+0000","last_peered":"2023-10-06T21:19:25.832621+0000","last_clean":"2023-10-06T21:19:25.832621+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:19:25.832621+0000","last_undegraded":"2023-10-06T21:19:25.832621+0000","last_fullsized":"2023-10-06T21:19:25.832621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835282+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:19:25.835282+0000","last_peered":"2023-10-06T21:19:25.835282+0000","last_clean":"2023-10-06T21:19:25.835282+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:19:25.835282+0000","last_undegraded":"2023-10-06T21:19:25.835282+0000","last_fullsized":"2023-10-06T21:19:25.835282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843808+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:19:25.843808+0000","last_peered":"2023-10-06T21:19:25.843808+0000","last_clean":"2023-10-06T21:19:25.843808+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:19:25.843808+0000","last_undegraded":"2023-10-06T21:19:25.843808+0000","last_fullsized":"2023-10-06T21:19:25.843808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841122+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:19:25.841122+0000","last_peered":"2023-10-06T21:19:25.841122+0000","last_clean":"2023-10-06T21:19:25.841122+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:19:25.841122+0000","last_undegraded":"2023-10-06T21:19:25.841122+0000","last_fullsized":"2023-10-06T21:19:25.841122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833138+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:19:25.833138+0000","last_peered":"2023-10-06T21:19:25.833138+0000","last_clean":"2023-10-06T21:19:25.833138+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:19:25.833138+0000","last_undegraded":"2023-10-06T21:19:25.833138+0000","last_fullsized":"2023-10-06T21:19:25.833138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835449+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:19:25.835449+0000","last_peered":"2023-10-06T21:19:25.835449+0000","last_clean":"2023-10-06T21:19:25.835449+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:19:25.835449+0000","last_undegraded":"2023-10-06T21:19:25.835449+0000","last_fullsized":"2023-10-06T21:19:25.835449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838913+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:19:25.838913+0000","last_peered":"2023-10-06T21:19:25.838913+0000","last_clean":"2023-10-06T21:19:25.838913+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:19:25.838913+0000","last_undegraded":"2023-10-06T21:19:25.838913+0000","last_fullsized":"2023-10-06T21:19:25.838913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832111+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:19:25.832111+0000","last_peered":"2023-10-06T21:19:25.832111+0000","last_clean":"2023-10-06T21:19:25.832111+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:19:25.832111+0000","last_undegraded":"2023-10-06T21:19:25.832111+0000","last_fullsized":"2023-10-06T21:19:25.832111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832065+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:19:25.832065+0000","last_peered":"2023-10-06T21:19:25.832065+0000","last_clean":"2023-10-06T21:19:25.832065+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:19:25.832065+0000","last_undegraded":"2023-10-06T21:19:25.832065+0000","last_fullsized":"2023-10-06T21:19:25.832065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843154+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:19:25.843154+0000","last_peered":"2023-10-06T21:19:25.843154+0000","last_clean":"2023-10-06T21:19:25.843154+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:19:25.843154+0000","last_undegraded":"2023-10-06T21:19:25.843154+0000","last_fullsized":"2023-10-06T21:19:25.843154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.842101+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:19:25.842101+0000","last_peered":"2023-10-06T21:19:25.842101+0000","last_clean":"2023-10-06T21:19:25.842101+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:19:25.842101+0000","last_undegraded":"2023-10-06T21:19:25.842101+0000","last_fullsized":"2023-10-06T21:19:25.842101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839761+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:19:25.839761+0000","last_peered":"2023-10-06T21:19:25.839761+0000","last_clean":"2023-10-06T21:19:25.839761+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:19:25.839761+0000","last_undegraded":"2023-10-06T21:19:25.839761+0000","last_fullsized":"2023-10-06T21:19:25.839761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840155+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:19:25.840155+0000","last_peered":"2023-10-06T21:19:25.840155+0000","last_clean":"2023-10-06T21:19:25.840155+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:19:25.840155+0000","last_undegraded":"2023-10-06T21:19:25.840155+0000","last_fullsized":"2023-10-06T21:19:25.840155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:45.932702+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:19:45.932702+0000","last_peered":"2023-10-06T21:19:45.932702+0000","last_clean":"2023-10-06T21:19:45.932702+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:19:45.932702+0000","last_undegraded":"2023-10-06T21:19:45.932702+0000","last_fullsized":"2023-10-06T21:19:45.932702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840953+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:19:25.840953+0000","last_peered":"2023-10-06T21:19:25.840953+0000","last_clean":"2023-10-06T21:19:25.840953+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:19:25.840953+0000","last_undegraded":"2023-10-06T21:19:25.840953+0000","last_fullsized":"2023-10-06T21:19:25.840953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841280+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:19:25.841280+0000","last_peered":"2023-10-06T21:19:25.841280+0000","last_clean":"2023-10-06T21:19:25.841280+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:19:25.841280+0000","last_undegraded":"2023-10-06T21:19:25.841280+0000","last_fullsized":"2023-10-06T21:19:25.841280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833139+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:19:25.833139+0000","last_peered":"2023-10-06T21:19:25.833139+0000","last_clean":"2023-10-06T21:19:25.833139+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:19:25.833139+0000","last_undegraded":"2023-10-06T21:19:25.833139+0000","last_fullsized":"2023-10-06T21:19:25.833139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.863512+0000","last_change":"2023-10-06T21:19:26.863512+0000","last_active":"2023-10-06T21:19:26.863512+0000","last_peered":"2023-10-06T21:19:26.863512+0000","last_clean":"2023-10-06T21:19:26.863512+0000","last_became_active":"2023-10-06T21:19:26.845274+0000","last_became_peered":"2023-10-06T21:19:26.845274+0000","last_unstale":"2023-10-06T21:19:26.863512+0000","last_undegraded":"2023-10-06T21:19:26.863512+0000","last_fullsized":"2023-10-06T21:19:26.863512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:16.751914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843466+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:19:25.843466+0000","last_peered":"2023-10-06T21:19:25.843466+0000","last_clean":"2023-10-06T21:19:25.843466+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:19:25.843466+0000","last_undegraded":"2023-10-06T21:19:25.843466+0000","last_fullsized":"2023-10-06T21:19:25.843466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838422+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:19:25.838422+0000","last_peered":"2023-10-06T21:19:25.838422+0000","last_clean":"2023-10-06T21:19:25.838422+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:19:25.838422+0000","last_undegraded":"2023-10-06T21:19:25.838422+0000","last_fullsized":"2023-10-06T21:19:25.838422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.991142+0000","last_change":"2023-10-06T21:19:26.991142+0000","last_active":"2023-10-06T21:19:26.991142+0000","last_peered":"2023-10-06T21:19:26.991142+0000","last_clean":"2023-10-06T21:19:26.991142+0000","last_became_active":"2023-10-06T21:19:26.990765+0000","last_became_peered":"2023-10-06T21:19:26.990765+0000","last_unstale":"2023-10-06T21:19:26.991142+0000","last_undegraded":"2023-10-06T21:19:26.991142+0000","last_fullsized":"2023-10-06T21:19:26.991142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:35:44.692174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:49.903112+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:19:49.903112+0000","last_peered":"2023-10-06T21:19:49.903112+0000","last_clean":"2023-10-06T21:19:49.903112+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:19:49.903112+0000","last_undegraded":"2023-10-06T21:19:49.903112+0000","last_fullsized":"2023-10-06T21:19:49.903112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837859+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:19:25.837859+0000","last_peered":"2023-10-06T21:19:25.837859+0000","last_clean":"2023-10-06T21:19:25.837859+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:19:25.837859+0000","last_undegraded":"2023-10-06T21:19:25.837859+0000","last_fullsized":"2023-10-06T21:19:25.837859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.998407+0000","last_change":"2023-10-06T21:19:26.998407+0000","last_active":"2023-10-06T21:19:26.998407+0000","last_peered":"2023-10-06T21:19:26.998407+0000","last_clean":"2023-10-06T21:19:26.998407+0000","last_became_active":"2023-10-06T21:19:26.991918+0000","last_became_peered":"2023-10-06T21:19:26.991918+0000","last_unstale":"2023-10-06T21:19:26.998407+0000","last_undegraded":"2023-10-06T21:19:26.998407+0000","last_fullsized":"2023-10-06T21:19:26.998407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:05.903796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.846902+0000","last_change":"2023-10-06T21:19:26.846902+0000","last_active":"2023-10-06T21:19:26.846902+0000","last_peered":"2023-10-06T21:19:26.846902+0000","last_clean":"2023-10-06T21:19:26.846902+0000","last_became_active":"2023-10-06T21:19:26.846491+0000","last_became_peered":"2023-10-06T21:19:26.846491+0000","last_unstale":"2023-10-06T21:19:26.846902+0000","last_undegraded":"2023-10-06T21:19:26.846902+0000","last_fullsized":"2023-10-06T21:19:26.846902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:18:06.115026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838901+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:19:25.838901+0000","last_peered":"2023-10-06T21:19:25.838901+0000","last_clean":"2023-10-06T21:19:25.838901+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:19:25.838901+0000","last_undegraded":"2023-10-06T21:19:25.838901+0000","last_fullsized":"2023-10-06T21:19:25.838901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835625+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:19:25.835625+0000","last_peered":"2023-10-06T21:19:25.835625+0000","last_clean":"2023-10-06T21:19:25.835625+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:19:25.835625+0000","last_undegraded":"2023-10-06T21:19:25.835625+0000","last_fullsized":"2023-10-06T21:19:25.835625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839531+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:19:25.839531+0000","last_peered":"2023-10-06T21:19:25.839531+0000","last_clean":"2023-10-06T21:19:25.839531+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:19:25.839531+0000","last_undegraded":"2023-10-06T21:19:25.839531+0000","last_fullsized":"2023-10-06T21:19:25.839531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839103+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:19:25.839103+0000","last_peered":"2023-10-06T21:19:25.839103+0000","last_clean":"2023-10-06T21:19:25.839103+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:19:25.839103+0000","last_undegraded":"2023-10-06T21:19:25.839103+0000","last_fullsized":"2023-10-06T21:19:25.839103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:50.903859+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:19:50.903859+0000","last_peered":"2023-10-06T21:19:50.903859+0000","last_clean":"2023-10-06T21:19:50.903859+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:19:50.903859+0000","last_undegraded":"2023-10-06T21:19:50.903859+0000","last_fullsized":"2023-10-06T21:19:50.903859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843810+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:19:25.843810+0000","last_peered":"2023-10-06T21:19:25.843810+0000","last_clean":"2023-10-06T21:19:25.843810+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:19:25.843810+0000","last_undegraded":"2023-10-06T21:19:25.843810+0000","last_fullsized":"2023-10-06T21:19:25.843810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833096+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:19:25.833096+0000","last_peered":"2023-10-06T21:19:25.833096+0000","last_clean":"2023-10-06T21:19:25.833096+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:19:25.833096+0000","last_undegraded":"2023-10-06T21:19:25.833096+0000","last_fullsized":"2023-10-06T21:19:25.833096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833053+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:19:25.833053+0000","last_peered":"2023-10-06T21:19:25.833053+0000","last_clean":"2023-10-06T21:19:25.833053+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:19:25.833053+0000","last_undegraded":"2023-10-06T21:19:25.833053+0000","last_fullsized":"2023-10-06T21:19:25.833053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843136+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:19:25.843136+0000","last_peered":"2023-10-06T21:19:25.843136+0000","last_clean":"2023-10-06T21:19:25.843136+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:19:25.843136+0000","last_undegraded":"2023-10-06T21:19:25.843136+0000","last_fullsized":"2023-10-06T21:19:25.843136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843582+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:19:25.843582+0000","last_peered":"2023-10-06T21:19:25.843582+0000","last_clean":"2023-10-06T21:19:25.843582+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:19:25.843582+0000","last_undegraded":"2023-10-06T21:19:25.843582+0000","last_fullsized":"2023-10-06T21:19:25.843582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":162,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:50.903472+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:19:50.903472+0000","last_peered":"2023-10-06T21:19:50.903472+0000","last_clean":"2023-10-06T21:19:50.903472+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:19:50.903472+0000","last_undegraded":"2023-10-06T21:19:50.903472+0000","last_fullsized":"2023-10-06T21:19:50.903472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58981,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837984+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:19:25.837984+0000","last_peered":"2023-10-06T21:19:25.837984+0000","last_clean":"2023-10-06T21:19:25.837984+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:19:25.837984+0000","last_undegraded":"2023-10-06T21:19:25.837984+0000","last_fullsized":"2023-10-06T21:19:25.837984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839795+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:19:25.839795+0000","last_peered":"2023-10-06T21:19:25.839795+0000","last_clean":"2023-10-06T21:19:25.839795+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:19:25.839795+0000","last_undegraded":"2023-10-06T21:19:25.839795+0000","last_fullsized":"2023-10-06T21:19:25.839795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838622+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:19:25.838622+0000","last_peered":"2023-10-06T21:19:25.838622+0000","last_clean":"2023-10-06T21:19:25.838622+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:19:25.838622+0000","last_undegraded":"2023-10-06T21:19:25.838622+0000","last_fullsized":"2023-10-06T21:19:25.838622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.842403+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:19:25.842403+0000","last_peered":"2023-10-06T21:19:25.842403+0000","last_clean":"2023-10-06T21:19:25.842403+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:19:25.842403+0000","last_undegraded":"2023-10-06T21:19:25.842403+0000","last_fullsized":"2023-10-06T21:19:25.842403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833007+0000","last_change":"2023-10-06T21:18:54.860485+0000","last_active":"2023-10-06T21:19:25.833007+0000","last_peered":"2023-10-06T21:19:25.833007+0000","last_clean":"2023-10-06T21:19:25.833007+0000","last_became_active":"2023-10-06T21:18:54.860171+0000","last_became_peered":"2023-10-06T21:18:54.860171+0000","last_unstale":"2023-10-06T21:19:25.833007+0000","last_undegraded":"2023-10-06T21:19:25.833007+0000","last_fullsized":"2023-10-06T21:19:25.833007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:46:50.890861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:49.939800+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:19:49.939800+0000","last_peered":"2023-10-06T21:19:49.939800+0000","last_clean":"2023-10-06T21:19:49.939800+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:19:49.939800+0000","last_undegraded":"2023-10-06T21:19:49.939800+0000","last_fullsized":"2023-10-06T21:19:49.939800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832537+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:19:25.832537+0000","last_peered":"2023-10-06T21:19:25.832537+0000","last_clean":"2023-10-06T21:19:25.832537+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:19:25.832537+0000","last_undegraded":"2023-10-06T21:19:25.832537+0000","last_fullsized":"2023-10-06T21:19:25.832537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.862636+0000","last_change":"2023-10-06T21:19:26.862156+0000","last_active":"2023-10-06T21:19:26.862636+0000","last_peered":"2023-10-06T21:19:26.862636+0000","last_clean":"2023-10-06T21:19:26.862636+0000","last_became_active":"2023-10-06T21:19:26.844987+0000","last_became_peered":"2023-10-06T21:19:26.844987+0000","last_unstale":"2023-10-06T21:19:26.862636+0000","last_undegraded":"2023-10-06T21:19:26.862636+0000","last_fullsized":"2023-10-06T21:19:26.862636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:12:56.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840419+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:19:25.840419+0000","last_peered":"2023-10-06T21:19:25.840419+0000","last_clean":"2023-10-06T21:19:25.840419+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:19:25.840419+0000","last_undegraded":"2023-10-06T21:19:25.840419+0000","last_fullsized":"2023-10-06T21:19:25.840419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838617+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:19:25.838617+0000","last_peered":"2023-10-06T21:19:25.838617+0000","last_clean":"2023-10-06T21:19:25.838617+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:19:25.838617+0000","last_undegraded":"2023-10-06T21:19:25.838617+0000","last_fullsized":"2023-10-06T21:19:25.838617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841625+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:19:25.841625+0000","last_peered":"2023-10-06T21:19:25.841625+0000","last_clean":"2023-10-06T21:19:25.841625+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:19:25.841625+0000","last_undegraded":"2023-10-06T21:19:25.841625+0000","last_fullsized":"2023-10-06T21:19:25.841625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839426+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:19:25.839426+0000","last_peered":"2023-10-06T21:19:25.839426+0000","last_clean":"2023-10-06T21:19:25.839426+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:19:25.839426+0000","last_undegraded":"2023-10-06T21:19:25.839426+0000","last_fullsized":"2023-10-06T21:19:25.839426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841000+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:19:25.841000+0000","last_peered":"2023-10-06T21:19:25.841000+0000","last_clean":"2023-10-06T21:19:25.841000+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:19:25.841000+0000","last_undegraded":"2023-10-06T21:19:25.841000+0000","last_fullsized":"2023-10-06T21:19:25.841000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:46.934600+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:19:46.934600+0000","last_peered":"2023-10-06T21:19:46.934600+0000","last_clean":"2023-10-06T21:19:46.934600+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:19:46.934600+0000","last_undegraded":"2023-10-06T21:19:46.934600+0000","last_fullsized":"2023-10-06T21:19:46.934600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840826+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:19:25.840826+0000","last_peered":"2023-10-06T21:19:25.840826+0000","last_clean":"2023-10-06T21:19:25.840826+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:19:25.840826+0000","last_undegraded":"2023-10-06T21:19:25.840826+0000","last_fullsized":"2023-10-06T21:19:25.840826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840309+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:19:25.840309+0000","last_peered":"2023-10-06T21:19:25.840309+0000","last_clean":"2023-10-06T21:19:25.840309+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:19:25.840309+0000","last_undegraded":"2023-10-06T21:19:25.840309+0000","last_fullsized":"2023-10-06T21:19:25.840309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:46.934301+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:19:46.934301+0000","last_peered":"2023-10-06T21:19:46.934301+0000","last_clean":"2023-10-06T21:19:46.934301+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:19:46.934301+0000","last_undegraded":"2023-10-06T21:19:46.934301+0000","last_fullsized":"2023-10-06T21:19:46.934301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.836095+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:19:25.836095+0000","last_peered":"2023-10-06T21:19:25.836095+0000","last_clean":"2023-10-06T21:19:25.836095+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:19:25.836095+0000","last_undegraded":"2023-10-06T21:19:25.836095+0000","last_fullsized":"2023-10-06T21:19:25.836095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835492+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:19:25.835492+0000","last_peered":"2023-10-06T21:19:25.835492+0000","last_clean":"2023-10-06T21:19:25.835492+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:19:25.835492+0000","last_undegraded":"2023-10-06T21:19:25.835492+0000","last_fullsized":"2023-10-06T21:19:25.835492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":320,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832687+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:19:25.832687+0000","last_peered":"2023-10-06T21:19:25.832687+0000","last_clean":"2023-10-06T21:19:25.832687+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:19:25.832687+0000","last_undegraded":"2023-10-06T21:19:25.832687+0000","last_fullsized":"2023-10-06T21:19:25.832687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837150+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:19:25.837150+0000","last_peered":"2023-10-06T21:19:25.837150+0000","last_clean":"2023-10-06T21:19:25.837150+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:19:25.837150+0000","last_undegraded":"2023-10-06T21:19:25.837150+0000","last_fullsized":"2023-10-06T21:19:25.837150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.865423+0000","last_change":"2023-10-06T21:19:26.864852+0000","last_active":"2023-10-06T21:19:26.865423+0000","last_peered":"2023-10-06T21:19:26.865423+0000","last_clean":"2023-10-06T21:19:26.865423+0000","last_became_active":"2023-10-06T21:19:26.846795+0000","last_became_peered":"2023-10-06T21:19:26.846795+0000","last_unstale":"2023-10-06T21:19:26.865423+0000","last_undegraded":"2023-10-06T21:19:26.865423+0000","last_fullsized":"2023-10-06T21:19:26.865423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:15:22.977641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838087+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:19:25.838087+0000","last_peered":"2023-10-06T21:19:25.838087+0000","last_clean":"2023-10-06T21:19:25.838087+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:19:25.838087+0000","last_undegraded":"2023-10-06T21:19:25.838087+0000","last_fullsized":"2023-10-06T21:19:25.838087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839228+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:19:25.839228+0000","last_peered":"2023-10-06T21:19:25.839228+0000","last_clean":"2023-10-06T21:19:25.839228+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:19:25.839228+0000","last_undegraded":"2023-10-06T21:19:25.839228+0000","last_fullsized":"2023-10-06T21:19:25.839228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839860+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:19:25.839860+0000","last_peered":"2023-10-06T21:19:25.839860+0000","last_clean":"2023-10-06T21:19:25.839860+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:19:25.839860+0000","last_undegraded":"2023-10-06T21:19:25.839860+0000","last_fullsized":"2023-10-06T21:19:25.839860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841073+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:19:25.841073+0000","last_peered":"2023-10-06T21:19:25.841073+0000","last_clean":"2023-10-06T21:19:25.841073+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:19:25.841073+0000","last_undegraded":"2023-10-06T21:19:25.841073+0000","last_fullsized":"2023-10-06T21:19:25.841073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835559+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:19:25.835559+0000","last_peered":"2023-10-06T21:19:25.835559+0000","last_clean":"2023-10-06T21:19:25.835559+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:19:25.835559+0000","last_undegraded":"2023-10-06T21:19:25.835559+0000","last_fullsized":"2023-10-06T21:19:25.835559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.841631+0000","last_change":"2023-10-06T21:19:26.841631+0000","last_active":"2023-10-06T21:19:26.841631+0000","last_peered":"2023-10-06T21:19:26.841631+0000","last_clean":"2023-10-06T21:19:26.841631+0000","last_became_active":"2023-10-06T21:19:26.841352+0000","last_became_peered":"2023-10-06T21:19:26.841352+0000","last_unstale":"2023-10-06T21:19:26.841631+0000","last_undegraded":"2023-10-06T21:19:26.841631+0000","last_fullsized":"2023-10-06T21:19:26.841631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:26:07.482368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843282+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:19:25.843282+0000","last_peered":"2023-10-06T21:19:25.843282+0000","last_clean":"2023-10-06T21:19:25.843282+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:19:25.843282+0000","last_undegraded":"2023-10-06T21:19:25.843282+0000","last_fullsized":"2023-10-06T21:19:25.843282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.837413+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:19:25.837413+0000","last_peered":"2023-10-06T21:19:25.837413+0000","last_clean":"2023-10-06T21:19:25.837413+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:19:25.837413+0000","last_undegraded":"2023-10-06T21:19:25.837413+0000","last_fullsized":"2023-10-06T21:19:25.837413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835460+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:19:25.835460+0000","last_peered":"2023-10-06T21:19:25.835460+0000","last_clean":"2023-10-06T21:19:25.835460+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:19:25.835460+0000","last_undegraded":"2023-10-06T21:19:25.835460+0000","last_fullsized":"2023-10-06T21:19:25.835460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843427+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:19:25.843427+0000","last_peered":"2023-10-06T21:19:25.843427+0000","last_clean":"2023-10-06T21:19:25.843427+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:19:25.843427+0000","last_undegraded":"2023-10-06T21:19:25.843427+0000","last_fullsized":"2023-10-06T21:19:25.843427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839449+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:19:25.839449+0000","last_peered":"2023-10-06T21:19:25.839449+0000","last_clean":"2023-10-06T21:19:25.839449+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:19:25.839449+0000","last_undegraded":"2023-10-06T21:19:25.839449+0000","last_fullsized":"2023-10-06T21:19:25.839449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835663+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:19:25.835663+0000","last_peered":"2023-10-06T21:19:25.835663+0000","last_clean":"2023-10-06T21:19:25.835663+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:19:25.835663+0000","last_undegraded":"2023-10-06T21:19:25.835663+0000","last_fullsized":"2023-10-06T21:19:25.835663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838171+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:19:25.838171+0000","last_peered":"2023-10-06T21:19:25.838171+0000","last_clean":"2023-10-06T21:19:25.838171+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:19:25.838171+0000","last_undegraded":"2023-10-06T21:19:25.838171+0000","last_fullsized":"2023-10-06T21:19:25.838171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832919+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:19:25.832919+0000","last_peered":"2023-10-06T21:19:25.832919+0000","last_clean":"2023-10-06T21:19:25.832919+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:19:25.832919+0000","last_undegraded":"2023-10-06T21:19:25.832919+0000","last_fullsized":"2023-10-06T21:19:25.832919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:46.934695+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:19:46.934695+0000","last_peered":"2023-10-06T21:19:46.934695+0000","last_clean":"2023-10-06T21:19:46.934695+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:19:46.934695+0000","last_undegraded":"2023-10-06T21:19:46.934695+0000","last_fullsized":"2023-10-06T21:19:46.934695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833245+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:19:25.833245+0000","last_peered":"2023-10-06T21:19:25.833245+0000","last_clean":"2023-10-06T21:19:25.833245+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:19:25.833245+0000","last_undegraded":"2023-10-06T21:19:25.833245+0000","last_fullsized":"2023-10-06T21:19:25.833245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843021+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:19:25.843021+0000","last_peered":"2023-10-06T21:19:25.843021+0000","last_clean":"2023-10-06T21:19:25.843021+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:19:25.843021+0000","last_undegraded":"2023-10-06T21:19:25.843021+0000","last_fullsized":"2023-10-06T21:19:25.843021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833228+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:19:25.833228+0000","last_peered":"2023-10-06T21:19:25.833228+0000","last_clean":"2023-10-06T21:19:25.833228+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:19:25.833228+0000","last_undegraded":"2023-10-06T21:19:25.833228+0000","last_fullsized":"2023-10-06T21:19:25.833228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.851061+0000","last_change":"2023-10-06T21:19:26.851061+0000","last_active":"2023-10-06T21:19:26.851061+0000","last_peered":"2023-10-06T21:19:26.851061+0000","last_clean":"2023-10-06T21:19:26.851061+0000","last_became_active":"2023-10-06T21:19:26.845797+0000","last_became_peered":"2023-10-06T21:19:26.845797+0000","last_unstale":"2023-10-06T21:19:26.851061+0000","last_undegraded":"2023-10-06T21:19:26.851061+0000","last_fullsized":"2023-10-06T21:19:26.851061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:48.212793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.833210+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:19:25.833210+0000","last_peered":"2023-10-06T21:19:25.833210+0000","last_clean":"2023-10-06T21:19:25.833210+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:19:25.833210+0000","last_undegraded":"2023-10-06T21:19:25.833210+0000","last_fullsized":"2023-10-06T21:19:25.833210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843522+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:19:25.843522+0000","last_peered":"2023-10-06T21:19:25.843522+0000","last_clean":"2023-10-06T21:19:25.843522+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:19:25.843522+0000","last_undegraded":"2023-10-06T21:19:25.843522+0000","last_fullsized":"2023-10-06T21:19:25.843522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.844504+0000","last_change":"2023-10-06T21:19:26.844504+0000","last_active":"2023-10-06T21:19:26.844504+0000","last_peered":"2023-10-06T21:19:26.844504+0000","last_clean":"2023-10-06T21:19:26.844504+0000","last_became_active":"2023-10-06T21:19:26.844105+0000","last_became_peered":"2023-10-06T21:19:26.844105+0000","last_unstale":"2023-10-06T21:19:26.844504+0000","last_undegraded":"2023-10-06T21:19:26.844504+0000","last_fullsized":"2023-10-06T21:19:26.844504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:08.655868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839315+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:19:25.839315+0000","last_peered":"2023-10-06T21:19:25.839315+0000","last_clean":"2023-10-06T21:19:25.839315+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:19:25.839315+0000","last_undegraded":"2023-10-06T21:19:25.839315+0000","last_fullsized":"2023-10-06T21:19:25.839315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.832281+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:19:25.832281+0000","last_peered":"2023-10-06T21:19:25.832281+0000","last_clean":"2023-10-06T21:19:25.832281+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:19:25.832281+0000","last_undegraded":"2023-10-06T21:19:25.832281+0000","last_fullsized":"2023-10-06T21:19:25.832281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.839150+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:19:25.839150+0000","last_peered":"2023-10-06T21:19:25.839150+0000","last_clean":"2023-10-06T21:19:25.839150+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:19:25.839150+0000","last_undegraded":"2023-10-06T21:19:25.839150+0000","last_fullsized":"2023-10-06T21:19:25.839150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841475+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:19:25.841475+0000","last_peered":"2023-10-06T21:19:25.841475+0000","last_clean":"2023-10-06T21:19:25.841475+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:19:25.841475+0000","last_undegraded":"2023-10-06T21:19:25.841475+0000","last_fullsized":"2023-10-06T21:19:25.841475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840949+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:19:25.840949+0000","last_peered":"2023-10-06T21:19:25.840949+0000","last_clean":"2023-10-06T21:19:25.840949+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:19:25.840949+0000","last_undegraded":"2023-10-06T21:19:25.840949+0000","last_fullsized":"2023-10-06T21:19:25.840949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.840977+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:19:25.840977+0000","last_peered":"2023-10-06T21:19:25.840977+0000","last_clean":"2023-10-06T21:19:25.840977+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:19:25.840977+0000","last_undegraded":"2023-10-06T21:19:25.840977+0000","last_fullsized":"2023-10-06T21:19:25.840977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T21:19:26.847714+0000","last_change":"2023-10-06T21:19:26.847714+0000","last_active":"2023-10-06T21:19:26.847714+0000","last_peered":"2023-10-06T21:19:26.847714+0000","last_clean":"2023-10-06T21:19:26.847714+0000","last_became_active":"2023-10-06T21:19:26.847302+0000","last_became_peered":"2023-10-06T21:19:26.847302+0000","last_unstale":"2023-10-06T21:19:26.847714+0000","last_undegraded":"2023-10-06T21:19:26.847714+0000","last_fullsized":"2023-10-06T21:19:26.847714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:10:24.079203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.835340+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:19:25.835340+0000","last_peered":"2023-10-06T21:19:25.835340+0000","last_clean":"2023-10-06T21:19:25.835340+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:19:25.835340+0000","last_undegraded":"2023-10-06T21:19:25.835340+0000","last_fullsized":"2023-10-06T21:19:25.835340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.843439+0000","last_change":"2023-10-06T21:18:54.859301+0000","last_active":"2023-10-06T21:19:25.843439+0000","last_peered":"2023-10-06T21:19:25.843439+0000","last_clean":"2023-10-06T21:19:25.843439+0000","last_became_active":"2023-10-06T21:18:54.858936+0000","last_became_peered":"2023-10-06T21:18:54.858936+0000","last_unstale":"2023-10-06T21:19:25.843439+0000","last_undegraded":"2023-10-06T21:19:25.843439+0000","last_fullsized":"2023-10-06T21:19:25.843439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:29:35.239801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.838994+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:19:25.838994+0000","last_peered":"2023-10-06T21:19:25.838994+0000","last_clean":"2023-10-06T21:19:25.838994+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:19:25.838994+0000","last_undegraded":"2023-10-06T21:19:25.838994+0000","last_fullsized":"2023-10-06T21:19:25.838994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:19:25.841157+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:19:25.841157+0000","last_peered":"2023-10-06T21:19:25.841157+0000","last_clean":"2023-10-06T21:19:25.841157+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:19:25.841157+0000","last_undegraded":"2023-10-06T21:19:25.841157+0000","last_fullsized":"2023-10-06T21:19:25.841157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65793,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":538,"num_read_kb":341,"num_write":234,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197379,"data_compressed":5511,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460246,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.749},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.89600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.996}]},{"osd":10,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.92700000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612639,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1110209,"data_compressed":43346,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.626},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.81499999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.81000000000000005,"15min":0.79100000000000004},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.054,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80800000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.86599999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1000000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.96899999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.069,"5min":1.532,"15min":1.532},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.052,"5min":0.98799999999999999,"15min":0.97699999999999998},"min":{"1min":0.60299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.5300000000000002,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.995}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2070000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.071},{"interface":"front","average":{"1min":0.878,"5min":0.96899999999999997,"15min":0.98399999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1140000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0229999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.151,"15min":1.1020000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.827999999999999,"5min":10.827999999999999,"15min":10.827999999999999},"last":1.04},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.1020000000000001,"15min":1.0589999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":10.808999999999999,"5min":10.808999999999999,"15min":10.808999999999999},"last":0.747}]},{"osd":11,"last update":"Fri Oct 6 21:19:01 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8479999999999999,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0089999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":460064,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73499999999999999,"5min":0.749,"15min":0.749},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":1.131,"15min":1.131},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93899999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85099999999999998,"5min":1.097,"15min":1.097},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":1.21,"15min":1.21},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2030000000000001,"5min":1.242,"15min":1.242},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.90600000000000003},"min":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94799999999999995,"5min":1.222,"15min":1.222},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.98499999999999999,"5min":1.288,"15min":1.288},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99099999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.90300000000000002,"15min":0.93300000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0840000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.433},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.93000000000000005,"15min":0.95599999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.036,"15min":1.0600000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0880000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.016,"15min":1.04},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.621}]},{"osd":8,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.03,"15min":1.0580000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.248,"5min":1.379,"15min":1.379},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.028,"15min":1.056},"min":{"1min":0.69599999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1020000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1779999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.0369999999999999,"5min":1.575,"15min":1.575},"last":0.754}]}]},{"osd":8,"up_from":60,"seq":257698037800,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459999,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.751},{"interface":"front","average":{"1min":0.874,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.878}]},{"osd":2,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.749,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.196,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.60199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.167,"5min":0.92100000000000004,"15min":0.89000000000000001},"min":{"1min":0.58399999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.63100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2450000000000001,"5min":1.468,"15min":1.468},"last":1.008},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.89800000000000002,"15min":0.88900000000000001},"min":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.266,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0449999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.306,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.67300000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.0800000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85899999999999999,"15min":0.84399999999999997},"min":{"1min":0.62,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.048,"5min":0.92300000000000004,"15min":0.90200000000000002},"min":{"1min":0.73299999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.1220000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1052634,"data_compressed":41509,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.45900000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":1.044,"15min":1.044},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94499999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.627},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.376,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91300000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.073,"15min":1.073},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81399999999999995,"15min":0.77200000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.05,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82299999999999995,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.018,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75}]},{"osd":8,"last update":"Fri Oct 6 21:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":1.0469999999999999},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.119,"15min":1.119},"last":0.88800000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.88700000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.149,"5min":1.198,"15min":1.198},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.85299999999999998,"15min":0.85199999999999998},"min":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.98499999999999999,"5min":1.034,"15min":1.034},"last":0.76900000000000002}]},{"osd":10,"last update":"Fri Oct 6 21:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.119,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459787,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442}]},{"osd":1,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0720000000000001,"5min":1.121,"15min":1.121},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.81200000000000006,"5min":1.887,"15min":1.887},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.441},"max":{"1min":1.0469999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.84699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.92800000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.034,"5min":1.097,"15min":1.097},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80000000000000004,"15min":0.79200000000000004},"min":{"1min":0.66400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0509999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":5.6870000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.88600000000000001,"15min":0.91800000000000004},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98399999999999999,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97399999999999998,"5min":1.827,"15min":1.827},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.56699999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":5.6639999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.871,"15min":0.85999999999999999},"min":{"1min":0.68100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3280000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.874}]},{"osd":8,"last update":"Fri Oct 6 21:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.399,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.997,"5min":1.905,"15min":1.905},"last":5.7110000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.89500000000000002,"15min":0.90000000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.361,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.01,"5min":1.165,"15min":1.165},"last":0.77700000000000002}]},{"osd":10,"last update":"Fri Oct 6 21:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.073,"5min":1.232,"15min":1.232},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0009999999999999,"15min":1.0049999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.93300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":519675,"data_compressed":7614,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.016,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.69199999999999995,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.4390000000000001,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.48199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.96199999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":6.3419999999999996,"5min":6.3419999999999996,"15min":6.3419999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.60999999999999999,"15min":0.56000000000000005},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93700000000000006,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.59099999999999997,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.72299999999999998,"15min":0.64700000000000002},"min":{"1min":0.57199999999999995,"5min":0.433,"15min":0.433},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.079},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.73699999999999999,"15min":0.66200000000000003},"min":{"1min":0.66500000000000004,"5min":0.435,"15min":0.435},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.79000000000000004,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0910000000000002,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.76100000000000001},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":0.76100000000000001,"15min":0.68799999999999994},"min":{"1min":0.48799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":8.7620000000000005,"5min":8.7620000000000005,"15min":8.7620000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":0.78100000000000003,"15min":0.69099999999999995},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.4410000000000007,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":0.89700000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:19:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.83499999999999996,"15min":0.78900000000000003},"min":{"1min":0.73999999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.92700000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":0.877,"15min":0.82999999999999996},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":7.1760000000000002,"5min":7.1760000000000002,"15min":7.1760000000000002},"last":0.879},{"interface":"front","average":{"1min":1.038,"5min":0.89400000000000002,"15min":0.87},"min":{"1min":0.627,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.98099999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":0.93799999999999994,"15min":0.89400000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":6.0419999999999998,"5min":6.0419999999999998,"15min":6.0419999999999998},"last":0.997},{"interface":"front","average":{"1min":1.016,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.0980000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461120,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":2.6030000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.52900000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0580000000000001,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.873},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.012,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.45100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.502},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.99299999999999999,"5min":1.048,"15min":1.048},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.65700000000000003},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.091,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.433},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.67500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.85799999999999998,"15min":0.83499999999999996},"min":{"1min":0.65500000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.159,"5min":1.228,"15min":1.228},"last":0.90900000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.91200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.749}]},{"osd":10,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94099999999999995,"15min":0.95699999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.286,"15min":1.286},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.999,"5min":1,"15min":1},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:18:59 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":519622,"data_compressed":7614,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63400000000000001,"15min":0.57099999999999995},"min":{"1min":0.48799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.92200000000000004,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.70899999999999996,"15min":0.61499999999999999},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.65400000000000003,"15min":0.60799999999999998},"min":{"1min":0.48999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.044,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.877}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.997}]},{"osd":4,"last update":"Fri Oct 6 21:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.56899999999999995,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.056,"15min":1.056},"last":0.97799999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.123,"5min":1.298,"15min":1.298},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1220000000000001,"5min":1.266,"15min":1.266},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.88600000000000001,"15min":0.878},"min":{"1min":0.69799999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.198,"5min":1.266,"15min":1.266},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.85699999999999998,"15min":0.85599999999999998},"min":{"1min":0.68799999999999994,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.135,"5min":1.194,"15min":1.194},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.93500000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2490000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.071}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459618,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45000000000000001,"15min":0.40999999999999998},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71299999999999997,"15min":0.65400000000000003},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":1.0009999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.75,"15min":0.67200000000000004},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.1799999999999997,"5min":5.1799999999999997,"15min":5.1799999999999997},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.67300000000000004},"min":{"1min":0.56000000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.054,"5min":1.206,"15min":1.206},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.67100000000000004},"min":{"1min":0.54800000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:19:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.746,"15min":0.745},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.749},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.57799999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95099999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 21:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.26,"15min":1.26},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.077,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.96399999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1759999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.101},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.008,"5min":1.196,"15min":1.196},"last":0.80400000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.93400000000000005,"15min":0.94099999999999995},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.113,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0089999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":460032,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.36399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76300000000000001,"5min":0.874,"15min":0.874},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.42299999999999999},"min":{"1min":0.34699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":1.601,"15min":1.601},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65500000000000003,"15min":0.629},"min":{"1min":0.52600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84499999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70299999999999996,"15min":0.65000000000000002},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86499999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.77600000000000002,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.028,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.758,"15min":1.758},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.93400000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.46800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80900000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0629999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86799999999999999,"15min":0.84999999999999998},"min":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.96699999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.90300000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.044},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92400000000000004},"min":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.113,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.0589999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.89900000000000002,"15min":0.89700000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1052253,"data_compressed":41509,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.625,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.68700000000000006,"15min":0.63},"min":{"1min":0.56699999999999995,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0820000000000001,"5min":1.089,"15min":1.089},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51100000000000001,"15min":0.499},"min":{"1min":0.38700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74399999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.88600000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.5}]},{"osd":3,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68000000000000005,"15min":0.66200000000000003},"min":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.623,"5min":0.442,"15min":0.442},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74099999999999999,"15min":0.70399999999999996},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.157,"5min":1.196,"15min":1.196},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89400000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.84299999999999997,"15min":0.83499999999999996},"min":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.78400000000000003,"15min":0.76600000000000001},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.86099999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0369999999999999,"5min":1.339,"15min":1.339},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.871,"15min":0.876},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1040000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.874,"15min":0.85799999999999998},"min":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.083,"5min":1.262,"15min":1.262},"last":0.89900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60733,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60953,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61107,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:19:53.314 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-06T21:19:53.314 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T21:19:53.314 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-06T21:19:53.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-06T21:19:54.055 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:19:54.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:53 smithi142 ceph-mon[102951]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:54.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:53 smithi031 ceph-mon[98539]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:53 smithi146 ceph-mon[102809]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:19:54.906 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:54 smithi031 ceph-mon[98539]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:55.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:54 smithi142 ceph-mon[102951]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:54 smithi146 ceph-mon[102809]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:55.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 -- 172.21.15.31:0/3249017868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 msgr2=0x7f56d0107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:55.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 --2- 172.21.15.31:0/3249017868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d0107370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56c00099c0 tx=0x7f56c0009cd0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 -- 172.21.15.31:0/3249017868 shutdown_connections 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 --2- 172.21.15.31:0/3249017868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0101940 0x7f56d010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 --2- 172.21.15.31:0/3249017868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d0107370 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 --2- 172.21.15.31:0/3249017868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0100f80 0x7f56d0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 -- 172.21.15.31:0/3249017868 >> 172.21.15.31:0/3249017868 conn(0x7f56d00780a0 msgr2=0x7f56d00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 -- 172.21.15.31:0/3249017868 shutdown_connections 2023-10-06T21:19:55.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.543+0000 7f56d6250700 1 -- 172.21.15.31:0/3249017868 wait complete. 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.544+0000 7f56d6250700 1 Processor -- start 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.544+0000 7f56d6250700 1 -- start start 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0100f80 0x7f56d01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 0x7f56d0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d019e080 con 0x7f56d0106f90 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d019e1f0 con 0x7f56d0101940 2023-10-06T21:19:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56d6250700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d019e3b0 con 0x7f56d0100f80 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42870/0 (socket says 172.21.15.31:42870) 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 -- 172.21.15.31:0/1408537516 learned_addr learned my addr 172.21.15.31:0/1408537516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0100f80 0x7f56d01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56ceffd700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 0x7f56d0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 -- 172.21.15.31:0/1408537516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0100f80 msgr2=0x7f56d01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0100f80 0x7f56d01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 -- 172.21.15.31:0/1408537516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 msgr2=0x7f56d0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 0x7f56d0199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56cffff700 1 -- 172.21.15.31:0/1408537516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00095f0 con 0x7f56d0106f90 2023-10-06T21:19:55.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.545+0000 7f56ceffd700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 0x7f56d0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56cffff700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007f10 tx=0x7f56c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8017070 con 0x7f56d0106f90 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d019e690 con 0x7f56d0106f90 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56c800f040 con 0x7f56d0106f90 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8005710 con 0x7f56d0106f90 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.546+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d01a5ef0 con 0x7f56d0106f90 2023-10-06T21:19:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.547+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d004ebb0 con 0x7f56d0106f90 2023-10-06T21:19:55.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.549+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f56c8005870 con 0x7f56d0106f90 2023-10-06T21:19:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.550+0000 7f56ccff9700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 0x7f56b80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.551+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56c80cc640 con 0x7f56d0106f90 2023-10-06T21:19:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.551+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 0x7f56b80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.551+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 0x7f56b80a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006010 tx=0x7f56c0003680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.555+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f56c8095ac0 con 0x7f56d0106f90 2023-10-06T21:19:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.809+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f56d019a420 con 0x7f56d0106f90 2023-10-06T21:19:55.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.809+0000 7f56ccff9700 1 -- 172.21.15.31:0/1408537516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f56c80990e0 con 0x7f56d0106f90 2023-10-06T21:19:55.813 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:55.813 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T21:19:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 msgr2=0x7f56b80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 0x7f56b80a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006010 tx=0x7f56c0003680 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 msgr2=0x7f56d019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007f10 tx=0x7f56c800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 shutdown_connections 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f56b80a1590 0x7f56b80a3a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d0106f90 0x7f56d019da70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56d0100f80 0x7f56d01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 --2- 172.21.15.31:0/1408537516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56d0101940 0x7f56d0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 >> 172.21.15.31:0/1408537516 conn(0x7f56d00780a0 msgr2=0x7f56d010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 shutdown_connections 2023-10-06T21:19:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:55.812+0000 7f56d6250700 1 -- 172.21.15.31:0/1408537516 wait complete. 2023-10-06T21:19:56.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:55 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1408537516' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:19:56.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:55 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1408537516' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:19:56.241 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-06T21:19:56.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:19:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:55 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1408537516' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:19:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 -- 172.21.15.31:0/155890125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0102a40 msgr2=0x7f7cc0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/155890125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0102a40 0x7f7cc0109360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b0a0 tx=0x7f7cb800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 -- 172.21.15.31:0/155890125 shutdown_connections 2023-10-06T21:19:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/155890125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0102a40 0x7f7cc0109360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/155890125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0100ea0 0x7f7cc01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/155890125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 -- 172.21.15.31:0/155890125 >> 172.21.15.31:0/155890125 conn(0x7f7cc00fc420 msgr2=0x7f7cc00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 -- 172.21.15.31:0/155890125 shutdown_connections 2023-10-06T21:19:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.457+0000 7f7cc67ff700 1 -- 172.21.15.31:0/155890125 wait complete. 2023-10-06T21:19:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 Processor -- start 2023-10-06T21:19:56.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 -- start start 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 0x7f7cc019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019e780 con 0x7f7cc0100ea0 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019e8f0 con 0x7f7cc0102a40 2023-10-06T21:19:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.459+0000 7f7cc67ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019eab0 con 0x7f7cc0102080 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42894/0 (socket says 172.21.15.31:42894) 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 -- 172.21.15.31:0/750808382 learned_addr learned my addr 172.21.15.31:0/750808382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbf7fe700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 -- 172.21.15.31:0/750808382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 msgr2=0x7f7cc0197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cc4d9c700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 0x7f7cc019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 -- 172.21.15.31:0/750808382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 msgr2=0x7f7cc019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 0x7f7cc019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 -- 172.21.15.31:0/750808382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb8009d00 con 0x7f7cc0100ea0 2023-10-06T21:19:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbf7fe700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cc4d9c700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 0x7f7cc019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.460+0000 7f7cbffff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000c990 tx=0x7f7cb000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.461+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0007b00 con 0x7f7cc0100ea0 2023-10-06T21:19:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.461+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc019ed90 con 0x7f7cc0100ea0 2023-10-06T21:19:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.461+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cb0007c60 con 0x7f7cc0100ea0 2023-10-06T21:19:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.461+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0018660 con 0x7f7cc0100ea0 2023-10-06T21:19:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.461+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc019f1b0 con 0x7f7cc0100ea0 2023-10-06T21:19:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.462+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc0108460 con 0x7f7cc0100ea0 2023-10-06T21:19:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.464+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f7cb000f450 con 0x7f7cc0100ea0 2023-10-06T21:19:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.466+0000 7f7cbd7fa700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 0x7f7ca80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.466+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7cb00ccef0 con 0x7f7cc0100ea0 2023-10-06T21:19:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.466+0000 7f7cbf7fe700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 0x7f7ca80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.467+0000 7f7cbf7fe700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 0x7f7ca80a3a10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0100bc0 tx=0x7f7cb400b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:56.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.469+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7cb0096370 con 0x7f7cc0100ea0 2023-10-06T21:19:56.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.687+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cc010c610 con 0x7f7cc0100ea0 2023-10-06T21:19:56.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.688+0000 7f7cbd7fa700 1 -- 172.21.15.31:0/750808382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22700 (secure 0 0 0) 0x7f7cb0099990 con 0x7f7cc0100ea0 2023-10-06T21:19:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:56.692 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:19:26.829028+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:19:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 msgr2=0x7f7ca80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 0x7f7ca80a3a10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0100bc0 tx=0x7f7cb400b5f0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 msgr2=0x7f7cc0196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000c990 tx=0x7f7cb000cd50 comp rx=0 tx=0).stop 2023-10-06T21:19:56.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 shutdown_connections 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f7ca80a1550 0x7f7ca80a3a10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc0102a40 0x7f7cc019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cc0100ea0 0x7f7cc0196c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 --2- 172.21.15.31:0/750808382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cc0102080 0x7f7cc0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 >> 172.21.15.31:0/750808382 conn(0x7f7cc00fc420 msgr2=0x7f7cc01055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 shutdown_connections 2023-10-06T21:19:56.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.691+0000 7f7cc67ff700 1 -- 172.21.15.31:0/750808382 wait complete. 2023-10-06T21:19:56.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:19:56.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 -- 172.21.15.31:0/3598764972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404102070 msgr2=0x7fa4041024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 --2- 172.21.15.31:0/3598764972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404102070 0x7fa4041024f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009990 tx=0x7fa3f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 -- 172.21.15.31:0/3598764972 shutdown_connections 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 --2- 172.21.15.31:0/3598764972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102a30 0x7fa404109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 --2- 172.21.15.31:0/3598764972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404100e90 0x7fa4041012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 --2- 172.21.15.31:0/3598764972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404102070 0x7fa4041024f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 -- 172.21.15.31:0/3598764972 >> 172.21.15.31:0/3598764972 conn(0x7fa4040fc430 msgr2=0x7fa4040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 -- 172.21.15.31:0/3598764972 shutdown_connections 2023-10-06T21:19:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.918+0000 7fa40adff700 1 -- 172.21.15.31:0/3598764972 wait complete. 2023-10-06T21:19:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.919+0000 7fa40adff700 1 Processor -- start 2023-10-06T21:19:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 -- start start 2023-10-06T21:19:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 0x7fa4041992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 0x7fa4041a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041a08e0 con 0x7fa404100e90 2023-10-06T21:19:56.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041a0a50 con 0x7fa404102a30 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa40adff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041a0c10 con 0x7fa404102070 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa408b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa408b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42912/0 (socket says 172.21.15.31:42912) 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.920+0000 7fa408b9b700 1 -- 172.21.15.31:0/789593030 learned_addr learned my addr 172.21.15.31:0/789593030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa40939c700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 0x7fa4041a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa403fff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 0x7fa4041992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 -- 172.21.15.31:0/789593030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 msgr2=0x7fa4041992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 0x7fa4041992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 -- 172.21.15.31:0/789593030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 msgr2=0x7fa4041a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 0x7fa4041a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 -- 172.21.15.31:0/789593030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00095f0 con 0x7fa404100e90 2023-10-06T21:19:56.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa40939c700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 0x7fa4041a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa403fff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 0x7fa4041992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa408b9b700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d980 tx=0x7fa3f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800de60 con 0x7fa404100e90 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f80044b0 con 0x7fa404100e90 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4041a0ef0 con 0x7fa404100e90 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8003b40 con 0x7fa404100e90 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.921+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4041a1310 con 0x7fa404100e90 2023-10-06T21:19:56.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.922+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40410c600 con 0x7fa404100e90 2023-10-06T21:19:56.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.924+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa3f8003ca0 con 0x7fa404100e90 2023-10-06T21:19:56.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.927+0000 7fa401ffb700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 0x7fa3f40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:56.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.927+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa3f80ccc00 con 0x7fa404100e90 2023-10-06T21:19:56.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.927+0000 7fa403fff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 0x7fa3f40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:56.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.927+0000 7fa403fff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 0x7fa3f40a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00098e0 tx=0x7fa3f0004740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:56.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:56.931+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa3f8096080 con 0x7fa404100e90 2023-10-06T21:19:57.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:56 smithi031 ceph-mon[98539]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:57.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:56 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/750808382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:57.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.141+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4041a1560 con 0x7fa404100e90 2023-10-06T21:19:57.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.142+0000 7fa401ffb700 1 -- 172.21.15.31:0/789593030 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3f8009940 con 0x7fa404100e90 2023-10-06T21:19:57.145 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.144+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 msgr2=0x7fa3f40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.144+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 0x7fa3f40a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00098e0 tx=0x7fa3f0004740 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.144+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 msgr2=0x7fa404198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.144+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d980 tx=0x7fa3f800dd40 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 shutdown_connections 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3f40a1510 0x7fa3f40a39d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa404102a30 0x7fa4041a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa404100e90 0x7fa404198da0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 --2- 172.21.15.31:0/789593030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa404102070 0x7fa4041992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 >> 172.21.15.31:0/789593030 conn(0x7fa4040fc430 msgr2=0x7fa4040fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 shutdown_connections 2023-10-06T21:19:57.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.145+0000 7fa40adff700 1 -- 172.21.15.31:0/789593030 wait complete. 2023-10-06T21:19:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:56 smithi142 ceph-mon[102951]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:57.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:56 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/750808382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:57.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:19:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:56 smithi146 ceph-mon[102809]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-06T21:19:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:56 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/750808382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:19:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 -- 172.21.15.31:0/2062553273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018102070 msgr2=0x7f30181024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 --2- 172.21.15.31:0/2062553273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018102070 0x7f30181024f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3000009990 tx=0x7f3000009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 -- 172.21.15.31:0/2062553273 shutdown_connections 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 --2- 172.21.15.31:0/2062553273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f3018109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 --2- 172.21.15.31:0/2062553273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018100e90 0x7f30181012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 --2- 172.21.15.31:0/2062553273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018102070 0x7f30181024f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 -- 172.21.15.31:0/2062553273 >> 172.21.15.31:0/2062553273 conn(0x7f30180fc430 msgr2=0x7f30180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 -- 172.21.15.31:0/2062553273 shutdown_connections 2023-10-06T21:19:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.369+0000 7f301d9ff700 1 -- 172.21.15.31:0/2062553273 wait complete. 2023-10-06T21:19:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.370+0000 7f301d9ff700 1 Processor -- start 2023-10-06T21:19:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 -- start start 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 0x7f30180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f30180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018073f80 con 0x7f3018100e90 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180740f0 con 0x7f3018102070 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f301d9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181a0ba0 con 0x7f3018102a30 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f300effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f300effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42916/0 (socket says 172.21.15.31:42916) 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f300effd700 1 -- 172.21.15.31:0/167346505 learned_addr learned my addr 172.21.15.31:0/167346505 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.371+0000 7f3016ffd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 0x7f30180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f30177fe700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f30180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 -- 172.21.15.31:0/167346505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 msgr2=0x7f30180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f30180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 -- 172.21.15.31:0/167346505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 msgr2=0x7f30180732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 0x7f30180732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 -- 172.21.15.31:0/167346505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000095f0 con 0x7f3018100e90 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f30177fe700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f30180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f3016ffd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 0x7f30180732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f300effd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f300800d980 tx=0x7f300800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800de60 con 0x7f3018100e90 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30080044b0 con 0x7f3018100e90 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30181a0e80 con 0x7f3018100e90 2023-10-06T21:19:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008003b40 con 0x7f3018100e90 2023-10-06T21:19:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.372+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30181a12a0 con 0x7f3018100e90 2023-10-06T21:19:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.373+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301810c620 con 0x7f3018100e90 2023-10-06T21:19:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.375+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3008003ca0 con 0x7f3018100e90 2023-10-06T21:19:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.377+0000 7f301c9fd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 0x7f30040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.378+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f30080ccc00 con 0x7f3018100e90 2023-10-06T21:19:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.378+0000 7f3016ffd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 0x7f30040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.378+0000 7f3016ffd700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 0x7f30040a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f300000b5c0 tx=0x7f3000005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.382+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3008096080 con 0x7f3018100e90 2023-10-06T21:19:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.592+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3018107f40 con 0x7f3018100e90 2023-10-06T21:19:57.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.593+0000 7f301c9fd700 1 -- 172.21.15.31:0/167346505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3008009940 con 0x7f3018100e90 2023-10-06T21:19:57.596 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 msgr2=0x7f30040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 0x7f30040a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f300000b5c0 tx=0x7f3000005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 msgr2=0x7f3018074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f300800d980 tx=0x7f300800dd40 comp rx=0 tx=0).stop 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 shutdown_connections 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f30040a1510 0x7f30040a39d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3018102a30 0x7f30180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3018100e90 0x7f3018074c60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 --2- 172.21.15.31:0/167346505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3018102070 0x7f30180732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 >> 172.21.15.31:0/167346505 conn(0x7f30180fc430 msgr2=0x7f30180fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 shutdown_connections 2023-10-06T21:19:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.596+0000 7f301d9ff700 1 -- 172.21.15.31:0/167346505 wait complete. 2023-10-06T21:19:57.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:19:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 -- 172.21.15.31:0/2880030125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102a40 msgr2=0x7f2b0c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 --2- 172.21.15.31:0/2880030125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102a40 0x7f2b0c109360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400b0a0 tx=0x7f2b0400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 -- 172.21.15.31:0/2880030125 shutdown_connections 2023-10-06T21:19:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 --2- 172.21.15.31:0/2880030125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102a40 0x7f2b0c109360 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 --2- 172.21.15.31:0/2880030125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 --2- 172.21.15.31:0/2880030125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c102080 0x7f2b0c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 -- 172.21.15.31:0/2880030125 >> 172.21.15.31:0/2880030125 conn(0x7f2b0c0fc460 msgr2=0x7f2b0c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 -- 172.21.15.31:0/2880030125 shutdown_connections 2023-10-06T21:19:57.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.820+0000 7f2b109ff700 1 -- 172.21.15.31:0/2880030125 wait complete. 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.821+0000 7f2b109ff700 1 Processor -- start 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.821+0000 7f2b109ff700 1 -- start start 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.821+0000 7f2b109ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b109ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b109ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c102a40 0x7f2b0c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b109ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1a09a0 con 0x7f2b0c102080 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b109ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1a0ae0 con 0x7f2b0c100ea0 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b109ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1a0c70 con 0x7f2b0c102a40 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42938/0 (socket says 172.21.15.31:42938) 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 -- 172.21.15.31:0/351051253 learned_addr learned my addr 172.21.15.31:0/351051253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b0a59c700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b0ad9d700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c102a40 0x7f2b0c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 -- 172.21.15.31:0/351051253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c102a40 msgr2=0x7f2b0c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c102a40 0x7f2b0c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 -- 172.21.15.31:0/351051253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 msgr2=0x7f2b0c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b09d9b700 1 -- 172.21.15.31:0/351051253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b04009d00 con 0x7f2b0c102080 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.822+0000 7f2b0a59c700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2b09d9b700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000e970 tx=0x7f2b0000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00009790 con 0x7f2b0c102080 2023-10-06T21:19:57.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b0000c7e0 con 0x7f2b0c102080 2023-10-06T21:19:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0c1a0f50 con 0x7f2b0c102080 2023-10-06T21:19:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00003bd0 con 0x7f2b0c102080 2023-10-06T21:19:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.823+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0c1a13d0 con 0x7f2b0c102080 2023-10-06T21:19:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.824+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0c10c5c0 con 0x7f2b0c102080 2023-10-06T21:19:57.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.827+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2b0000c950 con 0x7f2b0c102080 2023-10-06T21:19:57.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.828+0000 7f2afb7fe700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 0x7f2af40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.828+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2b00014070 con 0x7f2b0c102080 2023-10-06T21:19:57.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.828+0000 7f2b0a59c700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 0x7f2af40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.829+0000 7f2b0a59c700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 0x7f2af40a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c101bf0 tx=0x7f2afc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:57.830+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b000966e0 con 0x7f2b0c102080 2023-10-06T21:19:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.043+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b0c1a1620 con 0x7f2b0c102080 2023-10-06T21:19:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.044+0000 7f2afb7fe700 1 -- 172.21.15.31:0/351051253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b00099d00 con 0x7f2b0c102080 2023-10-06T21:19:58.047 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:19:58.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 msgr2=0x7f2af40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 0x7f2af40a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c101bf0 tx=0x7f2afc00b410 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 msgr2=0x7f2b0c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000e970 tx=0x7f2b0000ec80 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 shutdown_connections 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2af40a1510 0x7f2af40a39d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b0c102a40 0x7f2b0c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b0c100ea0 0x7f2b0c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 --2- 172.21.15.31:0/351051253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c102080 0x7f2b0c199360 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.046+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 >> 172.21.15.31:0/351051253 conn(0x7f2b0c0fc460 msgr2=0x7f2b0c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.047+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 shutdown_connections 2023-10-06T21:19:58.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.047+0000 7f2b109ff700 1 -- 172.21.15.31:0/351051253 wait complete. 2023-10-06T21:19:58.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:19:58.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:57 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/789593030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:57 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/167346505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:57 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/789593030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:57 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/167346505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/3739083877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28102a30 msgr2=0x7f5f28109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/3739083877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28102a30 0x7f5f28109350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009f90 tx=0x7f5f2000b330 comp rx=0 tx=0).stop 2023-10-06T21:19:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/3739083877 shutdown_connections 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/3739083877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28102a30 0x7f5f28109350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/3739083877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28100e90 0x7f5f281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/3739083877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 0x7f5f281024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/3739083877 >> 172.21.15.31:0/3739083877 conn(0x7f5f280fc430 msgr2=0x7f5f280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/3739083877 shutdown_connections 2023-10-06T21:19:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.281+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/3739083877 wait complete. 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 Processor -- start 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 -- start start 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 0x7f5f280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28102a30 0x7f5f280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28073f80 con 0x7f5f28100e90 2023-10-06T21:19:58.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280740f0 con 0x7f5f28102070 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.282+0000 7f5f2f1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281a0c40 con 0x7f5f28102a30 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42966/0 (socket says 172.21.15.31:42966) 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 -- 172.21.15.31:0/1132958951 learned_addr learned my addr 172.21.15.31:0/1132958951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f27fff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 0x7f5f280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2d79c700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28102a30 0x7f5f280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 -- 172.21.15.31:0/1132958951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28102a30 msgr2=0x7f5f280737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28102a30 0x7f5f280737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 -- 172.21.15.31:0/1132958951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 msgr2=0x7f5f280732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 0x7f5f280732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 -- 172.21.15.31:0/1132958951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0095f0 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.283+0000 7f5f2cf9b700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ebf0 tx=0x7f5f1c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.284+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009a10 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.284+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f1c00cc90 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.284+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f20009c40 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.284+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c018680 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.284+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f281a1210 con 0x7f5f28100e90 2023-10-06T21:19:58.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.285+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2810c670 con 0x7f5f28100e90 2023-10-06T21:19:58.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.287+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5f1c010430 con 0x7f5f28100e90 2023-10-06T21:19:58.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.289+0000 7f5f25ffb700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 0x7f5f180a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.289+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f1c014070 con 0x7f5f28100e90 2023-10-06T21:19:58.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.289+0000 7f5f27fff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 0x7f5f180a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.290+0000 7f5f27fff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 0x7f5f180a3a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f28100bb0 tx=0x7f5f14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:58.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.292+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5f1c0968e0 con 0x7f5f28100e90 2023-10-06T21:19:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:57 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/789593030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:57 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/167346505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:19:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.558+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f2810c880 con 0x7f5f28100e90 2023-10-06T21:19:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.559+0000 7f5f25ffb700 1 -- 172.21.15.31:0/1132958951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f5f1c099f00 con 0x7f5f28100e90 2023-10-06T21:19:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:58.562 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:19:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.561+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 msgr2=0x7f5f180a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.561+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 0x7f5f180a3a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f28100bb0 tx=0x7f5f14009260 comp rx=0 tx=0).stop 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.561+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 msgr2=0x7f5f28074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.561+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ebf0 tx=0x7f5f1c00ef00 comp rx=0 tx=0).stop 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 shutdown_connections 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f180a15d0 0x7f5f180a3a90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f28102a30 0x7f5f280737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28100e90 0x7f5f28074c60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 --2- 172.21.15.31:0/1132958951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f28102070 0x7f5f280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 >> 172.21.15.31:0/1132958951 conn(0x7f5f280fc430 msgr2=0x7f5f280fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 shutdown_connections 2023-10-06T21:19:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.562+0000 7f5f2f1ff700 1 -- 172.21.15.31:0/1132958951 wait complete. 2023-10-06T21:19:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:19:58.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 -- 172.21.15.31:0/4016010722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600100ea0 msgr2=0x7f26001012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 --2- 172.21.15.31:0/4016010722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600100ea0 0x7f26001012c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 -- 172.21.15.31:0/4016010722 shutdown_connections 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 --2- 172.21.15.31:0/4016010722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2600102a40 0x7f2600109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 --2- 172.21.15.31:0/4016010722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600100ea0 0x7f26001012c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 --2- 172.21.15.31:0/4016010722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600102080 0x7f2600102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 -- 172.21.15.31:0/4016010722 >> 172.21.15.31:0/4016010722 conn(0x7f26000fc420 msgr2=0x7f26000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 -- 172.21.15.31:0/4016010722 shutdown_connections 2023-10-06T21:19:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.797+0000 7f2606dff700 1 -- 172.21.15.31:0/4016010722 wait complete. 2023-10-06T21:19:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.798+0000 7f2606dff700 1 Processor -- start 2023-10-06T21:19:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.798+0000 7f2606dff700 1 -- start start 2023-10-06T21:19:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600102080 0x7f26000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2600102a40 0x7f26000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2600073f80 con 0x7f2600100ea0 2023-10-06T21:19:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000740f0 con 0x7f2600102080 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2606dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26001a0b60 con 0x7f2600102a40 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2604b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2604b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42986/0 (socket says 172.21.15.31:42986) 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f2604b9b700 1 -- 172.21.15.31:0/3104902427 learned_addr learned my addr 172.21.15.31:0/3104902427 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.799+0000 7f25fffff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600102080 0x7f26000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 -- 172.21.15.31:0/3104902427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2600102a40 msgr2=0x7f26000737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2600102a40 0x7f26000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 -- 172.21.15.31:0/3104902427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600102080 msgr2=0x7f26000732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600102080 0x7f26000732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 -- 172.21.15.31:0/3104902427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009520 con 0x7f2600100ea0 2023-10-06T21:19:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2604b9b700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f25f40098e0 tx=0x7f25f4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f401d070 con 0x7f2600100ea0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25f400f460 con 0x7f2600100ea0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f40095f0 con 0x7f2600100ea0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4021670 con 0x7f2600100ea0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.800+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26001a1070 con 0x7f2600100ea0 2023-10-06T21:19:58.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.801+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2600108530 con 0x7f2600100ea0 2023-10-06T21:19:58.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.803+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f25f400f5d0 con 0x7f2600100ea0 2023-10-06T21:19:58.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.805+0000 7f25fdffb700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 0x7f25f00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:58.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.805+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f25f40ce390 con 0x7f2600100ea0 2023-10-06T21:19:58.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.805+0000 7f25fffff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 0x7f25f00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.806+0000 7f25fffff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 0x7f25f00a3a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2600100bc0 tx=0x7f25ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:58.807+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25f40978a0 con 0x7f2600100ea0 2023-10-06T21:19:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.052+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f260010c610 con 0x7f2600100ea0 2023-10-06T21:19:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.054+0000 7f25fdffb700 1 -- 172.21.15.31:0/3104902427 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22700 (secure 0 0 0) 0x7f25f4097030 con 0x7f2600100ea0 2023-10-06T21:19:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:19:59.057 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:19:26.829028+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:19:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.056+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 msgr2=0x7f25f00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.056+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 0x7f25f00a3a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2600100bc0 tx=0x7f25ec009260 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.056+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 msgr2=0x7f2600074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.056+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f25f40098e0 tx=0x7f25f4005ef0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 shutdown_connections 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f25f00a1550 0x7f25f00a3a10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2600102a40 0x7f26000737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2600100ea0 0x7f2600074c60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 --2- 172.21.15.31:0/3104902427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2600102080 0x7f26000732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 >> 172.21.15.31:0/3104902427 conn(0x7f26000fc420 msgr2=0x7f2600105670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 shutdown_connections 2023-10-06T21:19:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.057+0000 7f2606dff700 1 -- 172.21.15.31:0/3104902427 wait complete. 2023-10-06T21:19:59.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:19:59.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:19:59.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:58 smithi031 ceph-mon[98539]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:19:59.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:58 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/351051253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:19:59.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:58 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:58 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1132958951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:58 smithi142 ceph-mon[102951]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:19:59.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:58 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/351051253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:19:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:58 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:58 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1132958951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:58 smithi146 ceph-mon[102809]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:19:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:58 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/351051253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:19:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:58 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:58 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1132958951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 -- 172.21.15.31:0/1213505671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc1042b0 msgr2=0x7faffc1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 --2- 172.21.15.31:0/1213505671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc1042b0 0x7faffc1066a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faff0009990 tx=0x7faff0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 -- 172.21.15.31:0/1213505671 shutdown_connections 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 --2- 172.21.15.31:0/1213505671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 0x7faffc109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 --2- 172.21.15.31:0/1213505671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc101980 0x7faffc103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 --2- 172.21.15.31:0/1213505671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc1042b0 0x7faffc1066a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 -- 172.21.15.31:0/1213505671 >> 172.21.15.31:0/1213505671 conn(0x7faffc0fb370 msgr2=0x7faffc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 -- 172.21.15.31:0/1213505671 shutdown_connections 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.296+0000 7fb000fff700 1 -- 172.21.15.31:0/1213505671 wait complete. 2023-10-06T21:19:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 Processor -- start 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 -- start start 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc1042b0 0x7faffc0ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 0x7faffc0ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc100610 con 0x7faffc101980 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc100780 con 0x7faffc1042b0 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.297+0000 7fb000fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc1a0b80 con 0x7faffc106be0 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43000/0 (socket says 172.21.15.31:43000) 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 -- 172.21.15.31:0/21999565 learned_addr learned my addr 172.21.15.31:0/21999565 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffad9d700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 0x7faffc0ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faff9d9b700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc1042b0 0x7faffc0ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 -- 172.21.15.31:0/21999565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 msgr2=0x7faffc0ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 0x7faffc0ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 -- 172.21.15.31:0/21999565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc1042b0 msgr2=0x7faffc0ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc1042b0 0x7faffc0ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 -- 172.21.15.31:0/21999565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00095f0 con 0x7faffc101980 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7faffa59c700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9a0 tx=0x7fafec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec009790 con 0x7faffc101980 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc1a0e60 con 0x7faffc101980 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.298+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafec010430 con 0x7faffc101980 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.299+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00cea0 con 0x7faffc101980 2023-10-06T21:19:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.299+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc1a1280 con 0x7faffc101980 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.300+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc0fcef0 con 0x7faffc101980 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.302+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fafec0105a0 con 0x7faffc101980 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.303+0000 7fafeb7fe700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 0x7fafe40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.304+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fafec014070 con 0x7faffc101980 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.304+0000 7faff9d9b700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 0x7fafe40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.304+0000 7faff9d9b700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 0x7fafe40a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faff0005b00 tx=0x7faff0005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.307+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fafec052100 con 0x7faffc101980 2023-10-06T21:19:59.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.529+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faffc02d090 con 0x7faffc101980 2023-10-06T21:19:59.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.529+0000 7fafeb7fe700 1 -- 172.21.15.31:0/21999565 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafec096520 con 0x7faffc101980 2023-10-06T21:19:59.532 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:19:59.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 msgr2=0x7fafe40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 0x7fafe40a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faff0005b00 tx=0x7faff0005a90 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 msgr2=0x7faffc1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9a0 tx=0x7fafec00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 shutdown_connections 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fafe40a1510 0x7fafe40a39d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faffc106be0 0x7faffc0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faffc101980 0x7faffc1012f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 --2- 172.21.15.31:0/21999565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc1042b0 0x7faffc0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.531+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 >> 172.21.15.31:0/21999565 conn(0x7faffc0fb370 msgr2=0x7faffc1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:59.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.532+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 shutdown_connections 2023-10-06T21:19:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.532+0000 7fb000fff700 1 -- 172.21.15.31:0/21999565 wait complete. 2023-10-06T21:19:59.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:19:59.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 -- 172.21.15.31:0/2833829396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0074ef0 msgr2=0x7fa2f0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/2833829396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0073340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009990 tx=0x7fa2e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 -- 172.21.15.31:0/2833829396 shutdown_connections 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/2833829396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0108db0 0x7fa2f010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/2833829396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0073340 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/2833829396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 -- 172.21.15.31:0/2833829396 >> 172.21.15.31:0/2833829396 conn(0x7fa2f00fc460 msgr2=0x7fa2f00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.769+0000 7fa2f5fff700 1 -- 172.21.15.31:0/2833829396 shutdown_connections 2023-10-06T21:19:59.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.770+0000 7fa2f5fff700 1 -- 172.21.15.31:0/2833829396 wait complete. 2023-10-06T21:19:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 Processor -- start 2023-10-06T21:19:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 -- start start 2023-10-06T21:19:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 0x7fa2f01a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01a0890 con 0x7fa2f0108db0 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01a0a00 con 0x7fa2f0074ef0 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2f5fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01a0bc0 con 0x7fa2f0073910 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 0x7fa2f01a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2eeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2eeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54076/0 (socket says 172.21.15.31:54076) 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2eeffd700 1 -- 172.21.15.31:0/3218386205 learned_addr learned my addr 172.21.15.31:0/3218386205 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:19:59.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.771+0000 7fa2ef7fe700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 -- 172.21.15.31:0/3218386205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 msgr2=0x7fa2f0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 -- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 msgr2=0x7fa2f01a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 0x7fa2f01a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 -- 172.21.15.31:0/3218386205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00095f0 con 0x7fa2f0074ef0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2ef7fe700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2effff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 0x7fa2f01a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2eeffd700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e970 tx=0x7fa2e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.772+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4009790 con 0x7fa2f0074ef0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.773+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2e400c7e0 con 0x7fa2f0074ef0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.773+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f01a0ea0 con 0x7fa2f0074ef0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.773+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4003bd0 con 0x7fa2f0074ef0 2023-10-06T21:19:59.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.773+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f01a12d0 con 0x7fa2f0074ef0 2023-10-06T21:19:59.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.774+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f0107460 con 0x7fa2f0074ef0 2023-10-06T21:19:59.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.777+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa2e400c950 con 0x7fa2f0074ef0 2023-10-06T21:19:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.779+0000 7fa2ecff9700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 0x7fa2d80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.779+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa2e4014070 con 0x7fa2f0074ef0 2023-10-06T21:19:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.779+0000 7fa2ef7fe700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 0x7fa2d80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.780+0000 7fa2ef7fe700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 0x7fa2d80a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006010 tx=0x7fa2e000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:19:59.780+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa2e40964d0 con 0x7fa2f0074ef0 2023-10-06T21:20:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.004+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2f010ea30 con 0x7fa2f0074ef0 2023-10-06T21:20:00.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.005+0000 7fa2ecff9700 1 -- 172.21.15.31:0/3218386205 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2e4099af0 con 0x7fa2f0074ef0 2023-10-06T21:20:00.008 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 msgr2=0x7fa2d80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 0x7fa2d80a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006010 tx=0x7fa2e000b260 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 msgr2=0x7fa2f0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e970 tx=0x7fa2e400ec80 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 shutdown_connections 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa2d80a1590 0x7fa2d80a3a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f0108db0 0x7fa2f01a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2f0073910 0x7fa2f0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 --2- 172.21.15.31:0/3218386205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f0074ef0 0x7fa2f0199290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 >> 172.21.15.31:0/3218386205 conn(0x7fa2f00fc460 msgr2=0x7fa2f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:00.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 shutdown_connections 2023-10-06T21:20:00.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.008+0000 7fa2f5fff700 1 -- 172.21.15.31:0/3218386205 wait complete. 2023-10-06T21:20:00.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:59 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3104902427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:19:59 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/21999565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/988017928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58103110 msgr2=0x7f4c58103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/988017928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58103110 0x7f4c58103530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c480099c0 tx=0x7f4c48009cd0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/988017928 shutdown_connections 2023-10-06T21:20:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/988017928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58104ca0 0x7f4c5810b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/988017928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58103110 0x7f4c58103530 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/988017928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104300 0x7f4c58104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/988017928 >> 172.21.15.31:0/988017928 conn(0x7f4c580fe660 msgr2=0x7f4c58100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/988017928 shutdown_connections 2023-10-06T21:20:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.248+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/988017928 wait complete. 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.249+0000 7f4c5e7ff700 1 Processor -- start 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.249+0000 7f4c5e7ff700 1 -- start start 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.249+0000 7f4c5e7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58103110 0x7f4c58198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5e7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104ca0 0x7f4c581a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581a0850 con 0x7f4c58104300 2023-10-06T21:20:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5e7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581a09c0 con 0x7f4c58104ca0 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5e7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581a0b80 con 0x7f4c58103110 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c57fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58103110 0x7f4c58198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43036/0 (socket says 172.21.15.31:43036) 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 -- 172.21.15.31:0/611785570 learned_addr learned my addr 172.21.15.31:0/611785570 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c5cd9c700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104ca0 0x7f4c581a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 -- 172.21.15.31:0/611785570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58103110 msgr2=0x7f4c58198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58103110 0x7f4c58198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 -- 172.21.15.31:0/611785570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104ca0 msgr2=0x7f4c581a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104ca0 0x7f4c581a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.250+0000 7f4c577fe700 1 -- 172.21.15.31:0/611785570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480095f0 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c577fe700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e9a0 tx=0x7f4c4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c009790 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c581a0e60 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c4c010430 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00cea0 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.251+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c581a1290 con 0x7f4c58104300 2023-10-06T21:20:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.252+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5810e830 con 0x7f4c58104300 2023-10-06T21:20:00.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.255+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4c4c0105a0 con 0x7f4c58104300 2023-10-06T21:20:00.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:59 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3104902427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:00.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:19:59 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/21999565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.257+0000 7f4c557fa700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 0x7f4c400a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.258+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c4c014070 con 0x7f4c58104300 2023-10-06T21:20:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.258+0000 7f4c57fff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 0x7f4c400a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.258+0000 7f4c57fff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 0x7f4c400a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c48005ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:00.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.259+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4c4c096730 con 0x7f4c58104300 2023-10-06T21:20:00.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:59 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3104902427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:19:59 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/21999565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.483+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c5810a060 con 0x7f4c58104300 2023-10-06T21:20:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.483+0000 7f4c557fa700 1 -- 172.21.15.31:0/611785570 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c4c099d50 con 0x7f4c58104300 2023-10-06T21:20:00.486 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 msgr2=0x7f4c400a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 0x7f4c400a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c48005ab0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 msgr2=0x7f4c58199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e9a0 tx=0x7f4c4c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 shutdown_connections 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4c400a1510 0x7f4c400a39d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c58104ca0 0x7f4c581a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c58103110 0x7f4c58198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 --2- 172.21.15.31:0/611785570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c58104300 0x7f4c58199250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 >> 172.21.15.31:0/611785570 conn(0x7f4c580fe660 msgr2=0x7f4c58100930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 shutdown_connections 2023-10-06T21:20:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.486+0000 7f4c5e7ff700 1 -- 172.21.15.31:0/611785570 wait complete. 2023-10-06T21:20:00.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.719+0000 7f242adff700 1 -- 172.21.15.31:0/638408460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 msgr2=0x7f24241012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.719+0000 7f242adff700 1 --2- 172.21.15.31:0/638408460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f24241012b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2420009990 tx=0x7f2420009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 -- 172.21.15.31:0/638408460 shutdown_connections 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 --2- 172.21.15.31:0/638408460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102a30 0x7f2424109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 --2- 172.21.15.31:0/638408460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f24241012b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 --2- 172.21.15.31:0/638408460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102070 0x7f24241024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 -- 172.21.15.31:0/638408460 >> 172.21.15.31:0/638408460 conn(0x7f24240fc430 msgr2=0x7f24240fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:00.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.720+0000 7f242adff700 1 -- 172.21.15.31:0/638408460 shutdown_connections 2023-10-06T21:20:00.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.721+0000 7f242adff700 1 -- 172.21.15.31:0/638408460 wait complete. 2023-10-06T21:20:00.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 Processor -- start 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 -- start start 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f2424074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102a30 0x7f24240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419a9d0 con 0x7f2424100e90 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424073d20 con 0x7f2424102a30 2023-10-06T21:20:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.722+0000 7f242adff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424073ee0 con 0x7f2424102070 2023-10-06T21:20:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f2428b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f2424074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46378/0 (socket says 172.21.15.31:46378) 2023-10-06T21:20:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 -- 172.21.15.31:0/249142498 learned_addr learned my addr 172.21.15.31:0/249142498 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f242939c700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102a30 0x7f24240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 -- 172.21.15.31:0/249142498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102a30 msgr2=0x7f24240737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102a30 0x7f24240737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 -- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 msgr2=0x7f2424074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f2424074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 -- 172.21.15.31:0/249142498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200095f0 con 0x7f2424102070 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f2428b9b700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f2424074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.723+0000 7f241bfff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f241400e970 tx=0x7f241400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.724+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414009790 con 0x7f2424102070 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.724+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f241400c7e0 con 0x7f2424102070 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.724+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241a5380 con 0x7f2424102070 2023-10-06T21:20:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.724+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414003bd0 con 0x7f2424102070 2023-10-06T21:20:00.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.724+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24241a5800 con 0x7f2424102070 2023-10-06T21:20:00.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.725+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242410c600 con 0x7f2424102070 2023-10-06T21:20:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.728+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f241400c950 con 0x7f2424102070 2023-10-06T21:20:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.729+0000 7f2419ffb700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 0x7f240c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.730+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2414014070 con 0x7f2424102070 2023-10-06T21:20:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.730+0000 7f2428b9b700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 0x7f240c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.730+0000 7f2428b9b700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 0x7f240c0a3a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24200098e0 tx=0x7f242000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:00.731+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f24140965d0 con 0x7f2424102070 2023-10-06T21:20:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.007+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2424107eb0 con 0x7f2424102070 2023-10-06T21:20:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.008+0000 7f2419ffb700 1 -- 172.21.15.31:0/249142498 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f2414099bf0 con 0x7f2424102070 2023-10-06T21:20:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:01.011 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 msgr2=0x7f240c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 0x7f240c0a3a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24200098e0 tx=0x7f242000b260 comp rx=0 tx=0).stop 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 msgr2=0x7f24240732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f241400e970 tx=0x7f241400ec80 comp rx=0 tx=0).stop 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 shutdown_connections 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f240c0a15d0 0x7f240c0a3a90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2424102a30 0x7f24240737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2424100e90 0x7f2424074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 --2- 172.21.15.31:0/249142498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2424102070 0x7f24240732a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 >> 172.21.15.31:0/249142498 conn(0x7f24240fc430 msgr2=0x7f24240fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:01.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 shutdown_connections 2023-10-06T21:20:01.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.011+0000 7f242adff700 1 -- 172.21.15.31:0/249142498 wait complete. 2023-10-06T21:20:01.015 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:01.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:00 smithi142 ceph-mon[102951]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:00 smithi142 ceph-mon[102951]: overall HEALTH_OK 2023-10-06T21:20:01.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:00 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3218386205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:00 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/611785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.248+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1399440198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958102080 msgr2=0x7fd958102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.248+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1399440198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958102080 0x7fd958102500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009990 tx=0x7fd94c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1399440198 shutdown_connections 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1399440198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102a40 0x7fd958109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1399440198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958100ea0 0x7fd9581012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1399440198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958102080 0x7fd958102500 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1399440198 >> 172.21.15.31:0/1399440198 conn(0x7fd9580fc420 msgr2=0x7fd9580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:01.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1399440198 shutdown_connections 2023-10-06T21:20:01.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.249+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1399440198 wait complete. 2023-10-06T21:20:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 Processor -- start 2023-10-06T21:20:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 -- start start 2023-10-06T21:20:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 0x7fd958198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 0x7fd958199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 0x7fd9581a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9581a08d0 con 0x7fd958100ea0 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9581a0a10 con 0x7fd958102a40 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd95d5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9581a0ba0 con 0x7fd958102080 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd956ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 0x7fd958198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd956ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 0x7fd958198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43070/0 (socket says 172.21.15.31:43070) 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd956ffd700 1 -- 172.21.15.31:0/1458639335 learned_addr learned my addr 172.21.15.31:0/1458639335 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.250+0000 7fd9567fc700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 0x7fd958199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9577fe700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 0x7fd9581a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 -- 172.21.15.31:0/1458639335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 msgr2=0x7fd9581a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 0x7fd9581a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 -- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 msgr2=0x7fd958198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 0x7fd958198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 -- 172.21.15.31:0/1458639335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948009520 con 0x7fd958102080 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9577fe700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 0x7fd9581a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.251+0000 7fd9567fc700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 0x7fd958199290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd94c003f60 tx=0x7fd94c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.252+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c01d070 con 0x7fd958102080 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.252+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c0095f0 con 0x7fd958102080 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.252+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd94c00bdb0 con 0x7fd958102080 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.252+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c022cd0 con 0x7fd958102080 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.252+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9581a1110 con 0x7fd958102080 2023-10-06T21:20:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.253+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95810c640 con 0x7fd958102080 2023-10-06T21:20:01.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.255+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd94c022470 con 0x7fd958102080 2023-10-06T21:20:01.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.256+0000 7fd93ffff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 0x7fd9400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.256+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd94c0ce420 con 0x7fd958102080 2023-10-06T21:20:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:00 smithi031 ceph-mon[98539]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:00 smithi031 ceph-mon[98539]: overall HEALTH_OK 2023-10-06T21:20:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:00 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3218386205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:00 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/611785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.257+0000 7fd956ffd700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 0x7fd9400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.257+0000 7fd956ffd700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 0x7fd9400a3a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd948005950 tx=0x7fd948009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.259+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd94c097930 con 0x7fd958102080 2023-10-06T21:20:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:00 smithi146 ceph-mon[102809]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:00 smithi146 ceph-mon[102809]: overall HEALTH_OK 2023-10-06T21:20:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:00 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3218386205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:00 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/611785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:01.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.477+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9581a12a0 con 0x7fd958102080 2023-10-06T21:20:01.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.478+0000 7fd93ffff700 1 -- 172.21.15.31:0/1458639335 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22700 (secure 0 0 0) 0x7fd94c09af50 con 0x7fd958102080 2023-10-06T21:20:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:01.482 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:19:26.829028+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:01.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 msgr2=0x7fd9400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 0x7fd9400a3a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd948005950 tx=0x7fd948009260 comp rx=0 tx=0).stop 2023-10-06T21:20:01.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 msgr2=0x7fd958199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 0x7fd958199290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd94c003f60 tx=0x7fd94c004730 comp rx=0 tx=0).stop 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 shutdown_connections 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd9400a1550 0x7fd9400a3a10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd958102a40 0x7fd9581a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd958100ea0 0x7fd958198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 --2- 172.21.15.31:0/1458639335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd958102080 0x7fd958199290 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 >> 172.21.15.31:0/1458639335 conn(0x7fd9580fc420 msgr2=0x7fd9580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 shutdown_connections 2023-10-06T21:20:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.481+0000 7fd95d5ff700 1 -- 172.21.15.31:0/1458639335 wait complete. 2023-10-06T21:20:01.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.715+0000 7f0315fff700 1 -- 172.21.15.31:0/806020788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310108db0 msgr2=0x7f031010b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.715+0000 7f0315fff700 1 --2- 172.21.15.31:0/806020788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310108db0 0x7f031010b240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0308009f90 tx=0x7f030800b330 comp rx=0 tx=0).stop 2023-10-06T21:20:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 -- 172.21.15.31:0/806020788 shutdown_connections 2023-10-06T21:20:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 --2- 172.21.15.31:0/806020788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310108db0 0x7f031010b240 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 --2- 172.21.15.31:0/806020788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310074ef0 0x7f0310073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 --2- 172.21.15.31:0/806020788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 -- 172.21.15.31:0/806020788 >> 172.21.15.31:0/806020788 conn(0x7f03100fc420 msgr2=0x7f03100fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 -- 172.21.15.31:0/806020788 shutdown_connections 2023-10-06T21:20:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.716+0000 7f0315fff700 1 -- 172.21.15.31:0/806020788 wait complete. 2023-10-06T21:20:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.717+0000 7f0315fff700 1 Processor -- start 2023-10-06T21:20:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.717+0000 7f0315fff700 1 -- start start 2023-10-06T21:20:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 0x7f0310199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310108db0 0x7f03101a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101a0970 con 0x7f0310074ef0 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101a0ab0 con 0x7f0310073910 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f0315fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101a0c40 con 0x7f0310108db0 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f030f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f030effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 0x7f0310199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f030f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54146/0 (socket says 172.21.15.31:54146) 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f030f7fe700 1 -- 172.21.15.31:0/885164144 learned_addr learned my addr 172.21.15.31:0/885164144 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.718+0000 7f030ffff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310108db0 0x7f03101a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 -- 172.21.15.31:0/885164144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310108db0 msgr2=0x7f03101a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310108db0 0x7f03101a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 -- 172.21.15.31:0/885164144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 msgr2=0x7f0310198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 -- 172.21.15.31:0/885164144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040095f0 con 0x7f0310074ef0 2023-10-06T21:20:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030effd700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 0x7f0310199380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0304005fd0 tx=0x7f030400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030400cbd0 con 0x7f0310074ef0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f030400cd30 con 0x7f0310074ef0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304018680 con 0x7f0310074ef0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.719+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0308009c40 con 0x7f0310074ef0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.720+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03101a11f0 con 0x7f0310074ef0 2023-10-06T21:20:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.721+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0310107460 con 0x7f0310074ef0 2023-10-06T21:20:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.724+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0304010430 con 0x7f0310074ef0 2023-10-06T21:20:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.725+0000 7f030cff9700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 0x7f02f80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.725+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0304014070 con 0x7f0310074ef0 2023-10-06T21:20:01.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.726+0000 7f030f7fe700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 0x7f02f80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:01.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.726+0000 7f030f7fe700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 0x7f02f80a3a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0300009d20 tx=0x7f0300009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:01.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:01.727+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0304096bd0 con 0x7f0310074ef0 2023-10-06T21:20:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.005+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03100008d0 con 0x7f0310074ef0 2023-10-06T21:20:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.006+0000 7f030cff9700 1 -- 172.21.15.31:0/885164144 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f030409a1f0 con 0x7f0310074ef0 2023-10-06T21:20:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:02.009 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 msgr2=0x7f02f80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 0x7f02f80a3a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0300009d20 tx=0x7f0300009260 comp rx=0 tx=0).stop 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 msgr2=0x7f0310199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 0x7f0310199380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0304005fd0 tx=0x7f030400ee70 comp rx=0 tx=0).stop 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 shutdown_connections 2023-10-06T21:20:02.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f02f80a1550 0x7f02f80a3a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0310108db0 0x7f03101a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0310073910 0x7f0310198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 --2- 172.21.15.31:0/885164144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0310074ef0 0x7f0310199380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 >> 172.21.15.31:0/885164144 conn(0x7f03100fc420 msgr2=0x7f03100fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 shutdown_connections 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.009+0000 7f0315fff700 1 -- 172.21.15.31:0/885164144 wait complete. 2023-10-06T21:20:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:02.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T21:18:54.346752+0000', 'modified': '2023-10-06T21:19:01.835058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24602, 'mds_1': 24565, 'mds_2': 24526, 'mds_3': 24518, 'mds_4': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14694': {'gid': 14694, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/1272583602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 1272583602}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 1272583602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14712': {'gid': 14712, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/847340369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 847340369}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 847340369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/1755294522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 1755294522}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 1755294522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.142:6833/1989266338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 1989266338}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 1989266338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.142:6835/4077995395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 4077995395}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 4077995395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.146:6835/3172848071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 3172848071}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 3172848071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6837/824494679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 824494679}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 824494679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/1999670279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1999670279}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1999670279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.146:6839/1025732473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 1025732473}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 1025732473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2643221655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2643221655}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2643221655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T21:20:02.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-06T21:20:02.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-10-06T21:20:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/249142498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1458639335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:01 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1483023690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 msgr2=0x7fbc34073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1483023690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc34073d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1483023690 shutdown_connections 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1483023690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3410d4b0 0x7fbc3410f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1483023690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc34074ef0 0x7fbc34073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1483023690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc34073d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1483023690 >> 172.21.15.31:0/1483023690 conn(0x7fbc340fc430 msgr2=0x7fbc340fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1483023690 shutdown_connections 2023-10-06T21:20:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.236+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1483023690 wait complete. 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.237+0000 7fbc3a5ff700 1 Processor -- start 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.237+0000 7fbc3a5ff700 1 -- start start 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc34074ef0 0x7fbc3419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3410d4b0 0x7fbc341a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc341a4d10 con 0x7fbc34073910 2023-10-06T21:20:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc341a4e50 con 0x7fbc3410d4b0 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc3a5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc341a4fe0 con 0x7fbc34074ef0 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc34074ef0 0x7fbc3419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc38b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3410d4b0 0x7fbc341a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43106/0 (socket says 172.21.15.31:43106) 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 -- 172.21.15.31:0/1275833857 learned_addr learned my addr 172.21.15.31:0/1275833857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 -- 172.21.15.31:0/1275833857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc34074ef0 msgr2=0x7fbc3419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc34074ef0 0x7fbc3419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 -- 172.21.15.31:0/1275833857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3410d4b0 msgr2=0x7fbc341a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3410d4b0 0x7fbc341a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.238+0000 7fbc2bfff700 1 -- 172.21.15.31:0/1275833857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c0095f0 con 0x7fbc34073910 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9a0 tx=0x7fbc2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc31ffb700 1 -- 172.21.15.31:0/1275833857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24009790 con 0x7fbc34073910 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc31ffb700 1 -- 172.21.15.31:0/1275833857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc24010430 con 0x7fbc34073910 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc341a52c0 con 0x7fbc34073910 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc31ffb700 1 -- 172.21.15.31:0/1275833857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400cea0 con 0x7fbc34073910 2023-10-06T21:20:02.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.239+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc341a5820 con 0x7fbc34073910 2023-10-06T21:20:02.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.242+0000 7fbc31ffb700 1 -- 172.21.15.31:0/1275833857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbc240105a0 con 0x7fbc34073910 2023-10-06T21:20:02.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.243+0000 7fbc4de90b80 1 Processor -- start 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 -- start start 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc31ffb700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 0x7fbc200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc31ffb700 1 -- 172.21.15.31:0/1275833857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbc24014070 con 0x7fbc34073910 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 0x7fbc200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6da10 0x5628b2f6de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2ed4410 con 0x5628b2f6c210 2023-10-06T21:20:02.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2ec5bd0 con 0x5628b2f6da10 2023-10-06T21:20:02.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2ec34f0 con 0x5628b2f6cb70 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46444/0 (socket says 172.21.15.31:46444) 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.244+0000 7fbc33fff700 1 -- 172.21.15.31:0/2865913746 learned_addr learned my addr 172.21.15.31:0/2865913746 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6da10 0x5628b2f6de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 -- 172.21.15.31:0/2865913746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6da10 msgr2=0x5628b2f6de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6da10 0x5628b2f6de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 -- 172.21.15.31:0/2865913746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 msgr2=0x5628b2f6c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f6c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 -- 172.21.15.31:0/2865913746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b2ce9d90 con 0x5628b2f6cb70 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 0x7fbc200a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c003f20 tx=0x7fbc2c009ba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a55767338536fae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc2a7fc700 1 -- 172.21.15.31:0/2865913746 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c01d070 con 0x5628b2f6cb70 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc2a7fc700 1 -- 172.21.15.31:0/2865913746 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1c01e040 con 0x5628b2f6cb70 2023-10-06T21:20:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.245+0000 7fbc2a7fc700 1 -- 172.21.15.31:0/2865913746 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00b970 con 0x5628b2f6cb70 2023-10-06T21:20:02.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.246+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2865913746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 msgr2=0x5628b2f6d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.246+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c005f00 comp rx=0 tx=0).stop 2023-10-06T21:20:02.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2865913746 shutdown_connections 2023-10-06T21:20:02.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6da10 0x5628b2f6de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2865913746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f6d430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2865913746 >> 172.21.15.31:0/2865913746 conn(0x5628b2ee0210 msgr2=0x5628b2f670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2865913746 shutdown_connections 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2865913746 wait complete. 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.247+0000 7fbc4de90b80 1 -- start start 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.248+0000 7fbc4de90b80 1 Processor -- start 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.248+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6cb70 0x5628b2f764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2ebede0 con 0x5628b2f6cb70 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2e5b6c0 con 0x5628b2f6c210 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2e5b360 con 0x5628b2f6da10 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc4de90b80 10 client.0 _ll_register_callbacks cb 0x5628b2cb87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6cb70 0x5628b2f764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54184/0 (socket says 172.21.15.31:54184) 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 -- 172.21.15.31:0/874204795 learned_addr learned my addr 172.21.15.31:0/874204795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 -- 172.21.15.31:0/874204795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 msgr2=0x5628b2f7d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 -- 172.21.15.31:0/874204795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6cb70 msgr2=0x5628b2f764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6cb70 0x5628b2f764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 -- 172.21.15.31:0/874204795 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc34110250 con 0x5628b2f6c210 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.249+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc2409a100 tx=0x7fbc2409a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee39e95c7f5fe11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a2270 con 0x5628b2f6c210 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628b2f5eec0 con 0x5628b2f6c210 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbc2409a170 con 0x5628b2f6c210 2023-10-06T21:20:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc2409e2c0 con 0x5628b2f6c210 2023-10-06T21:20:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc4de90b80 10 client.24959 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628b2cd2c10 con 0x5628b2f6c210 2023-10-06T21:20:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5628b2f5eec0 con 0x5628b2f6c210 2023-10-06T21:20:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.250+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbc24095f60 con 0x5628b2f6c210 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 10 client.24959 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 10 client.24959 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 20 client.24959 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2f81b70 0x5628b2f80690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 4 client.24959 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628b2eb7dc0 con 0x5628b2f81b70 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.251+0000 7fbc33fff700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2f81b70 0x5628b2f80690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.252+0000 7fbc33fff700 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2f81b70 0x5628b2f80690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34b68ff0286e24be server_cookie=98710dd5eac83f4 in_seq=0 out_seq=0 2023-10-06T21:20:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.252+0000 7fbc137fe700 10 client.24959 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.252+0000 7fbc137fe700 1 -- 172.21.15.31:0/874204795 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5628b2eb7dc0 con 0x5628b2f81b70 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.252+0000 7fbc137fe700 10 client.24959 handle_command_reply: tid=0 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 client.24959 shutdown 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 msgr2=0x5628b2f75fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc2409a100 tx=0x7fbc2409a6b0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 shutdown_connections 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2f81b70 0x5628b2f80690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f75fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc137fe700 0 client.24959 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/874204795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6cb70 0x5628b2f764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 >> 172.21.15.31:0/874204795 conn(0x5628b2ee0210 msgr2=0x5628b2f66f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 shutdown_connections 2023-10-06T21:20:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 1 -- 172.21.15.31:0/874204795 wait complete. 2023-10-06T21:20:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.253+0000 7fbc4de90b80 20 client.24959 trim_cache size 0 max 16384 2023-10-06T21:20:02.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/249142498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1458639335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:01 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 Processor -- start 2023-10-06T21:20:02.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 -- start start 2023-10-06T21:20:02.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2d3f8d0 con 0x5628b2f6c210 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2eb7dc0 con 0x5628b2f6cb70 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2f82900 con 0x5628b2f6da10 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54188/0 (socket says 172.21.15.31:54188) 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43128/0 (socket says 172.21.15.31:43128) 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc33fff700 1 -- 172.21.15.31:0/2222243258 learned_addr learned my addr 172.21.15.31:0/2222243258 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.273+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 -- 172.21.15.31:0/2222243258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 msgr2=0x5628b2f7d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 -- 172.21.15.31:0/2222243258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 msgr2=0x5628b2f709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 -- 172.21.15.31:0/2222243258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b2d10200 con 0x5628b2f6cb70 2023-10-06T21:20:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc33fff700 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec647065ba903274 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc2affd700 1 -- 172.21.15.31:0/2222243258 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c01d040 con 0x5628b2f6cb70 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc2affd700 1 -- 172.21.15.31:0/2222243258 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1c003d30 con 0x5628b2f6cb70 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.274+0000 7fbc2affd700 1 -- 172.21.15.31:0/2222243258 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c035880 con 0x5628b2f6cb70 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2222243258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 msgr2=0x5628b2f70ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c01a040 comp rx=0 tx=0).stop 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2222243258 shutdown_connections 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6da10 0x5628b2f7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6c210 0x5628b2f709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/2222243258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6cb70 0x5628b2f70ef0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2222243258 >> 172.21.15.31:0/2222243258 conn(0x5628b2ee0210 msgr2=0x5628b2f74c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2222243258 shutdown_connections 2023-10-06T21:20:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.275+0000 7fbc4de90b80 1 -- 172.21.15.31:0/2222243258 wait complete. 2023-10-06T21:20:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.276+0000 7fbc4de90b80 1 -- start start 2023-10-06T21:20:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.276+0000 7fbc4de90b80 1 Processor -- start 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6da10 0x5628b2f7ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2f7d230 con 0x5628b2f6da10 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2f7d3a0 con 0x5628b2f6c210 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b2f7d560 con 0x5628b2f6cb70 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc4de90b80 10 client.0 _ll_register_callbacks cb 0x7fbc24099ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54194/0 (socket says 172.21.15.31:54194) 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 -- 172.21.15.31:0/812512979 learned_addr learned my addr 172.21.15.31:0/812512979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc38b9c700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6da10 0x5628b2f7ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 -- 172.21.15.31:0/812512979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 msgr2=0x5628b2f79cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f79cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.277+0000 7fbc2bfff700 1 -- 172.21.15.31:0/812512979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6da10 msgr2=0x5628b2f7ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6da10 0x5628b2f7ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc2bfff700 1 -- 172.21.15.31:0/812512979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b2ce9d90 con 0x5628b2f6c210 2023-10-06T21:20:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc33fff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc2bfff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c2d0 tx=0x7fbc240ca8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3164316c8c42cd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc137fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2409a170 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc1c01e040 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbc2409e2c0 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc240a2500 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc4de90b80 10 client.24965 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.278+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628b2ce8260 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1c01e040 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbc240b8200 con 0x5628b2f6c210 2023-10-06T21:20:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 10 client.24965 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 10 client.24965 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 20 client.24965 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2fd9d80 0x5628b2fd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 4 client.24965 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628b2eb7ae0 con 0x5628b2fd9d80 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.279+0000 7fbc33fff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2fd9d80 0x5628b2fd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc33fff700 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2fd9d80 0x5628b2fd6210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7792f152c20ecb server_cookie=68b76cc921b200c2 in_seq=0 out_seq=0 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc137fe700 10 client.24965 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc137fe700 1 -- 172.21.15.31:0/812512979 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5628b2eb7ae0 con 0x5628b2fd9d80 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc137fe700 10 client.24965 handle_command_reply: tid=0 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 client.24965 shutdown 2023-10-06T21:20:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 msgr2=0x5628b2f79790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c2d0 tx=0x7fbc240ca8a0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 shutdown_connections 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x5628b2fd9d80 0x5628b2fd6210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5628b2f6da10 0x5628b2f7ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628b2f6c210 0x5628b2f79790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 --2- 172.21.15.31:0/812512979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628b2f6cb70 0x5628b2f79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc137fe700 0 client.24965 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 >> 172.21.15.31:0/812512979 conn(0x5628b2ee0210 msgr2=0x5628b2f709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 shutdown_connections 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.280+0000 7fbc4de90b80 1 -- 172.21.15.31:0/812512979 wait complete. 2023-10-06T21:20:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc4de90b80 20 client.24965 trim_cache size 0 max 16384 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24476, 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 61.449752425, 2023-10-06T21:20:02.289 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 88, 2023-10-06T21:20:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 105.49343218200001 2023-10-06T21:20:02.290 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 msgr2=0x7fbc200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 0x7fbc200a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c003f20 tx=0x7fbc2c009ba0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 msgr2=0x7fbc3419d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9a0 tx=0x7fbc2400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 shutdown_connections 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc200a1590 0x7fbc200a3a50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc3410d4b0 0x7fbc341a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc34073910 0x7fbc3419d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 --2- 172.21.15.31:0/1275833857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc34074ef0 0x7fbc3419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.281+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 >> 172.21.15.31:0/1275833857 conn(0x7fbc340fc430 msgr2=0x7fbc340fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.282+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 shutdown_connections 2023-10-06T21:20:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.282+0000 7fbc3a5ff700 1 -- 172.21.15.31:0/1275833857 wait complete. 2023-10-06T21:20:02.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/249142498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1458639335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:01 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:02.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-06T21:20:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 -- 172.21.15.31:0/3652832385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 msgr2=0x7fbfc8073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 --2- 172.21.15.31:0/3652832385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8073d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009990 tx=0x7fbfb4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 -- 172.21.15.31:0/3652832385 shutdown_connections 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 --2- 172.21.15.31:0/3652832385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8108da0 0x7fbfc810b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 --2- 172.21.15.31:0/3652832385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8074ef0 0x7fbfc8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 --2- 172.21.15.31:0/3652832385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8073d90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 -- 172.21.15.31:0/3652832385 >> 172.21.15.31:0/3652832385 conn(0x7fbfc80fc430 msgr2=0x7fbfc80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 -- 172.21.15.31:0/3652832385 shutdown_connections 2023-10-06T21:20:02.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.511+0000 7fbfcedff700 1 -- 172.21.15.31:0/3652832385 wait complete. 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.512+0000 7fbfcedff700 1 Processor -- start 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.512+0000 7fbfcedff700 1 -- start start 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8074ef0 0x7fbfc81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8108da0 0x7fbfc81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81a08f0 con 0x7fbfc8073910 2023-10-06T21:20:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81a0a30 con 0x7fbfc8108da0 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcedff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81a0bc0 con 0x7fbfc8074ef0 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43154/0 (socket says 172.21.15.31:43154) 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 -- 172.21.15.31:0/220528338 learned_addr learned my addr 172.21.15.31:0/220528338 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfcd39c700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8108da0 0x7fbfc81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8074ef0 0x7fbfc81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 -- 172.21.15.31:0/220528338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8074ef0 msgr2=0x7fbfc81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8074ef0 0x7fbfc81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 -- 172.21.15.31:0/220528338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8108da0 msgr2=0x7fbfc81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8108da0 0x7fbfc81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.513+0000 7fbfccb9b700 1 -- 172.21.15.31:0/220528338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40095f0 con 0x7fbfc8073910 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d950 tx=0x7fbfbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfc5ffb700 1 -- 172.21.15.31:0/220528338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00f890 con 0x7fbfc8073910 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfc5ffb700 1 -- 172.21.15.31:0/220528338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc00fe80 con 0x7fbfc8073910 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfc5ffb700 1 -- 172.21.15.31:0/220528338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00e3b0 con 0x7fbfc8073910 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc81a0ea0 con 0x7fbfc8073910 2023-10-06T21:20:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.514+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc81a1270 con 0x7fbfc8073910 2023-10-06T21:20:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.518+0000 7fbfc5ffb700 1 -- 172.21.15.31:0/220528338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbfbc010460 con 0x7fbfc8073910 2023-10-06T21:20:02.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.519+0000 7fbfc5ffb700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 0x7fbfb80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.519+0000 7fbfe267cb80 1 Processor -- start 2023-10-06T21:20:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.519+0000 7fbfc5ffb700 1 -- 172.21.15.31:0/220528338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfbc0ccc60 con 0x7fbfc8073910 2023-10-06T21:20:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.519+0000 7fbfe267cb80 1 -- start start 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.519+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 0x7fbfb80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35b70 0x55fa62a36430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a36e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6299d410 con 0x55fa62a35b70 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6298ebd0 con 0x55fa62a35210 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6298c4f0 con 0x55fa62a36a10 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 0x7fbfb80a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400b5c0 tx=0x7fbfb4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35b70 0x55fa62a36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54224/0 (socket says 172.21.15.31:54224) 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1155641006 learned_addr learned my addr 172.21.15.31:0/1155641006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.520+0000 7fbfcd39c700 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a36e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1155641006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 msgr2=0x55fa62a36e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a36e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1155641006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35b70 msgr2=0x55fa62a36430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35b70 0x55fa62a36430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1155641006 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa627b2d90 con 0x55fa62a35210 2023-10-06T21:20:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0a1c40 tx=0x7fbfbc010760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dbf030a57c7ade3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1155641006 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc099de0 con 0x55fa62a35210 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1155641006 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc09df30 con 0x55fa62a35210 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.521+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1155641006 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0992d0 con 0x55fa62a35210 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1155641006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 msgr2=0x55fa62a35630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0a1c40 tx=0x7fbfbc010760 comp rx=0 tx=0).stop 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1155641006 shutdown_connections 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a36e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35210 0x55fa62a35630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1155641006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35b70 0x55fa62a36430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1155641006 >> 172.21.15.31:0/1155641006 conn(0x55fa629a9210 msgr2=0x55fa62a300f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1155641006 shutdown_connections 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.522+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1155641006 wait complete. 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.523+0000 7fbfe267cb80 1 -- start start 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.523+0000 7fbfe267cb80 1 Processor -- start 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35b70 0x55fa62a3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a36a10 0x55fa62a46450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62987de0 con 0x55fa62a35210 2023-10-06T21:20:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa629246c0 con 0x55fa62a36a10 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62924360 con 0x55fa62a35b70 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfe267cb80 10 client.0 _ll_register_callbacks cb 0x55fa627817b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.524+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43182/0 (socket says 172.21.15.31:43182) 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 -- 172.21.15.31:0/2987547203 learned_addr learned my addr 172.21.15.31:0/2987547203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfcd39c700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a36a10 0x55fa62a46450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35b70 0x55fa62a3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 -- 172.21.15.31:0/2987547203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35b70 msgr2=0x55fa62a3f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35b70 0x55fa62a3f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 -- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a36a10 msgr2=0x55fa62a46450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a36a10 0x55fa62a46450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 -- 172.21.15.31:0/2987547203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc81a1270 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc098130 tx=0x7fbfbc099500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b28a2e6452222f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0a2170 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfbc0ca820 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.525+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa62a27ec0 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfbc099de0 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 10 client.15138 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa6279bc10 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fa62a27ec0 con 0x55fa62a35210 2023-10-06T21:20:02.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfbc05fbb0 con 0x55fa62a35210 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 10 client.15138 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 20 client.15138 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62a4a610 0x55fa62a49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 4 client.15138 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.526+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa62980dc0 con 0x55fa62a4a610 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.527+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62a4a610 0x55fa62a49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.527+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62a4a610 0x55fa62a49380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cc569afd62702a2 server_cookie=69ab5cbbbbfe7555 in_seq=0 out_seq=0 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.527+0000 7fbfa7fff700 10 client.15138 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.528+0000 7fbfa7fff700 1 -- 172.21.15.31:0/2987547203 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fa62980dc0 con 0x55fa62a4a610 2023-10-06T21:20:02.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.528+0000 7fbfa7fff700 10 client.15138 handle_command_reply: tid=0 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.528+0000 7fbfe267cb80 1 client.15138 shutdown 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 msgr2=0x55fa62a3eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc098130 tx=0x7fbfbc099500 comp rx=0 tx=0).stop 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 shutdown_connections 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62a4a610 0x55fa62a49380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a36a10 0x55fa62a46450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3eed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/2987547203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35b70 0x55fa62a3f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfa7fff700 0 client.15138 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 >> 172.21.15.31:0/2987547203 conn(0x55fa629a9210 msgr2=0x55fa62a2ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 shutdown_connections 2023-10-06T21:20:02.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 1 -- 172.21.15.31:0/2987547203 wait complete. 2023-10-06T21:20:02.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.529+0000 7fbfe267cb80 20 client.15138 trim_cache size 0 max 16384 2023-10-06T21:20:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 Processor -- start 2023-10-06T21:20:02.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 -- start start 2023-10-06T21:20:02.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a46c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa628088d0 con 0x55fa62a35210 2023-10-06T21:20:02.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62a2a2b0 con 0x55fa62a35b70 2023-10-06T21:20:02.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.548+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62980dc0 con 0x55fa62a36a10 2023-10-06T21:20:02.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43188/0 (socket says 172.21.15.31:43188) 2023-10-06T21:20:02.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1779021566 learned_addr learned my addr 172.21.15.31:0/1779021566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfcd39c700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a46c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1779021566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 msgr2=0x55fa62a46c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a46c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1779021566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 msgr2=0x55fa62a46410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 -- 172.21.15.31:0/1779021566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa627d9200 con 0x55fa62a35210 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfcd39c700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a46c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.549+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc096850 tx=0x7fbfbc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddae49e8443812e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfb37fe700 1 -- 172.21.15.31:0/1779021566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc099570 con 0x55fa62a35210 2023-10-06T21:20:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfb37fe700 1 -- 172.21.15.31:0/1779021566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc095f50 con 0x55fa62a35210 2023-10-06T21:20:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfb37fe700 1 -- 172.21.15.31:0/1779021566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc099de0 con 0x55fa62a35210 2023-10-06T21:20:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1779021566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 msgr2=0x55fa62a3cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc096850 tx=0x7fbfbc000bc0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1779021566 shutdown_connections 2023-10-06T21:20:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a36a10 0x55fa62a46c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a35210 0x55fa62a3cec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/1779021566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.550+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1779021566 >> 172.21.15.31:0/1779021566 conn(0x55fa629a9210 msgr2=0x55fa62a3db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.551+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1779021566 shutdown_connections 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.551+0000 7fbfe267cb80 1 -- 172.21.15.31:0/1779021566 wait complete. 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.551+0000 7fbfe267cb80 1 -- start start 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.551+0000 7fbfe267cb80 1 Processor -- start 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35210 0x55fa62a45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 0x55fa62a42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62a43080 con 0x55fa62a36a10 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62a431f0 con 0x55fa62a35b70 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa62a433b0 con 0x55fa62a35210 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfe267cb80 10 client.0 _ll_register_callbacks cb 0x7fbfb4005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfcd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 0x55fa62a42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfcd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 0x55fa62a42970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43202/0 (socket says 172.21.15.31:43202) 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfcd39c700 1 -- 172.21.15.31:0/4185352429 learned_addr learned my addr 172.21.15.31:0/4185352429 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35210 0x55fa62a45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 -- 172.21.15.31:0/4185352429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35210 msgr2=0x55fa62a45dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35210 0x55fa62a45dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 -- 172.21.15.31:0/4185352429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 msgr2=0x55fa62a42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 0x55fa62a42970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.552+0000 7fbfc7fff700 1 -- 172.21.15.31:0/4185352429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa6279bc10 con 0x55fa62a35b70 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfc7fff700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c010 tx=0x7fbfb4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650d3b9e7b570737 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40343a0 con 0x55fa62a35b70 2023-10-06T21:20:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfbc09df30 con 0x55fa62a35b70 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfb4034500 con 0x55fa62a35b70 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb4034a80 con 0x55fa62a35b70 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfe267cb80 10 client.24983 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.553+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa627b1260 con 0x55fa62a35b70 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc09df30 con 0x55fa62a35b70 2023-10-06T21:20:02.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfb4045020 con 0x55fa62a35b70 2023-10-06T21:20:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 10 client.24983 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 10 client.24983 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T21:20:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 20 client.24983 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62aa2f70 0x55fa62a9f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 4 client.24983 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa62980ae0 con 0x55fa62aa2f70 2023-10-06T21:20:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.554+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62aa2f70 0x55fa62a9f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfccb9b700 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62aa2f70 0x55fa62a9f3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c74a22a28b36119 server_cookie=103b6c4c9998c5ea in_seq=0 out_seq=0 2023-10-06T21:20:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfa7fff700 10 client.24983 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfa7fff700 1 -- 172.21.15.31:0/4185352429 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fa62980ae0 con 0x55fa62aa2f70 2023-10-06T21:20:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfa7fff700 10 client.24983 handle_command_reply: tid=0 2023-10-06T21:20:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 client.24983 shutdown 2023-10-06T21:20:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 msgr2=0x55fa62a46300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c010 tx=0x7fbfb4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 shutdown_connections 2023-10-06T21:20:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa62a36a10 0x55fa62a42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fa62a35210 0x55fa62a45dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55fa62aa2f70 0x55fa62a9f3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 --2- 172.21.15.31:0/4185352429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa62a35b70 0x55fa62a46300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfa7fff700 0 client.24983 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 >> 172.21.15.31:0/4185352429 conn(0x55fa629a9210 msgr2=0x55fa62a46c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 shutdown_connections 2023-10-06T21:20:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.555+0000 7fbfe267cb80 1 -- 172.21.15.31:0/4185352429 wait complete. 2023-10-06T21:20:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfe267cb80 20 client.24983 trim_cache size 0 max 16384 2023-10-06T21:20:02.572 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24518, 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 62.732428912000003, 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 88, 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 96.220447399999998 2023-10-06T21:20:02.574 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:02.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 msgr2=0x7fbfb80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 0x7fbfb80a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400b5c0 tx=0x7fbfb4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 msgr2=0x7fbfc8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d950 tx=0x7fbfbc00dc60 comp rx=0 tx=0).stop 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 shutdown_connections 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbfb80a1510 0x7fbfb80a39d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc8108da0 0x7fbfc81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc8073910 0x7fbfc8198d70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 --2- 172.21.15.31:0/220528338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfc8074ef0 0x7fbfc81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.556+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 >> 172.21.15.31:0/220528338 conn(0x7fbfc80fc430 msgr2=0x7fbfc80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.557+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 shutdown_connections 2023-10-06T21:20:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.557+0000 7fbfcedff700 1 -- 172.21.15.31:0/220528338 wait complete. 2023-10-06T21:20:02.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-06T21:20:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 -- 172.21.15.31:0/3933035354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 msgr2=0x7f54f01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 --2- 172.21.15.31:0/3933035354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f01024f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009990 tx=0x7f54e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 -- 172.21.15.31:0/3933035354 shutdown_connections 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 --2- 172.21.15.31:0/3933035354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0102a30 0x7f54f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 --2- 172.21.15.31:0/3933035354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0100e90 0x7f54f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 --2- 172.21.15.31:0/3933035354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f01024f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.788+0000 7f54f87ff700 1 -- 172.21.15.31:0/3933035354 >> 172.21.15.31:0/3933035354 conn(0x7f54f00fc430 msgr2=0x7f54f00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.789+0000 7f54f87ff700 1 -- 172.21.15.31:0/3933035354 shutdown_connections 2023-10-06T21:20:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.789+0000 7f54f87ff700 1 -- 172.21.15.31:0/3933035354 wait complete. 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.790+0000 7f54f87ff700 1 Processor -- start 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.790+0000 7f54f87ff700 1 -- start start 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 0x7f54f0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f0197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 0x7f54f019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f019e6d0 con 0x7f54f0100e90 2023-10-06T21:20:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f019e810 con 0x7f54f0102a30 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f87ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f019e9a0 con 0x7f54f0102070 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 0x7f54f0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 0x7f54f0196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43224/0 (socket says 172.21.15.31:43224) 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f5d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f0197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f659b700 1 -- 172.21.15.31:0/2807992375 learned_addr learned my addr 172.21.15.31:0/2807992375 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.791+0000 7f54f6d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 0x7f54f019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 -- 172.21.15.31:0/2807992375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 msgr2=0x7f54f019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 0x7f54f019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 -- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 msgr2=0x7f54f0196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 0x7f54f0196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 -- 172.21.15.31:0/2807992375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e00095f0 con 0x7f54f0102070 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 0x7f54f019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f0197090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54e0003f60 tx=0x7f54e0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54e77fe700 1 -- 172.21.15.31:0/2807992375 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e001d070 con 0x7f54f0102070 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54e77fe700 1 -- 172.21.15.31:0/2807992375 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54e000bdb0 con 0x7f54f0102070 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54e77fe700 1 -- 172.21.15.31:0/2807992375 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0022cd0 con 0x7f54f0102070 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f019ec20 con 0x7f54f0102070 2023-10-06T21:20:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.792+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f019eff0 con 0x7f54f0102070 2023-10-06T21:20:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.796+0000 7f54e77fe700 1 -- 172.21.15.31:0/2807992375 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f54e0022470 con 0x7f54f0102070 2023-10-06T21:20:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.797+0000 7f550c14db80 1 Processor -- start 2023-10-06T21:20:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.797+0000 7f550c14db80 1 -- start start 2023-10-06T21:20:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.797+0000 7f550c14db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572944a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f550c14db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f550c14db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f550c14db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595728a8410 con 0x559572944f80 2023-10-06T21:20:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f550c14db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559572899bd0 con 0x559572944620 2023-10-06T21:20:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f550c14db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595728974f0 con 0x559572945e20 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f6d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572944a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f6d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572944a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54284/0 (socket says 172.21.15.31:54284) 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f6d9c700 1 -- 172.21.15.31:0/1550923382 learned_addr learned my addr 172.21.15.31:0/1550923382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43240/0 (socket says 172.21.15.31:43240) 2023-10-06T21:20:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54e77fe700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 0x7f54dc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1550923382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 msgr2=0x559572944a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572944a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1550923382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 msgr2=0x559572945840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.798+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1550923382 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595726bdd90 con 0x559572945e20 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54f659b700 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 0x7f54dc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54e77fe700 1 -- 172.21.15.31:0/2807992375 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f54e00ce350 con 0x7f54f0102070 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729462a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54e00cbc50 tx=0x7f54e002d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39890701c0355b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54e4ff9700 1 -- 172.21.15.31:0/1550923382 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0097070 con 0x559572945e20 2023-10-06T21:20:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54e4ff9700 1 -- 172.21.15.31:0/1550923382 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54e009b2f0 con 0x559572945e20 2023-10-06T21:20:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.799+0000 7f54e4ff9700 1 -- 172.21.15.31:0/1550923382 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e009bc20 con 0x559572945e20 2023-10-06T21:20:02.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 -- 172.21.15.31:0/1550923382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 msgr2=0x5595729462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729462a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54e00cbc50 tx=0x7f54e002d000 comp rx=0 tx=0).stop 2023-10-06T21:20:02.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 0x7f54dc0a3a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009fd0 tx=0x7f54e8004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 -- 172.21.15.31:0/1550923382 shutdown_connections 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729462a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572944a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 --2- 172.21.15.31:0/1550923382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x559572945840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 -- 172.21.15.31:0/1550923382 >> 172.21.15.31:0/1550923382 conn(0x5595728b4210 msgr2=0x55957293b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 -- 172.21.15.31:0/1550923382 shutdown_connections 2023-10-06T21:20:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.800+0000 7f550c14db80 1 -- 172.21.15.31:0/1550923382 wait complete. 2023-10-06T21:20:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.801+0000 7f550c14db80 1 -- start start 2023-10-06T21:20:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.801+0000 7f550c14db80 1 Processor -- start 2023-10-06T21:20:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x55957294e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559572892de0 con 0x559572944f80 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957282f6c0 con 0x559572944620 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957282f360 con 0x559572945e20 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f550c14db80 10 client.0 _ll_register_callbacks cb 0x55957268c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f54f5d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f54f5d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43254/0 (socket says 172.21.15.31:43254) 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f54f5d9a700 1 -- 172.21.15.31:0/3053379803 learned_addr learned my addr 172.21.15.31:0/3053379803 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.802+0000 7f54f659b700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x55957294e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 -- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 msgr2=0x5595729555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 -- 172.21.15.31:0/3053379803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 msgr2=0x55957294e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x55957294e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 -- 172.21.15.31:0/3053379803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f019eff0 con 0x559572944f80 2023-10-06T21:20:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005910 tx=0x7f54e0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b086b45f753fc45e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0097040 con 0x559572944f80 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f54e009b520 con 0x559572944f80 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559572932ec0 con 0x559572944f80 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54e009f670 con 0x559572944f80 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.803+0000 7f550c14db80 10 client.15156 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595726a6c10 con 0x559572944f80 2023-10-06T21:20:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54e00b7080 con 0x559572944f80 2023-10-06T21:20:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f54e00054a0 con 0x559572944f80 2023-10-06T21:20:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 10 client.15156 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T21:20:02.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 20 client.15156 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x559572959750 0x5595729584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 4 client.15156 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.804+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55957288bdc0 con 0x559572959750 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.805+0000 7f54f659b700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x559572959750 0x5595729584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.805+0000 7f54f659b700 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x559572959750 0x5595729584d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad7f085127fc9991 server_cookie=6d2b75364b8c7d7b in_seq=0 out_seq=0 2023-10-06T21:20:02.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.805+0000 7f54d9ffb700 10 client.15156 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f54d9ffb700 1 -- 172.21.15.31:0/3053379803 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55957288bdc0 con 0x559572959750 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f54d9ffb700 10 client.15156 handle_command_reply: tid=0 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 client.15156 shutdown 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 msgr2=0x55957294e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005910 tx=0x7f54e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 shutdown_connections 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x559572959750 0x5595729584d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572945e20 0x5595729555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x55957294e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 --2- 172.21.15.31:0/3053379803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944f80 0x55957294e560 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f54d9ffb700 0 client.15156 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 >> 172.21.15.31:0/3053379803 conn(0x5595728b4210 msgr2=0x55957293af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.806+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 shutdown_connections 2023-10-06T21:20:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.807+0000 7f550c14db80 1 -- 172.21.15.31:0/3053379803 wait complete. 2023-10-06T21:20:02.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.807+0000 7f550c14db80 20 client.15156 trim_cache size 0 max 16384 2023-10-06T21:20:02.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.825+0000 7f550c14db80 1 Processor -- start 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.825+0000 7f550c14db80 1 -- start start 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572955dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595727138d0 con 0x559572945e20 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957288bdc0 con 0x559572944620 2023-10-06T21:20:02.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f550c14db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957294f210 con 0x559572944f80 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f54f6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f54f6d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43266/0 (socket says 172.21.15.31:43266) 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f54f6d9c700 1 -- 172.21.15.31:0/1064168891 learned_addr learned my addr 172.21.15.31:0/1064168891 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.826+0000 7f54f659b700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572955dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1064168891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 msgr2=0x559572955dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572955dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1064168891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 msgr2=0x5595729584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 -- 172.21.15.31:0/1064168891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595726e4200 con 0x559572944f80 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009520 tx=0x7f54e0005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21d5943a92976e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54e57fa700 1 -- 172.21.15.31:0/1064168891 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0097040 con 0x559572944f80 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54e57fa700 1 -- 172.21.15.31:0/1064168891 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559572932ec0 con 0x559572944f80 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.827+0000 7f54e57fa700 1 -- 172.21.15.31:0/1064168891 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e009b520 con 0x559572944f80 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- 172.21.15.31:0/1064168891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 msgr2=0x559572955560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009520 tx=0x7f54e0005ac0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- 172.21.15.31:0/1064168891 shutdown_connections 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572945e20 0x5595729584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572944620 0x559572955dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 --2- 172.21.15.31:0/1064168891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- 172.21.15.31:0/1064168891 >> 172.21.15.31:0/1064168891 conn(0x5595728b4210 msgr2=0x55957293af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- 172.21.15.31:0/1064168891 shutdown_connections 2023-10-06T21:20:02.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- 172.21.15.31:0/1064168891 wait complete. 2023-10-06T21:20:02.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.828+0000 7f550c14db80 1 -- start start 2023-10-06T21:20:02.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 Processor -- start 2023-10-06T21:20:02.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572945e20 0x55957294a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957294b040 con 0x559572944620 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595729ad140 con 0x559572945e20 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595729ad280 con 0x559572944f80 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f550c14db80 10 client.0 _ll_register_callbacks cb 0x7f54e8003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.829+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43276/0 (socket says 172.21.15.31:43276) 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 -- 172.21.15.31:0/4147042203 learned_addr learned my addr 172.21.15.31:0/4147042203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f6d9c700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572945e20 0x55957294a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 -- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 msgr2=0x559572955560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 -- 172.21.15.31:0/4147042203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572945e20 msgr2=0x55957294a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572945e20 0x55957294a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 -- 172.21.15.31:0/4147042203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e80047a0 con 0x559572944620 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54f659b700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55957295a480 tx=0x7f54ec01c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef36ac4c1ffc797a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.830+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec015040 con 0x559572944620 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54e009b2f0 con 0x559572944620 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f54ec01cd20 con 0x559572944620 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ec0045a0 con 0x559572944620 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 10 client.15162 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:02.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595726a6c10 con 0x559572944620 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54e009b2f0 con 0x559572944620 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f54ec022070 con 0x559572944620 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 10 client.15162 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 20 client.15162 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x5595729b24b0 0x5595729ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 4 client.15162 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.831+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55957288bae0 con 0x5595729b24b0 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.832+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x5595729b24b0 0x5595729ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:02.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.832+0000 7f54f5d9a700 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x5595729b24b0 0x5595729ae910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91e401bbd8ce1589 server_cookie=ceefc3580247e431 in_seq=0 out_seq=0 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.832+0000 7f54d9ffb700 10 client.15162 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f54d9ffb700 1 -- 172.21.15.31:0/4147042203 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55957288bae0 con 0x5595729b24b0 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f54d9ffb700 10 client.15162 handle_command_reply: tid=0 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 client.15162 shutdown 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 msgr2=0x55957294a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55957295a480 tx=0x7f54ec01c5a0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 shutdown_connections 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x5595729b24b0 0x5595729ae910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559572945e20 0x55957294a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559572944620 0x55957294a2a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 --2- 172.21.15.31:0/4147042203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559572944f80 0x559572955560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 >> 172.21.15.31:0/4147042203 conn(0x5595728b4210 msgr2=0x55957294c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f54d9ffb700 0 client.15162 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 shutdown_connections 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 1 -- 172.21.15.31:0/4147042203 wait complete. 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.833+0000 7f550c14db80 20 client.15162 trim_cache size 0 max 16384 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:02.841 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24526, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 64.016442694000006, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 88, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 93.411653724999994 2023-10-06T21:20:02.842 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:02.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 msgr2=0x7f54dc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 0x7f54dc0a3a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009fd0 tx=0x7f54e8004ad0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 msgr2=0x7f54f0197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f0197090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54e0003f60 tx=0x7f54e0004730 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 shutdown_connections 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f54dc0a1590 0x7f54dc0a3a50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f0102a30 0x7f54f019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54f0100e90 0x7f54f0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 --2- 172.21.15.31:0/2807992375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54f0102070 0x7f54f0197090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 >> 172.21.15.31:0/2807992375 conn(0x7f54f00fc430 msgr2=0x7f54f00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:02.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 shutdown_connections 2023-10-06T21:20:02.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:02.834+0000 7f54f87ff700 1 -- 172.21.15.31:0/2807992375 wait complete. 2023-10-06T21:20:02.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.060+0000 7facb63ff700 1 -- 172.21.15.31:0/2778126170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0102a30 msgr2=0x7facb0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.060+0000 7facb63ff700 1 --2- 172.21.15.31:0/2778126170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0102a30 0x7facb0109350 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faca800b0a0 tx=0x7faca800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 -- 172.21.15.31:0/2778126170 shutdown_connections 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 --2- 172.21.15.31:0/2778126170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0102a30 0x7facb0109350 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 --2- 172.21.15.31:0/2778126170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0100e90 0x7facb01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 --2- 172.21.15.31:0/2778126170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102070 0x7facb01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 -- 172.21.15.31:0/2778126170 >> 172.21.15.31:0/2778126170 conn(0x7facb00fc430 msgr2=0x7facb00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 -- 172.21.15.31:0/2778126170 shutdown_connections 2023-10-06T21:20:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.061+0000 7facb63ff700 1 -- 172.21.15.31:0/2778126170 wait complete. 2023-10-06T21:20:03.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 Processor -- start 2023-10-06T21:20:03.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 -- start start 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0102070 0x7facb0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102a30 0x7facb01a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb01a0990 con 0x7facb0100e90 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb01a0ad0 con 0x7facb0102a30 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facb63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb01a0c60 con 0x7facb0102070 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48752/0 (socket says 172.21.15.31:48752) 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facaffff700 1 -- 172.21.15.31:0/3386038326 learned_addr learned my addr 172.21.15.31:0/3386038326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.062+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0102070 0x7facb0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facb499c700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102a30 0x7facb01a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 -- 172.21.15.31:0/3386038326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0102070 msgr2=0x7facb0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0102070 0x7facb0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 -- 172.21.15.31:0/3386038326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102a30 msgr2=0x7facb01a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102a30 0x7facb01a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 -- 172.21.15.31:0/3386038326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009520 con 0x7facb0100e90 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facaffff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faca000ea50 tx=0x7faca000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facad7fa700 1 -- 172.21.15.31:0/3386038326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca000cbd0 con 0x7facb0100e90 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca8009d00 con 0x7facb0100e90 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facad7fa700 1 -- 172.21.15.31:0/3386038326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faca000cd30 con 0x7facb0100e90 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facad7fa700 1 -- 172.21.15.31:0/3386038326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0018680 con 0x7facb0100e90 2023-10-06T21:20:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.063+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb01a11d0 con 0x7facb0100e90 2023-10-06T21:20:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.067+0000 7facad7fa700 1 -- 172.21.15.31:0/3386038326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7faca0010430 con 0x7facb0100e90 2023-10-06T21:20:03.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 Processor -- start 2023-10-06T21:20:03.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 -- start start 2023-10-06T21:20:03.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facad7fa700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 0x7fac980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facad7fa700 1 -- 172.21.15.31:0/3386038326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faca0014070 con 0x7facb0100e90 2023-10-06T21:20:03.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260abae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260a21410 con 0x559260ab9b70 2023-10-06T21:20:03.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260a12bd0 con 0x559260ab9210 2023-10-06T21:20:03.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260a104f0 con 0x559260abaa10 2023-10-06T21:20:03.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.068+0000 7facb499c700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 0x7fac980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:35404/0 (socket says 172.21.15.31:35404) 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 -- 172.21.15.31:0/4288338779 learned_addr learned my addr 172.21.15.31:0/4288338779 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaf7fe700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facb499c700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260abae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 -- 172.21.15.31:0/4288338779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 msgr2=0x559260abae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260abae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 -- 172.21.15.31:0/4288338779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 msgr2=0x559260aba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 -- 172.21.15.31:0/4288338779 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559260836d90 con 0x559260ab9210 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facb499c700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260abae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facaffff700 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faca0096b60 tx=0x7faca009a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb4f5f3c62dd3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.069+0000 7facb499c700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 0x7fac980a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faca800b780 tx=0x7faca8009320 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7fac967fc700 1 -- 172.21.15.31:0/4288338779 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca009ad60 con 0x559260ab9210 2023-10-06T21:20:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7fac967fc700 1 -- 172.21.15.31:0/4288338779 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faca009a660 con 0x559260ab9210 2023-10-06T21:20:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7fac967fc700 1 -- 172.21.15.31:0/4288338779 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00cd0f0 con 0x559260ab9210 2023-10-06T21:20:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 -- 172.21.15.31:0/4288338779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 msgr2=0x559260ab9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faca0096b60 tx=0x7faca009a1a0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 -- 172.21.15.31:0/4288338779 shutdown_connections 2023-10-06T21:20:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260abae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab9630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/4288338779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 -- 172.21.15.31:0/4288338779 >> 172.21.15.31:0/4288338779 conn(0x559260a2d210 msgr2=0x559260ab40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 -- 172.21.15.31:0/4288338779 shutdown_connections 2023-10-06T21:20:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.070+0000 7facc9d4ab80 1 -- 172.21.15.31:0/4288338779 wait complete. 2023-10-06T21:20:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.071+0000 7facc9d4ab80 1 -- start start 2023-10-06T21:20:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.071+0000 7facc9d4ab80 1 Processor -- start 2023-10-06T21:20:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9b70 0x559260ac3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260aca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260a0bde0 con 0x559260ab9210 2023-10-06T21:20:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592609a86c0 con 0x559260ab9b70 2023-10-06T21:20:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592609a8360 con 0x559260abaa10 2023-10-06T21:20:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facc9d4ab80 10 client.0 _ll_register_callbacks cb 0x5592608057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48772/0 (socket says 172.21.15.31:48772) 2023-10-06T21:20:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 -- 172.21.15.31:0/3646567326 learned_addr learned my addr 172.21.15.31:0/3646567326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9b70 0x559260ac3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facb499c700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260aca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 -- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 msgr2=0x559260aca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260aca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 -- 172.21.15.31:0/3646567326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9b70 msgr2=0x559260ac3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9b70 0x559260ac3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 -- 172.21.15.31:0/3646567326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb01a11d0 con 0x559260ab9210 2023-10-06T21:20:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7facaffff700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faca0005950 tx=0x7faca009ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1777e577235159d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.072+0000 7fac8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca009aac0 con 0x559260ab9210 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559260aabec0 con 0x559260ab9210 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faca00a29f0 con 0x559260ab9210 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faca0099b50 con 0x559260ab9210 2023-10-06T21:20:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7facc9d4ab80 10 client.15174 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55926081fc10 con 0x559260ab9210 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559260aabec0 con 0x559260ab9210 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faca00964e0 con 0x559260ab9210 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.073+0000 7facc9d4ab80 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facc9d4ab80 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facc9d4ab80 20 client.15174 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260ace730 0x559260acd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facc9d4ab80 4 client.15174 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559260a04dc0 con 0x559260ace730 2023-10-06T21:20:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260ace730 0x559260acd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260ace730 0x559260acd3f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53e0060ee1dcc9f8 server_cookie=271ca80ff849700b in_seq=0 out_seq=0 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.074+0000 7fac8f7fe700 10 client.15174 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.075+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3646567326 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559260a04dc0 con 0x559260ace730 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.075+0000 7fac8f7fe700 10 client.15174 handle_command_reply: tid=0 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.075+0000 7facc9d4ab80 1 client.15174 shutdown 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.075+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 msgr2=0x559260ac2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.075+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faca0005950 tx=0x7faca009ad60 comp rx=0 tx=0).stop 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 shutdown_connections 2023-10-06T21:20:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260ace730 0x559260acd3f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260aca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9210 0x559260ac2fd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7fac8f7fe700 0 client.15174 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3646567326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9b70 0x559260ac3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 >> 172.21.15.31:0/3646567326 conn(0x559260a2d210 msgr2=0x559260ab3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 shutdown_connections 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3646567326 wait complete. 2023-10-06T21:20:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.076+0000 7facc9d4ab80 20 client.15174 trim_cache size 0 max 16384 2023-10-06T21:20:03.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 Processor -- start 2023-10-06T21:20:03.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 -- start start 2023-10-06T21:20:03.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ab3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926088c8d0 con 0x559260ab9b70 2023-10-06T21:20:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260a07ed0 con 0x559260ab9210 2023-10-06T21:20:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592609b24d0 con 0x559260abaa10 2023-10-06T21:20:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.095+0000 7facaf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48780/0 (socket says 172.21.15.31:48780) 2023-10-06T21:20:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 -- 172.21.15.31:0/527614140 learned_addr learned my addr 172.21.15.31:0/527614140 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaffff700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facb499c700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ab3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 -- 172.21.15.31:0/527614140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 msgr2=0x559260ab3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ab3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 -- 172.21.15.31:0/527614140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 msgr2=0x559260ab2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 -- 172.21.15.31:0/527614140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55926085d200 con 0x559260ab9b70 2023-10-06T21:20:03.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaffff700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7facaf7fe700 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faca4000c00 tx=0x7faca4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef2b675a588881a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7fac96ffd700 1 -- 172.21.15.31:0/527614140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4017070 con 0x559260ab9b70 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7fac96ffd700 1 -- 172.21.15.31:0/527614140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faca4018040 con 0x559260ab9b70 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.096+0000 7fac96ffd700 1 -- 172.21.15.31:0/527614140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4003c70 con 0x559260ab9b70 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 -- 172.21.15.31:0/527614140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 msgr2=0x559260aca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faca4000c00 tx=0x7faca4009260 comp rx=0 tx=0).stop 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 -- 172.21.15.31:0/527614140 shutdown_connections 2023-10-06T21:20:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ab3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ab2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/527614140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260aca510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 -- 172.21.15.31:0/527614140 >> 172.21.15.31:0/527614140 conn(0x559260a2d210 msgr2=0x559260ac1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 -- 172.21.15.31:0/527614140 shutdown_connections 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.097+0000 7facc9d4ab80 1 -- 172.21.15.31:0/527614140 wait complete. 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 -- start start 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 Processor -- start 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ac9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ac9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260aca570 con 0x559260ab9b70 2023-10-06T21:20:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260aca6b0 con 0x559260ab9210 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559260aca870 con 0x559260abaa10 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facc9d4ab80 10 client.0 _ll_register_callbacks cb 0x5592608057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.098+0000 7facaf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48784/0 (socket says 172.21.15.31:48784) 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaf7fe700 1 -- 172.21.15.31:0/3684488031 learned_addr learned my addr 172.21.15.31:0/3684488031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ac9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facb499c700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ac9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 -- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 msgr2=0x559260ac9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ac9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 -- 172.21.15.31:0/3684488031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 msgr2=0x559260ac9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 -- 172.21.15.31:0/3684488031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4009b30 con 0x559260ab9210 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facaffff700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ac9520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faca009a360 tx=0x7faca000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac2bb99e24b6c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7fac8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca009add0 con 0x559260ab9210 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faca00993c0 con 0x559260ab9210 2023-10-06T21:20:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.099+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faca009a660 con 0x559260ab9210 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faca00a29f0 con 0x559260ab9210 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 10 client.25019 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559260835260 con 0x559260ab9210 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faca009a660 con 0x559260ab9210 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faca009f4f0 con 0x559260ab9210 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 10 client.25019 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 10 client.25019 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T21:20:03.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 20 client.25019 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260b26e20 0x559260b232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 4 client.25019 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.100+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559260a04ae0 con 0x559260b26e20 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260b26e20 0x559260b232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7facaf7fe700 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260b26e20 0x559260b232a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87083b48beb99282 server_cookie=89ea803a5d69596c in_seq=0 out_seq=0 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7fac8f7fe700 10 client.25019 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7fac8f7fe700 1 -- 172.21.15.31:0/3684488031 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559260a04ae0 con 0x559260b26e20 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7fac8f7fe700 10 client.25019 handle_command_reply: tid=0 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.101+0000 7facc9d4ab80 1 client.25019 shutdown 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 msgr2=0x559260ac9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ac9520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faca009a360 tx=0x7faca000eef0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 shutdown_connections 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559260b26e20 0x559260b232a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559260abaa10 0x559260ac9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559260ab9210 0x559260ac9520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7fac8f7fe700 0 client.25019 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 --2- 172.21.15.31:0/3684488031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559260ab9b70 0x559260ac9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 >> 172.21.15.31:0/3684488031 conn(0x559260a2d210 msgr2=0x559260ab2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 shutdown_connections 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 1 -- 172.21.15.31:0/3684488031 wait complete. 2023-10-06T21:20:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facc9d4ab80 20 client.25019 trim_cache size 0 max 16384 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24565, 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 65.218278022000007, 2023-10-06T21:20:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 88, 2023-10-06T21:20:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 84.257954470000001 2023-10-06T21:20:03.120 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 msgr2=0x7fac980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 0x7fac980a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faca800b780 tx=0x7faca8009320 comp rx=0 tx=0).stop 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 msgr2=0x7facb0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.102+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faca000ea50 tx=0x7faca000ed60 comp rx=0 tx=0).stop 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 shutdown_connections 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fac980a1510 0x7fac980a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0102a30 0x7facb01a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb0100e90 0x7facb0198e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 --2- 172.21.15.31:0/3386038326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb0102070 0x7facb0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 >> 172.21.15.31:0/3386038326 conn(0x7facb00fc430 msgr2=0x7facb0107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 shutdown_connections 2023-10-06T21:20:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.103+0000 7facb63ff700 1 -- 172.21.15.31:0/3386038326 wait complete. 2023-10-06T21:20:03.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:03.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.122 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/885164144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:03.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:03.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:03.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:02 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/885164144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:03.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:02 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 -- 172.21.15.31:0/1190344716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 msgr2=0x7f3c541012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1190344716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c541012b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009990 tx=0x7f3c44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 -- 172.21.15.31:0/1190344716 shutdown_connections 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1190344716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 0x7f3c54109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1190344716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c541012b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1190344716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 0x7f3c541024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 -- 172.21.15.31:0/1190344716 >> 172.21.15.31:0/1190344716 conn(0x7f3c540fc430 msgr2=0x7f3c540fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 -- 172.21.15.31:0/1190344716 shutdown_connections 2023-10-06T21:20:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.324+0000 7f3c58bff700 1 -- 172.21.15.31:0/1190344716 wait complete. 2023-10-06T21:20:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.325+0000 7f3c58bff700 1 Processor -- start 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 -- start start 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 0x7f3c54197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 0x7f3c5419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5419e680 con 0x7f3c54100e90 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5419e7c0 con 0x7f3c54102a30 2023-10-06T21:20:03.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c58bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5419e950 con 0x7f3c54102070 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48798/0 (socket says 172.21.15.31:48798) 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1886153603 learned_addr learned my addr 172.21.15.31:0/1886153603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 0x7f3c5419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 0x7f3c54197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1886153603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 msgr2=0x7f3c54197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 0x7f3c54197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1886153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 msgr2=0x7f3c5419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 0x7f3c5419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.326+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1886153603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c440095f0 con 0x7f3c54100e90 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c010 tx=0x7f3c44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c4bfff700 1 -- 172.21.15.31:0/1886153603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4401d070 con 0x7f3c54100e90 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c4bfff700 1 -- 172.21.15.31:0/1886153603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c440054c0 con 0x7f3c54100e90 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5419ebd0 con 0x7f3c54100e90 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c4bfff700 1 -- 172.21.15.31:0/1886153603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44003ab0 con 0x7f3c54100e90 2023-10-06T21:20:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.327+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5419f050 con 0x7f3c54100e90 2023-10-06T21:20:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.330+0000 7f3c4bfff700 1 -- 172.21.15.31:0/1886153603 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3c440047a0 con 0x7f3c54100e90 2023-10-06T21:20:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c4bfff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 0x7f3c400a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 Processor -- start 2023-10-06T21:20:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c4bfff700 1 -- 172.21.15.31:0/1886153603 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3c440cde20 con 0x7f3c54100e90 2023-10-06T21:20:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 0x7f3c400a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 -- start start 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e829410 con 0x56357e8c1210 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e81abd0 con 0x56357e8c1b70 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e8184f0 con 0x56357e8c2a10 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.332+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 0x7f3c400a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005950 tx=0x7f3c3c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48804/0 (socket says 172.21.15.31:48804) 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1755968229 learned_addr learned my addr 172.21.15.31:0/1755968229 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1755968229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 msgr2=0x56357e8c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1755968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 msgr2=0x56357e8c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1755968229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357e63ed90 con 0x56357e8c1210 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005ac0 tx=0x7f3c44005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b495aa4cc97b4df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.333+0000 7f3c48d99700 1 -- 172.21.15.31:0/1755968229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4409f1a0 con 0x56357e8c1210 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c48d99700 1 -- 172.21.15.31:0/1755968229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c440a3350 con 0x56357e8c1210 2023-10-06T21:20:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c48d99700 1 -- 172.21.15.31:0/1755968229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4409a540 con 0x56357e8c1210 2023-10-06T21:20:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1755968229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 msgr2=0x56357e8c1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005ac0 tx=0x7f3c44005720 comp rx=0 tx=0).stop 2023-10-06T21:20:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1755968229 shutdown_connections 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8c1630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1755968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1755968229 >> 172.21.15.31:0/1755968229 conn(0x56357e835210 msgr2=0x56357e8bc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1755968229 shutdown_connections 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.334+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1755968229 wait complete. 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.335+0000 7f3c6c59cb80 1 -- start start 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.335+0000 7f3c6c59cb80 1 Processor -- start 2023-10-06T21:20:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 0x56357e8cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e813de0 con 0x56357e8c2a10 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e7b06c0 con 0x56357e8c1b70 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e7b0360 con 0x56357e8c1210 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c6c59cb80 10 client.0 _ll_register_callbacks cb 0x56357e60d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48820/0 (socket says 172.21.15.31:48820) 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c52d9d700 1 -- 172.21.15.31:0/1609684211 learned_addr learned my addr 172.21.15.31:0/1609684211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.336+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 0x56357e8cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 -- 172.21.15.31:0/1609684211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 msgr2=0x56357e8cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 0x56357e8cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 -- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 msgr2=0x56357e8cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 -- 172.21.15.31:0/1609684211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5419f050 con 0x56357e8c2a10 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 0x56357e8cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c012da0 tx=0x7f3c4c010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1b4fa819aaabe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c004bb0 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56357e8b3ec0 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.337+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3c4c004d10 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c4c010b30 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 10 client.15192 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56357e627c10 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56357e8b3ec0 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c4c0143a0 con 0x56357e8c2a10 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 20 client.15192 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:03.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e8d68d0 0x56357e8d5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 4 client.15192 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.338+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357e80cdc0 con 0x56357e8d68d0 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.339+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e8d68d0 0x56357e8d5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.339+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e8d68d0 0x56357e8d5370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f80c9668286508 server_cookie=afedf74120527d70 in_seq=0 out_seq=0 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.339+0000 7f3c39ffb700 10 client.15192 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1609684211 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56357e80cdc0 con 0x56357e8d68d0 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c39ffb700 10 client.15192 handle_command_reply: tid=0 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 client.15192 shutdown 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 msgr2=0x56357e8d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c012da0 tx=0x7f3c4c010520 comp rx=0 tx=0).stop 2023-10-06T21:20:03.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 shutdown_connections 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e8d68d0 0x56357e8d5370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c2a10 0x56357e8d2260 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c1210 0x56357e8cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1609684211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c39ffb700 0 client.15192 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 >> 172.21.15.31:0/1609684211 conn(0x56357e835210 msgr2=0x56357e8bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 shutdown_connections 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.340+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1609684211 wait complete. 2023-10-06T21:20:03.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.341+0000 7f3c6c59cb80 20 client.15192 trim_cache size 0 max 16384 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 Processor -- start 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 -- start start 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1210 0x56357e8bbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e6948d0 con 0x56357e8c1b70 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e8bb290 con 0x56357e8c1210 2023-10-06T21:20:03.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e7ba4d0 con 0x56357e8c2a10 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48824/0 (socket says 172.21.15.31:48824) 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c5259c700 1 -- 172.21.15.31:0/3378537894 learned_addr learned my addr 172.21.15.31:0/3378537894 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.359+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1210 0x56357e8bbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 -- 172.21.15.31:0/3378537894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 msgr2=0x56357e8c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 -- 172.21.15.31:0/3378537894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1210 msgr2=0x56357e8bbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1210 0x56357e8bbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 -- 172.21.15.31:0/3378537894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357e665200 con 0x56357e8c1b70 2023-10-06T21:20:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c5259c700 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011520 tx=0x7f3c3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c307a385490992d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c4959a700 1 -- 172.21.15.31:0/3378537894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c029470 con 0x56357e8c1b70 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c4959a700 1 -- 172.21.15.31:0/3378537894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c3c029a60 con 0x56357e8c1b70 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.360+0000 7f3c4959a700 1 -- 172.21.15.31:0/3378537894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c020940 con 0x56357e8c1b70 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/3378537894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 msgr2=0x56357e8d2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011520 tx=0x7f3c3c011000 comp rx=0 tx=0).stop 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/3378537894 shutdown_connections 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1210 0x56357e8bbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/3378537894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1b70 0x56357e8d2180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/3378537894 >> 172.21.15.31:0/3378537894 conn(0x56357e835210 msgr2=0x56357e8df230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/3378537894 shutdown_connections 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/3378537894 wait complete. 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.361+0000 7f3c6c59cb80 1 -- start start 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 Processor -- start 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e8cee20 con 0x56357e8c1210 2023-10-06T21:20:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e8cef90 con 0x56357e8c1b70 2023-10-06T21:20:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357e8cf150 con 0x56357e8c2a10 2023-10-06T21:20:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c6c59cb80 10 client.0 _ll_register_callbacks cb 0x7f3c4c012660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48832/0 (socket says 172.21.15.31:48832) 2023-10-06T21:20:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1454239541 learned_addr learned my addr 172.21.15.31:0/1454239541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.362+0000 7f3c52d9d700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1454239541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 msgr2=0x56357e8ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 msgr2=0x56357e8d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 -- 172.21.15.31:0/1454239541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c00ed20 con 0x56357e8c1210 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c4a59c700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3c440095c0 tx=0x7f3c4409b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68783181b44fbf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440a10b0 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3c440a1210 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c440a3350 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c440b90c0 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c6c59cb80 10 client.15198 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56357e63d260 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.363+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c440a3350 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c440a2ec0 con 0x56357e8c1210 2023-10-06T21:20:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 20 client.15198 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e92edc0 0x56357e92b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 4 client.15198 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56357e80cae0 con 0x56357e92edc0 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.364+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e92edc0 0x56357e92b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c5259c700 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e92edc0 0x56357e92b1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382b2af15374bb39 server_cookie=aa1ae2b5edcb6649 in_seq=0 out_seq=0 2023-10-06T21:20:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c39ffb700 10 client.15198 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c39ffb700 1 -- 172.21.15.31:0/1454239541 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56357e80cae0 con 0x56357e92edc0 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c39ffb700 10 client.15198 handle_command_reply: tid=0 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 client.15198 shutdown 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 msgr2=0x56357e8d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3c440095c0 tx=0x7f3c4409b890 comp rx=0 tx=0).stop 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 shutdown_connections 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56357e8c2a10 0x56357e8ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56357e92edc0 0x56357e92b1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357e8c1210 0x56357e8d1ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 --2- 172.21.15.31:0/1454239541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56357e8c1b70 0x56357e8d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c39ffb700 0 client.15198 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 >> 172.21.15.31:0/1454239541 conn(0x56357e835210 msgr2=0x56357e8bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 shutdown_connections 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 1 -- 172.21.15.31:0/1454239541 wait complete. 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.365+0000 7f3c6c59cb80 20 client.15198 trim_cache size 0 max 16384 2023-10-06T21:20:03.376 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24602, 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 68.505680347999999, 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 88, 2023-10-06T21:20:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 75.232317796999993 2023-10-06T21:20:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 msgr2=0x7f3c400a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 0x7f3c400a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005950 tx=0x7f3c3c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 msgr2=0x7f3c54196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c010 tx=0x7f3c44004730 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 shutdown_connections 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c400a1590 0x7f3c400a3a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c54102a30 0x7f3c5419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54100e90 0x7f3c54196b00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 --2- 172.21.15.31:0/1886153603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c54102070 0x7f3c54197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 >> 172.21.15.31:0/1886153603 conn(0x7f3c540fc430 msgr2=0x7f3c540fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 shutdown_connections 2023-10-06T21:20:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:03.366+0000 7f3c58bff700 1 -- 172.21.15.31:0/1886153603 wait complete. 2023-10-06T21:20:03.387 INFO:tasks.cephadm:Setup complete, yielding 2023-10-06T21:20:03.387 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T21:20:03.398 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:20:03.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/885164144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:03.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:03.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:02 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' 2023-10-06T21:20:04.135 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:05.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:04 smithi031 ceph-mon[98539]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:04 smithi142 ceph-mon[102951]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:04 smithi146 ceph-mon[102809]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.710+0000 7f70999ef700 1 -- 172.21.15.31:0/1470435011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 msgr2=0x7f70941093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.710+0000 7f70999ef700 1 --2- 172.21.15.31:0/1470435011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f70941093c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f708c00b0d0 tx=0x7f708c00b3e0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 -- 172.21.15.31:0/1470435011 shutdown_connections 2023-10-06T21:20:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 --2- 172.21.15.31:0/1470435011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f70941093c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 --2- 172.21.15.31:0/1470435011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 0x7f7094101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 --2- 172.21.15.31:0/1470435011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 0x7f70941025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 -- 172.21.15.31:0/1470435011 >> 172.21.15.31:0/1470435011 conn(0x7f70940fc4d0 msgr2=0x7f70940fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 -- 172.21.15.31:0/1470435011 shutdown_connections 2023-10-06T21:20:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.711+0000 7f70999ef700 1 -- 172.21.15.31:0/1470435011 wait complete. 2023-10-06T21:20:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 Processor -- start 2023-10-06T21:20:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 -- start start 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 0x7f7094196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 0x7f7094197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419e780 con 0x7f7094102b10 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419e8c0 con 0x7f7094100f70 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.712+0000 7f70999ef700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419ea80 con 0x7f7094102170 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48838/0 (socket says 172.21.15.31:48838) 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 -- 172.21.15.31:0/3693414810 learned_addr learned my addr 172.21.15.31:0/3693414810 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f7092ffd700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 0x7f7094196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:05.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70927fc700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 0x7f7094197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 -- 172.21.15.31:0/3693414810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 msgr2=0x7f7094197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 0x7f7094197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 -- 172.21.15.31:0/3693414810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 msgr2=0x7f7094196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 0x7f7094196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 -- 172.21.15.31:0/3693414810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c009d00 con 0x7f7094102b10 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70937fe700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d010 tx=0x7f708c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00e040 con 0x7f7094102b10 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f708c003e10 con 0x7f7094102b10 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.713+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709419ed00 con 0x7f7094102b10 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.714+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c0129f0 con 0x7f7094102b10 2023-10-06T21:20:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.714+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709419f110 con 0x7f7094102b10 2023-10-06T21:20:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.715+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7094108510 con 0x7f7094102b10 2023-10-06T21:20:05.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.717+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f708c012b70 con 0x7f7094102b10 2023-10-06T21:20:05.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.718+0000 7f70989ed700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 0x7f707c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:05.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.718+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f708c0ce7b0 con 0x7f7094102b10 2023-10-06T21:20:05.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.718+0000 7f7092ffd700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 0x7f707c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:05.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.719+0000 7f7092ffd700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 0x7f707c0a3a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7094198110 tx=0x7f708400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.722+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f708c097cc0 con 0x7f7094102b10 2023-10-06T21:20:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.949+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f709410c630 con 0x7f707c0a1590 2023-10-06T21:20:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.950+0000 7f70989ed700 1 -- 172.21.15.31:0/3693414810 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f709410c630 con 0x7f707c0a1590 2023-10-06T21:20:05.953 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2023-10-06T21:20:05.953 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2023-10-06T21:20:05.953 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 msgr2=0x7f707c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 0x7f707c0a3a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7094198110 tx=0x7f708400b410 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 msgr2=0x7f709419e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d010 tx=0x7f708c0094c0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 shutdown_connections 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f707c0a1590 0x7f707c0a3a50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7094102b10 0x7f709419e1b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7094100f70 0x7f7094196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 --2- 172.21.15.31:0/3693414810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7094102170 0x7f7094197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.952+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 >> 172.21.15.31:0/3693414810 conn(0x7f70940fc4d0 msgr2=0x7f7094105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.953+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 shutdown_connections 2023-10-06T21:20:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:05.953+0000 7f70999ef700 1 -- 172.21.15.31:0/3693414810 wait complete. 2023-10-06T21:20:06.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-06T21:20:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:06 smithi142 ceph-mon[102951]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:07.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:06 smithi142 ceph-mon[102951]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:07.156 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:06 smithi031 ceph-mon[98539]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:07.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:06 smithi031 ceph-mon[98539]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:06 smithi146 ceph-mon[102809]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:06 smithi146 ceph-mon[102809]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 -- 172.21.15.31:0/704826490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102b10 msgr2=0x7f3c801093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 --2- 172.21.15.31:0/704826490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102b10 0x7f3c801093c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b0d0 tx=0x7f3c7800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 -- 172.21.15.31:0/704826490 shutdown_connections 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 --2- 172.21.15.31:0/704826490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102b10 0x7f3c801093c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 --2- 172.21.15.31:0/704826490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 --2- 172.21.15.31:0/704826490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102170 0x7f3c801025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 -- 172.21.15.31:0/704826490 >> 172.21.15.31:0/704826490 conn(0x7f3c800fc4d0 msgr2=0x7f3c800fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 -- 172.21.15.31:0/704826490 shutdown_connections 2023-10-06T21:20:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.644+0000 7f3c869db700 1 -- 172.21.15.31:0/704826490 wait complete. 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.645+0000 7f3c869db700 1 Processor -- start 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.645+0000 7f3c869db700 1 -- start start 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 0x7f3c801a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801a0a10 con 0x7f3c80102170 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801a0b50 con 0x7f3c80100f70 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c869db700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801a0ce0 con 0x7f3c80102b10 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48850/0 (socket says 172.21.15.31:48850) 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:35542/0 (socket says 172.21.15.31:35542) 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c7f7fe700 1 -- 172.21.15.31:0/3723660488 learned_addr learned my addr 172.21.15.31:0/3723660488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.646+0000 7f3c84f78700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 0x7f3c801a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 -- 172.21.15.31:0/3723660488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 msgr2=0x7f3c801a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 0x7f3c801a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 -- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 msgr2=0x7f3c801993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 -- 172.21.15.31:0/3723660488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78009d00 con 0x7f3c80100f70 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7f7fe700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c84f78700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 0x7f3c801a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7ffff700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000d740 tx=0x7f3c7000da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70004220 con 0x7f3c80100f70 2023-10-06T21:20:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c801a0fc0 con 0x7f3c80100f70 2023-10-06T21:20:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c70004380 con 0x7f3c80100f70 2023-10-06T21:20:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70003c10 con 0x7f3c80100f70 2023-10-06T21:20:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.647+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c801a13e0 con 0x7f3c80100f70 2023-10-06T21:20:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.648+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80193470 con 0x7f3c80100f70 2023-10-06T21:20:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.651+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3c70003d70 con 0x7f3c80100f70 2023-10-06T21:20:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.652+0000 7f3c7d7fa700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 0x7f3c680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.652+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3c700cca40 con 0x7f3c80100f70 2023-10-06T21:20:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.652+0000 7f3c7f7fe700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 0x7f3c680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.653+0000 7f3c7f7fe700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 0x7f3c680a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c740104e0 tx=0x7f3c74007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.654+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3c70095ec0 con 0x7f3c80100f70 2023-10-06T21:20:08.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.874+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c8002d2b0 con 0x7f3c680a1510 2023-10-06T21:20:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.888+0000 7f3c7d7fa700 1 -- 172.21.15.31:0/3723660488 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f3c8002d2b0 con 0x7f3c680a1510 2023-10-06T21:20:08.891 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-06T21:20:08.891 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (84s) 6s ago 83s 16.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 a2f388d69f85 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi142 running (81s) 7s ago 80s 29.4M - 18.0.0-6616-g14d8d032 c754feb42fd5 1c6624c21e61 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi146 running (96s) 7s ago 96s 16.9M - 18.0.0-6616-g14d8d032 c754feb42fd5 4e3072c9ec4c 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (106s) 6s ago 105s 13.9M - 18.0.0-6616-g14d8d032 c754feb42fd5 ad1280ec686e 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi142 running (102s) 7s ago 102s 18.1M - 18.0.0-6616-g14d8d032 c754feb42fd5 86d74ae6dc85 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi146 running (99s) 7s ago 99s 18.3M - 18.0.0-6616-g14d8d032 c754feb42fd5 2edef506065c 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (87s) 6s ago 86s 16.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 b488988d89df 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi142 running (93s) 7s ago 93s 18.3M - 18.0.0-6616-g14d8d032 c754feb42fd5 f27725d1278e 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi146 running (90s) 7s ago 90s 18.5M - 18.0.0-6616-g14d8d032 c754feb42fd5 baf3c43e16a4 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (109s) 6s ago 108s 13.9M - 18.0.0-6616-g14d8d032 c754feb42fd5 2cc9e2cfbfe7 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi142 running (112s) 7s ago 112s 18.1M - 18.0.0-6616-g14d8d032 c754feb42fd5 584618a3fa7e 2023-10-06T21:20:08.892 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi146 running (115s) 7s ago 115s 16.9M - 18.0.0-6616-g14d8d032 c754feb42fd5 3d1747b0af87 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 6s ago 9m 527M - 18.0.0-6616-g14d8d032 c754feb42fd5 eab7bc355e22 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi142 *:8443,8765 running (7m) 7s ago 7m 454M - 18.0.0-6616-g14d8d032 c754feb42fd5 a531a2720592 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi146 *:8443,8765 running (7m) 7s ago 7m 452M - 18.0.0-6616-g14d8d032 c754feb42fd5 ff57dd51b63c 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 6s ago 9m 99.2M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 41c3e23ba601 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi142 running (7m) 7s ago 7m 64.7M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 10acf701570c 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi146 running (7m) 7s ago 7m 65.8M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 a059c332cf6d 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 6s ago 7m 69.4M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 697a4e518016 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi142 running (6m) 7s ago 6m 76.3M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 5eb4c77784e9 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi146 running (6m) 7s ago 6m 74.1M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 46f7d4ce560f 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (5m) 6s ago 5m 64.0M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 6603be31d6b8 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi142 running (5m) 7s ago 5m 62.0M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 443dbce564e4 2023-10-06T21:20:08.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi146 running (4m) 7s ago 4m 66.2M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 20195a50d0b8 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 6s ago 4m 60.5M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 d6a5ee8e3fcb 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi142 running (4m) 7s ago 4m 65.0M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 629175c4274a 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi146 running (3m) 7s ago 3m 62.3M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 97e8f28290c1 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 6s ago 3m 56.8M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 720759ec3383 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi142 running (2m) 7s ago 2m 63.2M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 1acb4af0514b 2023-10-06T21:20:08.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi146 running (2m) 7s ago 2m 53.0M 4316M 18.0.0-6616-g14d8d032 c754feb42fd5 4ba32dda1f3b 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.890+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 msgr2=0x7f3c680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.890+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 0x7f3c680a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c740104e0 tx=0x7f3c74007480 comp rx=0 tx=0).stop 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 msgr2=0x7f3c80198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000d740 tx=0x7f3c7000da50 comp rx=0 tx=0).stop 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 shutdown_connections 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c680a1510 0x7f3c680a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c80102b10 0x7f3c801a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c80100f70 0x7f3c80198e80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 --2- 172.21.15.31:0/3723660488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c80102170 0x7f3c801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 >> 172.21.15.31:0/3723660488 conn(0x7f3c800fc4d0 msgr2=0x7f3c80107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 shutdown_connections 2023-10-06T21:20:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:08.891+0000 7f3c869db700 1 -- 172.21.15.31:0/3723660488 wait complete. 2023-10-06T21:20:09.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:08 smithi142 ceph-mon[102951]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:08 smithi031 ceph-mon[98539]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:08 smithi146 ceph-mon[102809]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:09.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-06T21:20:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:10.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:09 smithi142 ceph-mon[102951]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:09 smithi031 ceph-mon[98539]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:10.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:09 smithi146 ceph-mon[102809]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:10.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:10 smithi031 ceph-mon[98539]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:11.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:10 smithi142 ceph-mon[102951]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:11.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:10 smithi146 ceph-mon[102809]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 -- 172.21.15.31:0/1887030659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 msgr2=0x7f01781093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 --2- 172.21.15.31:0/1887030659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f01781093c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f017000b0d0 tx=0x7f017000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 -- 172.21.15.31:0/1887030659 shutdown_connections 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 --2- 172.21.15.31:0/1887030659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f01781093c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 --2- 172.21.15.31:0/1887030659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178100f70 0x7f0178101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 --2- 172.21.15.31:0/1887030659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178102170 0x7f01781025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 -- 172.21.15.31:0/1887030659 >> 172.21.15.31:0/1887030659 conn(0x7f01780fc4d0 msgr2=0x7f01780fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 -- 172.21.15.31:0/1887030659 shutdown_connections 2023-10-06T21:20:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.602+0000 7f017d52d700 1 -- 172.21.15.31:0/1887030659 wait complete. 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.603+0000 7f017d52d700 1 Processor -- start 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.603+0000 7f017d52d700 1 -- start start 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 0x7f0178196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178102170 0x7f0178197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819e780 con 0x7f0178102b10 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819e8c0 con 0x7f0178100f70 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f017d52d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819ea80 con 0x7f0178102170 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48860/0 (socket says 172.21.15.31:48860) 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 -- 172.21.15.31:0/3236529234 learned_addr learned my addr 172.21.15.31:0/3236529234 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01767fc700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178102170 0x7f0178197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f0176ffd700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 0x7f0178196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 -- 172.21.15.31:0/3236529234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178102170 msgr2=0x7f0178197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178102170 0x7f0178197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 -- 172.21.15.31:0/3236529234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 msgr2=0x7f0178196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 0x7f0178196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f01777fe700 1 -- 172.21.15.31:0/3236529234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0170009d00 con 0x7f0178102b10 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.604+0000 7f0176ffd700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 0x7f0178196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f01777fe700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f017000d010 tx=0x7f01700094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000e040 con 0x7f0178102b10 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017819ed00 con 0x7f0178102b10 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0170003e10 con 0x7f0178102b10 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01700129f0 con 0x7f0178102b10 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.605+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017819f120 con 0x7f0178102b10 2023-10-06T21:20:11.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.606+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0178107eb0 con 0x7f0178102b10 2023-10-06T21:20:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.608+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0170012b70 con 0x7f0178102b10 2023-10-06T21:20:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.610+0000 7f015ffff700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 0x7f01600a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.610+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01700ce6e0 con 0x7f0178102b10 2023-10-06T21:20:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.610+0000 7f0176ffd700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 0x7f01600a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.610+0000 7f0176ffd700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 0x7f01600a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0168005950 tx=0x7f01680058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.614+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0170097bf0 con 0x7f0178102b10 2023-10-06T21:20:11.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.832+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f017802d270 con 0x7f01600a1510 2023-10-06T21:20:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.838+0000 7f015ffff700 1 -- 172.21.15.31:0/3236529234 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f017802d270 con 0x7f01600a1510 2023-10-06T21:20:11.841 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-06T21:20:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 10s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2023-10-06T21:20:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 10s ago 7m smithi031=x;smithi142=y;smithi146=z;count:3 2023-10-06T21:20:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 10s ago 8m smithi031:172.21.15.31=a;smithi142:172.21.15.142=b;smithi146:172.21.15.146=c;count:3 2023-10-06T21:20:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 10s ago - 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 msgr2=0x7f01600a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 0x7f01600a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0168005950 tx=0x7f01680058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 msgr2=0x7f017819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f017000d010 tx=0x7f01700094c0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 shutdown_connections 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f01600a1510 0x7f01600a39d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0178102b10 0x7f017819e1b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0178100f70 0x7f0178196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 --2- 172.21.15.31:0/3236529234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0178102170 0x7f0178197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 >> 172.21.15.31:0/3236529234 conn(0x7f01780fc4d0 msgr2=0x7f01780fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 shutdown_connections 2023-10-06T21:20:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:11.841+0000 7f017d52d700 1 -- 172.21.15.31:0/3236529234 wait complete. 2023-10-06T21:20:12.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-06T21:20:12.998 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:12 smithi142 ceph-mon[102951]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:13.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:12 smithi142 ceph-mon[102951]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:12 smithi031 ceph-mon[98539]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:13.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:12 smithi031 ceph-mon[98539]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:12 smithi146 ceph-mon[102809]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:12 smithi146 ceph-mon[102809]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/1381098772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88102b10 msgr2=0x7f2f881093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/1381098772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88102b10 0x7f2f881093c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b0d0 tx=0x7f2f8000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/1381098772 shutdown_connections 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/1381098772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88102b10 0x7f2f881093c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/1381098772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88100f70 0x7f2f88101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/1381098772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/1381098772 >> 172.21.15.31:0/1381098772 conn(0x7f2f880fc4d0 msgr2=0x7f2f880fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/1381098772 shutdown_connections 2023-10-06T21:20:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.520+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/1381098772 wait complete. 2023-10-06T21:20:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 Processor -- start 2023-10-06T21:20:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 -- start start 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 0x7f2f881a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881a0a20 con 0x7f2f88100f70 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881a0b60 con 0x7f2f88102170 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.521+0000 7f2f8e4d3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881a0cf0 con 0x7f2f88102b10 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54826/0 (socket says 172.21.15.31:54826) 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 -- 172.21.15.31:0/2767174825 learned_addr learned my addr 172.21.15.31:0/2767174825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f877fe700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 -- 172.21.15.31:0/2767174825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 msgr2=0x7f2f881a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f8ca70700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 0x7f2f881a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 0x7f2f881a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 -- 172.21.15.31:0/2767174825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 msgr2=0x7f2f881993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 -- 172.21.15.31:0/2767174825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80009d00 con 0x7f2f88100f70 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f877fe700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f8ca70700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 0x7f2f881a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.522+0000 7f2f87fff700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b530 tx=0x7f2f7800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.523+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78011830 con 0x7f2f88100f70 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.523+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f78011e20 con 0x7f2f88100f70 2023-10-06T21:20:14.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.523+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78010dd0 con 0x7f2f88100f70 2023-10-06T21:20:14.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.523+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f881a0fd0 con 0x7f2f88100f70 2023-10-06T21:20:14.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.523+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f881a1410 con 0x7f2f88100f70 2023-10-06T21:20:14.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.524+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f88193500 con 0x7f2f88100f70 2023-10-06T21:20:14.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.526+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2f78011990 con 0x7f2f88100f70 2023-10-06T21:20:14.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.527+0000 7f2f857fa700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 0x7f2f700a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:14.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.527+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2f780cc840 con 0x7f2f88100f70 2023-10-06T21:20:14.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.527+0000 7f2f877fe700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 0x7f2f700a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:14.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.528+0000 7f2f877fe700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 0x7f2f700a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005fd0 tx=0x7f2f7c005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:14.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.532+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2f78095dd0 con 0x7f2f88100f70 2023-10-06T21:20:14.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.746+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f8810c710 con 0x7f2f700a1590 2023-10-06T21:20:14.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.748+0000 7f2f857fa700 1 -- 172.21.15.31:0/2767174825 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f2f8810c710 con 0x7f2f700a1590 2023-10-06T21:20:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2023-10-06T21:20:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2023-10-06T21:20:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:smithi142 172.21.15.142 2023-10-06T21:20:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:smithi146 172.21.15.146 2023-10-06T21:20:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2023-10-06T21:20:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 msgr2=0x7f2f700a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 0x7f2f700a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005fd0 tx=0x7f2f7c005e30 comp rx=0 tx=0).stop 2023-10-06T21:20:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 msgr2=0x7f2f88198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b530 tx=0x7f2f7800b8f0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 shutdown_connections 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f2f700a1590 0x7f2f700a3a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f88102b10 0x7f2f881a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f88100f70 0x7f2f88198e90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 --2- 172.21.15.31:0/2767174825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f88102170 0x7f2f881993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 >> 172.21.15.31:0/2767174825 conn(0x7f2f880fc4d0 msgr2=0x7f2f88107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 shutdown_connections 2023-10-06T21:20:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:14.750+0000 7f2f8e4d3700 1 -- 172.21.15.31:0/2767174825 wait complete. 2023-10-06T21:20:15.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:14 smithi142 ceph-mon[102951]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:15.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:14 smithi031 ceph-mon[98539]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:15.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-06T21:20:15.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:14 smithi146 ceph-mon[102809]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:16.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:15 smithi142 ceph-mon[102951]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:16.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:15 smithi031 ceph-mon[98539]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:15 smithi146 ceph-mon[102809]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:17.117 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:16 smithi031 ceph-mon[98539]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:17.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:16 smithi142 ceph-mon[102951]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:17.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:16 smithi146 ceph-mon[102809]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:17.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 -- 172.21.15.31:0/1673290226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 msgr2=0x7fcaac073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 --2- 172.21.15.31:0/1673290226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac073d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009990 tx=0x7fca9c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 -- 172.21.15.31:0/1673290226 shutdown_connections 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 --2- 172.21.15.31:0/1673290226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 0x7fcaac10b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 --2- 172.21.15.31:0/1673290226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 --2- 172.21.15.31:0/1673290226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac073d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 -- 172.21.15.31:0/1673290226 >> 172.21.15.31:0/1673290226 conn(0x7fcaac0fc500 msgr2=0x7fcaac0fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 -- 172.21.15.31:0/1673290226 shutdown_connections 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.326+0000 7fcab4063700 1 -- 172.21.15.31:0/1673290226 wait complete. 2023-10-06T21:20:17.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.328+0000 7fcab4063700 1 Processor -- start 2023-10-06T21:20:17.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.328+0000 7fcab4063700 1 -- start start 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 0x7fcaac1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac1a08f0 con 0x7fcaac073910 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac1a0a30 con 0x7fcaac074ee0 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab4063700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac1a0bf0 con 0x7fcaac108e70 2023-10-06T21:20:17.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:17.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54844/0 (socket says 172.21.15.31:54844) 2023-10-06T21:20:17.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 -- 172.21.15.31:0/3788280246 learned_addr learned my addr 172.21.15.31:0/3788280246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:17.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab2600700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 0x7fcaac1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:17.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab15fe700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 -- 172.21.15.31:0/3788280246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 msgr2=0x7fcaac1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 0x7fcaac1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 -- 172.21.15.31:0/3788280246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 msgr2=0x7fcaac1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.329+0000 7fcab1dff700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcab1dff700 1 -- 172.21.15.31:0/3788280246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c0095f0 con 0x7fcaac073910 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcab15fe700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcab1dff700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800d950 tx=0x7fcaa800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:17.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004220 con 0x7fcaac073910 2023-10-06T21:20:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaa8007620 con 0x7fcaac073910 2023-10-06T21:20:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac1a0ed0 con 0x7fcaac073910 2023-10-06T21:20:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8005530 con 0x7fcaac073910 2023-10-06T21:20:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.330+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac1a12f0 con 0x7fcaac073910 2023-10-06T21:20:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.331+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac107520 con 0x7fcaac073910 2023-10-06T21:20:17.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.333+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fcaa8004750 con 0x7fcaac073910 2023-10-06T21:20:17.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.335+0000 7fcaa2ffd700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 0x7fca980a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:17.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.335+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcaa80ccd10 con 0x7fcaac073910 2023-10-06T21:20:17.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.335+0000 7fcab15fe700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 0x7fca980a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:17.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.336+0000 7fcab15fe700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 0x7fca980a3a50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b5c0 tx=0x7fca9c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:17.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.339+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcaa80d2020 con 0x7fcaac073910 2023-10-06T21:20:17.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.554+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaac02d2f0 con 0x7fca980a1590 2023-10-06T21:20:17.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.556+0000 7fcaa2ffd700 1 -- 172.21.15.31:0/3788280246 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fcaac02d2f0 con 0x7fca980a1590 2023-10-06T21:20:17.558 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-06T21:20:17.558 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-10-06T21:20:17.558 INFO:teuthology.orchestra.run.smithi031.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-06T21:20:17.559 INFO:teuthology.orchestra.run.smithi031.stdout:smithi146 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 msgr2=0x7fca980a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 0x7fca980a3a50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b5c0 tx=0x7fca9c005fd0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 msgr2=0x7fcaac198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800d950 tx=0x7fcaa800dc60 comp rx=0 tx=0).stop 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 shutdown_connections 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fca980a1590 0x7fca980a3a50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcaac108e70 0x7fcaac1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaac073910 0x7fcaac198da0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 --2- 172.21.15.31:0/3788280246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaac074ee0 0x7fcaac1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.558+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 >> 172.21.15.31:0/3788280246 conn(0x7fcaac0fc500 msgr2=0x7fcaac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.559+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 shutdown_connections 2023-10-06T21:20:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:17.559+0000 7fcab4063700 1 -- 172.21.15.31:0/3788280246 wait complete. 2023-10-06T21:20:18.011 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T21:20:18.020 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:20:18.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-06T21:20:18.757 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:18 smithi142 ceph-mon[102951]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:19.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:18 smithi142 ceph-mon[102951]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:19.188 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:18 smithi031 ceph-mon[98539]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:19.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:18 smithi031 ceph-mon[98539]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:18 smithi146 ceph-mon[102809]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:18 smithi146 ceph-mon[102809]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T21:20:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.284+0000 7f0840c88700 1 -- 172.21.15.31:0/3721976273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c102170 msgr2=0x7f083c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.284+0000 7f0840c88700 1 --2- 172.21.15.31:0/3721976273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c102170 0x7f083c1025d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08300099c0 tx=0x7f0830009cd0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 -- 172.21.15.31:0/3721976273 shutdown_connections 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 --2- 172.21.15.31:0/3721976273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102b10 0x7f083c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 --2- 172.21.15.31:0/3721976273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c100f70 0x7f083c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 --2- 172.21.15.31:0/3721976273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c102170 0x7f083c1025d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 -- 172.21.15.31:0/3721976273 >> 172.21.15.31:0/3721976273 conn(0x7f083c0fc4d0 msgr2=0x7f083c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 -- 172.21.15.31:0/3721976273 shutdown_connections 2023-10-06T21:20:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.285+0000 7f0840c88700 1 -- 172.21.15.31:0/3721976273 wait complete. 2023-10-06T21:20:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 Processor -- start 2023-10-06T21:20:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 -- start start 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102170 0x7f083c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 0x7f083c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1a08e0 con 0x7f083c100f70 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1a0a20 con 0x7f083c102b10 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f0840c88700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1a0be0 con 0x7f083c102170 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f083a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f083a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54858/0 (socket says 172.21.15.31:54858) 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.286+0000 7f083a59c700 1 -- 172.21.15.31:0/1397529556 learned_addr learned my addr 172.21.15.31:0/1397529556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083ad9d700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 0x7f083c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f0839d9b700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102170 0x7f083c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 -- 172.21.15.31:0/1397529556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102170 msgr2=0x7f083c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102170 0x7f083c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 -- 172.21.15.31:0/1397529556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 msgr2=0x7f083c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 0x7f083c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 -- 172.21.15.31:0/1397529556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300095f0 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083ad9d700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 0x7f083c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f083a59c700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d980 tx=0x7f082c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c00de60 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f082c0044b0 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c1a0ec0 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.287+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c003b40 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.288+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c1a12e0 con 0x7f083c100f70 2023-10-06T21:20:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.289+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c10c670 con 0x7f083c100f70 2023-10-06T21:20:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.291+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f082c003ca0 con 0x7f083c100f70 2023-10-06T21:20:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.292+0000 7f082b7fe700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 0x7f08240a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.292+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f082c0ccba0 con 0x7f083c100f70 2023-10-06T21:20:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.292+0000 7f0839d9b700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 0x7f08240a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.293+0000 7f0839d9b700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 0x7f08240a3ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f083000b580 tx=0x7f083001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.296+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f082c096020 con 0x7f083c100f70 2023-10-06T21:20:20.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.548+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f083c10c8b0 con 0x7f083c100f70 2023-10-06T21:20:20.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.549+0000 7f082b7fe700 1 -- 172.21.15.31:0/1397529556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2975 (secure 0 0 0) 0x7f082c009940 con 0x7f083c100f70 2023-10-06T21:20:20.551 INFO:teuthology.orchestra.run.smithi031.stdout:e27 2023-10-06T21:20:20.551 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:20:20.551 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 27 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-10-06T21:18:54.346752+0000 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2023-10-06T21:19:01.835058+0000 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2023-10-06T21:20:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 5 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2,3,4 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24602,1=24565,2=24526,3=24518,4=24476} 2023-10-06T21:20:20.553 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24602} state up:active seq 3 addr [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{0:24554} state up:standby-replay seq 1 addr [v2:172.21.15.142:6836/1999670279,v1:172.21.15.142:6837/1999670279] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{1:24565} state up:active seq 6 addr [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{1:24544} state up:standby-replay seq 1 addr [v2:172.21.15.146:6836/824494679,v1:172.21.15.146:6837/824494679] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{2:24526} state up:active seq 9 addr [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{2:24469} state up:standby-replay seq 1 addr [v2:172.21.15.146:6832/1755294522,v1:172.21.15.146:6833/1755294522] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{3:24518} state up:active seq 10 addr [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{3:14712} state up:standby-replay seq 1 addr [v2:172.21.15.31:6840/847340369,v1:172.21.15.31:6841/847340369] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{4:24476} state up:active seq 13 addr [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{4:14694} state up:standby-replay seq 1 addr [v2:172.21.15.31:6838/1272583602,v1:172.21.15.31:6839/1272583602] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:20.555 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:20.556 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2023-10-06T21:20:20.556 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:20.556 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14607} state up:standby seq 1 addr [v2:172.21.15.31:6834/3223954371,v1:172.21.15.31:6835/3223954371] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.556 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14619} state up:standby seq 1 addr [v2:172.21.15.31:6836/2454240095,v1:172.21.15.31:6837/2454240095] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 msgr2=0x7f08240a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 0x7f08240a3ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f083000b580 tx=0x7f083001a040 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 msgr2=0x7f083c198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d980 tx=0x7f082c00dd40 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 shutdown_connections 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f08240a1610 0x7f08240a3ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102b10 0x7f083c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f083c100f70 0x7f083c198d90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 --2- 172.21.15.31:0/1397529556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c102170 0x7f083c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 >> 172.21.15.31:0/1397529556 conn(0x7f083c0fc4d0 msgr2=0x7f083c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.551+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 shutdown_connections 2023-10-06T21:20:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:20.552+0000 7f0840c88700 1 -- 172.21.15.31:0/1397529556 wait complete. 2023-10-06T21:20:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:21.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-06T21:20:21.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:20 smithi142 ceph-mon[102951]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:21.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:20 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1397529556' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:20:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:20 smithi031 ceph-mon[98539]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:20 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1397529556' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:20:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:20 smithi146 ceph-mon[102809]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:21.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:20 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1397529556' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:20:21.774 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:23.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:22 smithi142 ceph-mon[102951]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:23.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:22 smithi031 ceph-mon[98539]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 -- 172.21.15.31:0/738806692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 msgr2=0x7fc0481093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 --2- 172.21.15.31:0/738806692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481093c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc04000b0d0 tx=0x7fc04000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 -- 172.21.15.31:0/738806692 shutdown_connections 2023-10-06T21:20:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 --2- 172.21.15.31:0/738806692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481093c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 --2- 172.21.15.31:0/738806692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 0x7fc048101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 --2- 172.21.15.31:0/738806692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 0x7fc0481025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 -- 172.21.15.31:0/738806692 >> 172.21.15.31:0/738806692 conn(0x7fc0480fc4d0 msgr2=0x7fc0480fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 -- 172.21.15.31:0/738806692 shutdown_connections 2023-10-06T21:20:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.276+0000 7fc04de03700 1 -- 172.21.15.31:0/738806692 wait complete. 2023-10-06T21:20:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.277+0000 7fc04de03700 1 Processor -- start 2023-10-06T21:20:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.277+0000 7fc04de03700 1 -- start start 2023-10-06T21:20:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 0x7fc048198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 0x7fc048199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481a09a0 con 0x7fc048102b10 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481a0ae0 con 0x7fc048102170 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc04de03700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481a0c70 con 0x7fc048100f70 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46100/0 (socket says 172.21.15.31:46100) 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 -- 172.21.15.31:0/2083128230 learned_addr learned my addr 172.21.15.31:0/2083128230 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc046ffd700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 0x7fc048199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc0477fe700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 0x7fc048198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 -- 172.21.15.31:0/2083128230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 msgr2=0x7fc048198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 0x7fc048198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.278+0000 7fc047fff700 1 -- 172.21.15.31:0/2083128230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 msgr2=0x7fc048199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:23.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc047fff700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 0x7fc048199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc047fff700 1 -- 172.21.15.31:0/2083128230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040009d00 con 0x7fc048102b10 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc047fff700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc040007880 tx=0x7fc0400078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04000e040 con 0x7fc048102b10 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc04000be30 con 0x7fc048102b10 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0481a0ef0 con 0x7fc048102b10 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04001b6c0 con 0x7fc048102b10 2023-10-06T21:20:23.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.279+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0481a1250 con 0x7fc048102b10 2023-10-06T21:20:23.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.280+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0481933d0 con 0x7fc048102b10 2023-10-06T21:20:23.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.282+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc040024400 con 0x7fc048102b10 2023-10-06T21:20:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.283+0000 7fc044ff9700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 0x7fc0300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.284+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0400ce190 con 0x7fc048102b10 2023-10-06T21:20:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.284+0000 7fc0477fe700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 0x7fc0300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.284+0000 7fc0477fe700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 0x7fc0300a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc048101fd0 tx=0x7fc038009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:23.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.288+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0400976a0 con 0x7fc048102b10 2023-10-06T21:20:23.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:22 smithi146 ceph-mon[102809]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-06T21:20:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.500+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc04810c800 con 0x7fc048102b10 2023-10-06T21:20:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.501+0000 7fc044ff9700 1 -- 172.21.15.31:0/2083128230 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5332 (secure 0 0 0) 0x7fc04009acc0 con 0x7fc048102b10 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 88 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-10-06T21:10:28.425727+0000 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2023-10-06T21:19:26.829028+0000 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2023-10-06T21:20:23.503 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release reef 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.69 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/811549482,v1:172.21.15.31:6803/811549482] [v2:172.21.15.31:6804/811549482,v1:172.21.15.31:6805/811549482] exists,up 3e79b590-506f-4710-912e-2de1d329bdcb 2023-10-06T21:20:23.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/2309964911,v1:172.21.15.142:6801/2309964911] [v2:172.21.15.142:6802/2309964911,v1:172.21.15.142:6803/2309964911] exists,up 0ed69e82-9719-4e1c-9a4c-3c78add6dd0f 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6800/1261175101,v1:172.21.15.146:6801/1261175101] [v2:172.21.15.146:6802/1261175101,v1:172.21.15.146:6803/1261175101] exists,up 7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/3470883528,v1:172.21.15.31:6811/3470883528] [v2:172.21.15.31:6812/3470883528,v1:172.21.15.31:6813/3470883528] exists,up 53868b14-df36-4751-89c7-54721dec7417 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/3887215066,v1:172.21.15.142:6809/3887215066] [v2:172.21.15.142:6810/3887215066,v1:172.21.15.142:6811/3887215066] exists,up 95502ff0-ad06-45c6-8fe0-de7706d59da0 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6808/4108707603,v1:172.21.15.146:6809/4108707603] [v2:172.21.15.146:6810/4108707603,v1:172.21.15.146:6811/4108707603] exists,up c6d0247c-93bc-4b4c-8855-087be5ba9a66 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/1906076219,v1:172.21.15.31:6819/1906076219] [v2:172.21.15.31:6820/1906076219,v1:172.21.15.31:6821/1906076219] exists,up 2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/722083176,v1:172.21.15.142:6817/722083176] [v2:172.21.15.142:6818/722083176,v1:172.21.15.142:6819/722083176] exists,up 5705f243-b05f-4005-90f0-b62dc8b0ffbb 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6816/1953163251,v1:172.21.15.146:6817/1953163251] [v2:172.21.15.146:6818/1953163251,v1:172.21.15.146:6819/1953163251] exists,up 0cc710f2-d725-47da-9841-e1c75d64bd0e 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 68 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/3073626060,v1:172.21.15.31:6827/3073626060] [v2:172.21.15.31:6828/3073626060,v1:172.21.15.31:6829/3073626060] exists,up 50342566-6d06-4bdd-9336-075df7afc340 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/1169171647,v1:172.21.15.142:6825/1169171647] [v2:172.21.15.142:6826/1169171647,v1:172.21.15.142:6827/1169171647] exists,up d033e7e5-3d15-45be-8b09-0fb8f7187f9e 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6824/740640181,v1:172.21.15.146:6825/740640181] [v2:172.21.15.146:6826/740640181,v1:172.21.15.146:6827/740640181] exists,up 6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4 2023-10-06T21:20:23.505 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.0 [7,11] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.e [2,4] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.10 [8,11] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1a [8,1] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1d [2,4] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.6 [3,6] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1b [4,8] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1d [4,8] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.21 [4,6] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.28 [5,7] 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3876681444 expires 2023-10-07T21:11:25.716471+0000 2023-10-06T21:20:23.506 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/484008040 expires 2023-10-07T21:11:25.716471+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2656198457 expires 2023-10-07T21:11:25.716471+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/461858081 expires 2023-10-07T21:10:58.882715+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3144929895 expires 2023-10-07T21:10:58.882715+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/461858081 expires 2023-10-07T21:10:58.882715+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/484008040 expires 2023-10-07T21:11:25.716471+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1560768070 expires 2023-10-07T21:10:58.882715+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2794001065 expires 2023-10-07T21:10:58.882715+0000 2023-10-06T21:20:23.507 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1457841953 expires 2023-10-07T21:11:25.716471+0000 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 msgr2=0x7fc0300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 0x7fc0300a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc048101fd0 tx=0x7fc038009260 comp rx=0 tx=0).stop 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 msgr2=0x7fc0481a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc040007880 tx=0x7fc0400078b0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 shutdown_connections 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc0300a1510 0x7fc0300a39d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc048102b10 0x7fc0481a03d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc048100f70 0x7fc048198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 --2- 172.21.15.31:0/2083128230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048102170 0x7fc048199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 >> 172.21.15.31:0/2083128230 conn(0x7fc0480fc4d0 msgr2=0x7fc048107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 shutdown_connections 2023-10-06T21:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:23.503+0000 7fc04de03700 1 -- 172.21.15.31:0/2083128230 wait complete. 2023-10-06T21:20:23.955 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-06T21:20:23.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:24.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:23 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2083128230' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:20:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 -- 172.21.15.31:0/707852783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102080 msgr2=0x7f0d48102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 --2- 172.21.15.31:0/707852783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102080 0x7f0d48102500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009990 tx=0x7f0d34009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 -- 172.21.15.31:0/707852783 shutdown_connections 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 --2- 172.21.15.31:0/707852783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48102a40 0x7f0d48109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 --2- 172.21.15.31:0/707852783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48100ea0 0x7f0d481012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 --2- 172.21.15.31:0/707852783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102080 0x7f0d48102500 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 -- 172.21.15.31:0/707852783 >> 172.21.15.31:0/707852783 conn(0x7f0d480fc420 msgr2=0x7f0d480fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 -- 172.21.15.31:0/707852783 shutdown_connections 2023-10-06T21:20:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.174+0000 7f0d4efff700 1 -- 172.21.15.31:0/707852783 wait complete. 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.175+0000 7f0d4efff700 1 Processor -- start 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.175+0000 7f0d4efff700 1 -- start start 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48100ea0 0x7f0d48196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48102080 0x7f0d48197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4819e6a0 con 0x7f0d48102a40 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4819e7e0 con 0x7f0d48100ea0 2023-10-06T21:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4efff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4819e970 con 0x7f0d48102080 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46124/0 (socket says 172.21.15.31:46124) 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 -- 172.21.15.31:0/3027437877 learned_addr learned my addr 172.21.15.31:0/3027437877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d47fff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48102080 0x7f0d48197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4cd9b700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48100ea0 0x7f0d48196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 -- 172.21.15.31:0/3027437877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48102080 msgr2=0x7f0d48197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48102080 0x7f0d48197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 -- 172.21.15.31:0/3027437877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48100ea0 msgr2=0x7f0d48196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48100ea0 0x7f0d48196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.176+0000 7f0d4d59c700 1 -- 172.21.15.31:0/3027437877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340095f0 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d4d59c700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ef30 tx=0x7f0d4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40018070 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d4000f040 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4819ebf0 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000cc70 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.177+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4819f100 con 0x7f0d48102a40 2023-10-06T21:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.178+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d48108500 con 0x7f0d48102a40 2023-10-06T21:20:24.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.180+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0d40007500 con 0x7f0d48102a40 2023-10-06T21:20:24.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.182+0000 7f0d45ffb700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 0x7f0d380a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.183+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d400cca00 con 0x7f0d48102a40 2023-10-06T21:20:24.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.183+0000 7f0d4cd9b700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 0x7f0d380a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.183+0000 7f0d4cd9b700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 0x7f0d380a3a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005950 tx=0x7f0d3c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.187+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0d40095f10 con 0x7f0d48102a40 2023-10-06T21:20:24.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:23 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2083128230' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:20:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:23 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2083128230' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:20:24.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.400+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d4810c840 con 0x7f0d48102a40 2023-10-06T21:20:24.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.402+0000 7f0d45ffb700 1 -- 172.21.15.31:0/3027437877 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22700 (secure 0 0 0) 0x7f0d40099530 con 0x7f0d48102a40 2023-10-06T21:20:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:24.404 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:19:26.829028+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:24.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 msgr2=0x7f0d380a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 0x7f0d380a3a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005950 tx=0x7f0d3c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 msgr2=0x7f0d4819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ef30 tx=0x7f0d4000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 shutdown_connections 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0d380a15d0 0x7f0d380a3a90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d48102a40 0x7f0d4819e0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d48100ea0 0x7f0d48196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 --2- 172.21.15.31:0/3027437877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d48102080 0x7f0d48197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 >> 172.21.15.31:0/3027437877 conn(0x7f0d480fc420 msgr2=0x7f0d48105640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 shutdown_connections 2023-10-06T21:20:24.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.404+0000 7f0d4efff700 1 -- 172.21.15.31:0/3027437877 wait complete. 2023-10-06T21:20:24.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:20:24.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:20:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 -- 172.21.15.31:0/1662094024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78102090 msgr2=0x7f0c78102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 --2- 172.21.15.31:0/1662094024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78102090 0x7f0c78102510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009990 tx=0x7f0c64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 -- 172.21.15.31:0/1662094024 shutdown_connections 2023-10-06T21:20:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 --2- 172.21.15.31:0/1662094024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 0x7f0c78109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 --2- 172.21.15.31:0/1662094024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78100f40 0x7f0c78101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 --2- 172.21.15.31:0/1662094024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78102090 0x7f0c78102510 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 -- 172.21.15.31:0/1662094024 >> 172.21.15.31:0/1662094024 conn(0x7f0c780fc4d0 msgr2=0x7f0c780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 -- 172.21.15.31:0/1662094024 shutdown_connections 2023-10-06T21:20:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.631+0000 7f0c7efff700 1 -- 172.21.15.31:0/1662094024 wait complete. 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 Processor -- start 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 -- start start 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78102090 0x7f0c78197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 0x7f0c7819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819e6a0 con 0x7f0c78100f40 2023-10-06T21:20:24.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819e7e0 con 0x7f0c78102090 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.632+0000 7f0c7efff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819e970 con 0x7f0c78102a50 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46150/0 (socket says 172.21.15.31:46150) 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 -- 172.21.15.31:0/2601248552 learned_addr learned my addr 172.21.15.31:0/2601248552 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c77fff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78102090 0x7f0c78197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7d59c700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 0x7f0c7819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 -- 172.21.15.31:0/2601248552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 msgr2=0x7f0c7819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 0x7f0c7819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 -- 172.21.15.31:0/2601248552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78102090 msgr2=0x7f0c78197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78102090 0x7f0c78197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 -- 172.21.15.31:0/2601248552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c640095f0 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c7cd9b700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00e9a0 tx=0x7f0c6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.633+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c009790 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.634+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c6c00cbd0 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.634+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7819ec50 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.634+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c018680 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.634+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7819f1b0 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.635+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78107f40 con 0x7f0c78100f40 2023-10-06T21:20:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.637+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0c6c00cd40 con 0x7f0c78100f40 2023-10-06T21:20:24.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.639+0000 7f0c75ffb700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 0x7f0c680a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.640+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0c6c014070 con 0x7f0c78100f40 2023-10-06T21:20:24.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.640+0000 7f0c77fff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 0x7f0c680a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.640+0000 7f0c77fff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 0x7f0c680a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006010 tx=0x7f0c6400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.641+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0c6c096970 con 0x7f0c78100f40 2023-10-06T21:20:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.856+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c7810c880 con 0x7f0c78100f40 2023-10-06T21:20:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.856+0000 7f0c75ffb700 1 -- 172.21.15.31:0/2601248552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c6c0260a0 con 0x7f0c78100f40 2023-10-06T21:20:24.858 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:20:24.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 msgr2=0x7f0c680a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 0x7f0c680a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006010 tx=0x7f0c6400b260 comp rx=0 tx=0).stop 2023-10-06T21:20:24.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 msgr2=0x7f0c78196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00e9a0 tx=0x7f0c6c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 shutdown_connections 2023-10-06T21:20:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0c680a1590 0x7f0c680a3a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c78102a50 0x7f0c7819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c78100f40 0x7f0c78196b20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 --2- 172.21.15.31:0/2601248552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c78102090 0x7f0c78197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 >> 172.21.15.31:0/2601248552 conn(0x7f0c780fc4d0 msgr2=0x7f0c780fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 shutdown_connections 2023-10-06T21:20:24.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:24.859+0000 7f0c7efff700 1 -- 172.21.15.31:0/2601248552 wait complete. 2023-10-06T21:20:24.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:20:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 -- 172.21.15.31:0/150929703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac100e90 msgr2=0x7f3cac1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/150929703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac100e90 0x7f3cac1012b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 -- 172.21.15.31:0/150929703 shutdown_connections 2023-10-06T21:20:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/150929703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/150929703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac100e90 0x7f3cac1012b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/150929703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac102070 0x7f3cac1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 -- 172.21.15.31:0/150929703 >> 172.21.15.31:0/150929703 conn(0x7f3cac0fc430 msgr2=0x7f3cac0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 -- 172.21.15.31:0/150929703 shutdown_connections 2023-10-06T21:20:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.084+0000 7f3cb1fff700 1 -- 172.21.15.31:0/150929703 wait complete. 2023-10-06T21:20:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.085+0000 7f3cb1fff700 1 Processor -- start 2023-10-06T21:20:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.085+0000 7f3cb1fff700 1 -- start start 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac100e90 0x7f3cac198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1a08f0 con 0x7f3cac102070 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1a0a30 con 0x7f3cac102a30 2023-10-06T21:20:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cb1fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1a0bc0 con 0x7f3cac100e90 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46172/0 (socket says 172.21.15.31:46172) 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 -- 172.21.15.31:0/233024205 learned_addr learned my addr 172.21.15.31:0/233024205 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cab7fe700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac100e90 0x7f3cac198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cabfff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 -- 172.21.15.31:0/233024205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac100e90 msgr2=0x7f3cac198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac100e90 0x7f3cac198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 -- 172.21.15.31:0/233024205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 msgr2=0x7f3cac1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3caaffd700 1 -- 172.21.15.31:0/233024205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0095f0 con 0x7f3cac102070 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.086+0000 7f3cabfff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3caaffd700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000d980 tx=0x7f3ca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004020 con 0x7f3cac102070 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ca0009730 con 0x7f3cac102070 2023-10-06T21:20:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0003a00 con 0x7f3cac102070 2023-10-06T21:20:25.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1a0ea0 con 0x7f3cac102070 2023-10-06T21:20:25.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.087+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1a12d0 con 0x7f3cac102070 2023-10-06T21:20:25.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.088+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac10c600 con 0x7f3cac102070 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.090+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3ca00098a0 con 0x7f3cac102070 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.093+0000 7f3ca8ff9700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 0x7f3c940a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.093+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ca00ccbb0 con 0x7f3cac102070 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.093+0000 7f3cab7fe700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 0x7f3c940a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.093+0000 7f3cab7fe700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 0x7f3c940a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005910 tx=0x7f3c9c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.097+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3ca0096030 con 0x7f3cac102070 2023-10-06T21:20:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:24 smithi142 ceph-mon[102951]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:24 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3027437877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:25.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:24 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2601248552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:25.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:24 smithi031 ceph-mon[98539]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:25.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:24 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3027437877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:25.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:24 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2601248552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:25.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:24 smithi146 ceph-mon[102809]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:25.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:24 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3027437877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:25.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:24 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2601248552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:25.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.308+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cac1a1520 con 0x7f3cac102070 2023-10-06T21:20:25.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.309+0000 7f3ca8ff9700 1 -- 172.21.15.31:0/233024205 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ca0099650 con 0x7f3cac102070 2023-10-06T21:20:25.311 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:25.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 msgr2=0x7f3c940a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 0x7f3c940a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005910 tx=0x7f3c9c00b260 comp rx=0 tx=0).stop 2023-10-06T21:20:25.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 msgr2=0x7f3cac1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000d980 tx=0x7f3ca000dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 shutdown_connections 2023-10-06T21:20:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3c940a1510 0x7f3c940a39d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cac102a30 0x7f3cac1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cac100e90 0x7f3cac198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 --2- 172.21.15.31:0/233024205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac102070 0x7f3cac1992b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 >> 172.21.15.31:0/233024205 conn(0x7f3cac0fc430 msgr2=0x7f3cac0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 shutdown_connections 2023-10-06T21:20:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.311+0000 7f3cb1fff700 1 -- 172.21.15.31:0/233024205 wait complete. 2023-10-06T21:20:25.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.534+0000 7fa31b5ff700 1 -- 172.21.15.31:0/1980984409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 msgr2=0x7fa314109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.534+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/1980984409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa314109350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00b0a0 tx=0x7fa30c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 -- 172.21.15.31:0/1980984409 shutdown_connections 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/1980984409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa314109350 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/1980984409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314100e90 0x7fa3141012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/1980984409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314102070 0x7fa3141024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 -- 172.21.15.31:0/1980984409 >> 172.21.15.31:0/1980984409 conn(0x7fa3140fc430 msgr2=0x7fa3140fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 -- 172.21.15.31:0/1980984409 shutdown_connections 2023-10-06T21:20:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.535+0000 7fa31b5ff700 1 -- 172.21.15.31:0/1980984409 wait complete. 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 Processor -- start 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 -- start start 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314100e90 0x7fa314074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 0x7fa3140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa314073f80 con 0x7fa314102a30 2023-10-06T21:20:25.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140740f0 con 0x7fa314102070 2023-10-06T21:20:25.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.536+0000 7fa31b5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141a0c20 con 0x7fa314100e90 2023-10-06T21:20:25.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46180/0 (socket says 172.21.15.31:46180) 2023-10-06T21:20:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 -- 172.21.15.31:0/770874732 learned_addr learned my addr 172.21.15.31:0/770874732 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa31939b700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314100e90 0x7fa314074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 -- 172.21.15.31:0/770874732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314100e90 msgr2=0x7fa314074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314100e90 0x7fa314074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 -- 172.21.15.31:0/770874732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 msgr2=0x7fa3140732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa318b9a700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 0x7fa3140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 0x7fa3140732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 -- 172.21.15.31:0/770874732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009d00 con 0x7fa314102a30 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa318b9a700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 0x7fa3140732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa319b9c700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00d010 tx=0x7fa30c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00e040 con 0x7fa314102a30 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa30c003e10 con 0x7fa314102a30 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3141a0ea0 con 0x7fa314102a30 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.537+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c005310 con 0x7fa314102a30 2023-10-06T21:20:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.538+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3141a1260 con 0x7fa314102a30 2023-10-06T21:20:25.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.539+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31410c5b0 con 0x7fa314102a30 2023-10-06T21:20:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.541+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa30c01daa0 con 0x7fa314102a30 2023-10-06T21:20:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.542+0000 7fa30a7fc700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.542+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa30c0ce030 con 0x7fa314102a30 2023-10-06T21:20:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.542+0000 7fa31939b700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.543+0000 7fa31939b700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 0x7fa3000a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa314101be0 tx=0x7fa310009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:25.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.546+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa30c0975a0 con 0x7fa314102a30 2023-10-06T21:20:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.758+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa314107e50 con 0x7fa314102a30 2023-10-06T21:20:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.759+0000 7fa30a7fc700 1 -- 172.21.15.31:0/770874732 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa30c09abc0 con 0x7fa314102a30 2023-10-06T21:20:25.761 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 msgr2=0x7fa3000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 0x7fa3000a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa314101be0 tx=0x7fa310009260 comp rx=0 tx=0).stop 2023-10-06T21:20:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 msgr2=0x7fa3140737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00d010 tx=0x7fa30c00bd70 comp rx=0 tx=0).stop 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 shutdown_connections 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa314102a30 0x7fa3140737e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa314100e90 0x7fa314074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 --2- 172.21.15.31:0/770874732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa314102070 0x7fa3140732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 >> 172.21.15.31:0/770874732 conn(0x7fa3140fc430 msgr2=0x7fa3140fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 shutdown_connections 2023-10-06T21:20:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:25.761+0000 7fa31b5ff700 1 -- 172.21.15.31:0/770874732 wait complete. 2023-10-06T21:20:25.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/3081962381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 msgr2=0x7ffa54103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/3081962381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54103d10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009990 tx=0x7ffa50009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/3081962381 shutdown_connections 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/3081962381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54106b80 0x7ffa54108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/3081962381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54103d10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/3081962381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa54104250 0x7ffa54106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/3081962381 >> 172.21.15.31:0/3081962381 conn(0x7ffa540fb310 msgr2=0x7ffa540fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.016+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/3081962381 shutdown_connections 2023-10-06T21:20:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.017+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/3081962381 wait complete. 2023-10-06T21:20:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 Processor -- start 2023-10-06T21:20:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 -- start start 2023-10-06T21:20:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 0x7ffa54196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa54106b80 0x7ffa5419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5419e590 con 0x7ffa54101920 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5419e6d0 con 0x7ffa54104250 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5419e860 con 0x7ffa54106b80 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46206/0 (socket says 172.21.15.31:46206) 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.018+0000 7ffa5959b700 1 -- 172.21.15.31:0/762023458 learned_addr learned my addr 172.21.15.31:0/762023458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa58d9a700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 0x7ffa54196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 -- 172.21.15.31:0/762023458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa54106b80 msgr2=0x7ffa5419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa54106b80 0x7ffa5419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 -- 172.21.15.31:0/762023458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 msgr2=0x7ffa54196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 0x7ffa54196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 -- 172.21.15.31:0/762023458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500095f0 con 0x7ffa54101920 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa58d9a700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 0x7ffa54196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5959b700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003f60 tx=0x7ffa50005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001d070 con 0x7ffa54101920 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5419eae0 con 0x7ffa54101920 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa5000f460 con 0x7ffa54101920 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50021670 con 0x7ffa54101920 2023-10-06T21:20:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.019+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5419ef90 con 0x7ffa54101920 2023-10-06T21:20:26.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.020+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa540fce90 con 0x7ffa54101920 2023-10-06T21:20:26.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.022+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ffa5000f5d0 con 0x7ffa54101920 2023-10-06T21:20:26.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.024+0000 7ffa4a7fc700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 0x7ffa400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.024+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8335+0+0 (secure 0 0 0) 0x7ffa500ce390 con 0x7ffa54101920 2023-10-06T21:20:26.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.024+0000 7ffa58d9a700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 0x7ffa400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.025+0000 7ffa58d9a700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 0x7ffa400a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009be0 tx=0x7ffa44009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.027+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa50097860 con 0x7ffa54101920 2023-10-06T21:20:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:25 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/233024205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:25 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/770874732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-06T21:20:26.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2023-10-06T21:20:26.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:25 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-10-06T21:20:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:25 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/233024205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:25 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/770874732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-06T21:20:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2023-10-06T21:20:26.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:25 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-10-06T21:20:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.284+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa5402d020 con 0x7ffa54101920 2023-10-06T21:20:26.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.284+0000 7ffa4a7fc700 1 -- 172.21.15.31:0/762023458 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7ffa5009ae80 con 0x7ffa54101920 2023-10-06T21:20:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:26.287 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 msgr2=0x7ffa400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 0x7ffa400a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009be0 tx=0x7ffa44009260 comp rx=0 tx=0).stop 2023-10-06T21:20:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 msgr2=0x7ffa54196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003f60 tx=0x7ffa50005ef0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 shutdown_connections 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffa400a1550 0x7ffa400a3a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa54106b80 0x7ffa5419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa54101920 0x7ffa54196a10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 --2- 172.21.15.31:0/762023458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa54104250 0x7ffa54196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 >> 172.21.15.31:0/762023458 conn(0x7ffa540fb310 msgr2=0x7ffa54102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 shutdown_connections 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.287+0000 7ffa5b7ff700 1 -- 172.21.15.31:0/762023458 wait complete. 2023-10-06T21:20:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:25 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/233024205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:25 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/770874732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-06T21:20:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2023-10-06T21:20:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:25 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2023-10-06T21:20:26.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 -- 172.21.15.31:0/1665930059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 msgr2=0x7ffaf4109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 --2- 172.21.15.31:0/1665930059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf4109070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b0a0 tx=0x7ffaec00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 -- 172.21.15.31:0/1665930059 shutdown_connections 2023-10-06T21:20:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 --2- 172.21.15.31:0/1665930059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf4109070 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 --2- 172.21.15.31:0/1665930059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41019a0 0x7ffaf4103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 --2- 172.21.15.31:0/1665930059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41042d0 0x7ffaf41066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 -- 172.21.15.31:0/1665930059 >> 172.21.15.31:0/1665930059 conn(0x7ffaf40fb3b0 msgr2=0x7ffaf40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 -- 172.21.15.31:0/1665930059 shutdown_connections 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.509+0000 7ffafbfff700 1 -- 172.21.15.31:0/1665930059 wait complete. 2023-10-06T21:20:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 Processor -- start 2023-10-06T21:20:26.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 -- start start 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 0x7ffaf4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 0x7ffaf4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41a0880 con 0x7ffaf4106c00 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41a09f0 con 0x7ffaf41042d0 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.511+0000 7ffafbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41a0bb0 con 0x7ffaf41019a0 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46222/0 (socket says 172.21.15.31:46222) 2023-10-06T21:20:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 -- 172.21.15.31:0/3641806843 learned_addr learned my addr 172.21.15.31:0/3641806843 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffaf959a700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 0x7ffaf4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffaf9d9b700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 0x7ffaf4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 -- 172.21.15.31:0/3641806843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 msgr2=0x7ffaf4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 0x7ffaf4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 -- 172.21.15.31:0/3641806843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 msgr2=0x7ffaf4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 0x7ffaf4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 -- 172.21.15.31:0/3641806843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec009d00 con 0x7ffaf4106c00 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffaf9d9b700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 0x7ffaf4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffaf959a700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 0x7ffaf4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffafa59c700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d010 tx=0x7ffaec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.512+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00e040 con 0x7ffaf4106c00 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.513+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffaec003e10 con 0x7ffaf4106c00 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.513+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf41a0e30 con 0x7ffaf4106c00 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.513+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0053c0 con 0x7ffaf4106c00 2023-10-06T21:20:26.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.513+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf41a11a0 con 0x7ffaf4106c00 2023-10-06T21:20:26.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.514+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf4108780 con 0x7ffaf4106c00 2023-10-06T21:20:26.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.516+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ffaec01db10 con 0x7ffaf4106c00 2023-10-06T21:20:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.517+0000 7ffaeaffd700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 0x7ffae00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.517+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8335+0+0 (secure 0 0 0) 0x7ffaec0ce1b0 con 0x7ffaf4106c00 2023-10-06T21:20:26.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.517+0000 7ffaf9d9b700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 0x7ffae00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.518+0000 7ffaf9d9b700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 0x7ffae00a3a10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffaf419a210 tx=0x7ffaf0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.521+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffaec097680 con 0x7ffaf4106c00 2023-10-06T21:20:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.734+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaf410c7d0 con 0x7ffaf4106c00 2023-10-06T21:20:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.735+0000 7ffaeaffd700 1 -- 172.21.15.31:0/3641806843 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22812 (secure 0 0 0) 0x7ffaec09aca0 con 0x7ffaf4106c00 2023-10-06T21:20:26.738 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:26.738 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:25.880763+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 msgr2=0x7ffae00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 0x7ffae00a3a10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffaf419a210 tx=0x7ffaf0009260 comp rx=0 tx=0).stop 2023-10-06T21:20:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 msgr2=0x7ffaf41a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d010 tx=0x7ffaec00bd70 comp rx=0 tx=0).stop 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 shutdown_connections 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7ffae00a1550 0x7ffae00a3a10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaf4106c00 0x7ffaf41a02b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf41019a0 0x7ffaf4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 --2- 172.21.15.31:0/3641806843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaf41042d0 0x7ffaf4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 >> 172.21.15.31:0/3641806843 conn(0x7ffaf40fb3b0 msgr2=0x7ffaf4100000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 shutdown_connections 2023-10-06T21:20:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.738+0000 7ffafbfff700 1 -- 172.21.15.31:0/3641806843 wait complete. 2023-10-06T21:20:26.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:20:26.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:20:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 -- 172.21.15.31:0/894521715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 msgr2=0x7f8030073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 --2- 172.21.15.31:0/894521715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f8030073340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f802c009990 tx=0x7f802c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 -- 172.21.15.31:0/894521715 shutdown_connections 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 --2- 172.21.15.31:0/894521715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f803010f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 --2- 172.21.15.31:0/894521715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f8030073340 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 --2- 172.21.15.31:0/894521715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 0x7f8030073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 -- 172.21.15.31:0/894521715 >> 172.21.15.31:0/894521715 conn(0x7f80300fc430 msgr2=0x7f80300fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 -- 172.21.15.31:0/894521715 shutdown_connections 2023-10-06T21:20:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.973+0000 7f8037bff700 1 -- 172.21.15.31:0/894521715 wait complete. 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.974+0000 7f8037bff700 1 Processor -- start 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.974+0000 7f8037bff700 1 -- start start 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 0x7f803019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f80301a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301a4cb0 con 0x7f8030074ef0 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301a4df0 con 0x7f803010d4b0 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f8037bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301a4f80 con 0x7f8030073910 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46242/0 (socket says 172.21.15.31:46242) 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 -- 172.21.15.31:0/1873852982 learned_addr learned my addr 172.21.15.31:0/1873852982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803599b700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 0x7f803019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803619c700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f80301a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 -- 172.21.15.31:0/1873852982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 msgr2=0x7f803019d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 0x7f803019d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 -- 172.21.15.31:0/1873852982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 msgr2=0x7f80301a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f80301a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 -- 172.21.15.31:0/1873852982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0095f0 con 0x7f8030074ef0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803619c700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f80301a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.975+0000 7f803519a700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f802000d980 tx=0x7f802000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.976+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020004020 con 0x7f8030074ef0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.976+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8020009730 con 0x7f8030074ef0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.976+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020003a00 con 0x7f8030074ef0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.976+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301a5260 con 0x7f8030074ef0 2023-10-06T21:20:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.976+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301a5720 con 0x7f8030074ef0 2023-10-06T21:20:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.977+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8030103000 con 0x7f8030074ef0 2023-10-06T21:20:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.979+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80200098a0 con 0x7f8030074ef0 2023-10-06T21:20:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.980+0000 7f8026ffd700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 0x7f801c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.980+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f80200cca30 con 0x7f8030074ef0 2023-10-06T21:20:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.980+0000 7f803599b700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 0x7f801c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.981+0000 7f803599b700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 0x7f801c0a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f802c00b5c0 tx=0x7f802c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:26.984+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8020095fd0 con 0x7f8030074ef0 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/762023458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:26 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3641806843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/762023458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:26 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3641806843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.199+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f803002d1e0 con 0x7f8030074ef0 2023-10-06T21:20:27.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.200+0000 7f8026ffd700 1 -- 172.21.15.31:0/1873852982 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80200995f0 con 0x7f8030074ef0 2023-10-06T21:20:27.202 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:20:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 msgr2=0x7f801c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 0x7f801c0a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f802c00b5c0 tx=0x7f802c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 msgr2=0x7f803019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f802000d980 tx=0x7f802000dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 shutdown_connections 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f801c0a1500 0x7f801c0a39c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f803010d4b0 0x7f80301a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8030073910 0x7f803019d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 --2- 172.21.15.31:0/1873852982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8030074ef0 0x7f803019d670 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 >> 172.21.15.31:0/1873852982 conn(0x7f80300fc430 msgr2=0x7f80300fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 shutdown_connections 2023-10-06T21:20:27.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.202+0000 7f8037bff700 1 -- 172.21.15.31:0/1873852982 wait complete. 2023-10-06T21:20:27.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: from='mgr.14148 172.21.15.31:0/3723166563' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/762023458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:26 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3641806843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 -- 172.21.15.31:0/3094932415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102a40 msgr2=0x7fed50109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 --2- 172.21.15.31:0/3094932415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102a40 0x7fed50109360 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b0a0 tx=0x7fed4800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 -- 172.21.15.31:0/3094932415 shutdown_connections 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 --2- 172.21.15.31:0/3094932415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102a40 0x7fed50109360 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 --2- 172.21.15.31:0/3094932415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50100ea0 0x7fed501012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 --2- 172.21.15.31:0/3094932415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50102080 0x7fed50102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 -- 172.21.15.31:0/3094932415 >> 172.21.15.31:0/3094932415 conn(0x7fed500fc460 msgr2=0x7fed500fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 -- 172.21.15.31:0/3094932415 shutdown_connections 2023-10-06T21:20:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.427+0000 7fed583ff700 1 -- 172.21.15.31:0/3094932415 wait complete. 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.428+0000 7fed583ff700 1 Processor -- start 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.428+0000 7fed583ff700 1 -- start start 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50100ea0 0x7fed50198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50102a40 0x7fed501a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501a09e0 con 0x7fed50102080 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501a0b20 con 0x7fed50100ea0 2023-10-06T21:20:27.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed583ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501a0cb0 con 0x7fed50102a40 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46268/0 (socket says 172.21.15.31:46268) 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 -- 172.21.15.31:0/1161065860 learned_addr learned my addr 172.21.15.31:0/1161065860 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5699c700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50102a40 0x7fed501a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5619b700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50100ea0 0x7fed50198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 -- 172.21.15.31:0/1161065860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50102a40 msgr2=0x7fed501a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50102a40 0x7fed501a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 -- 172.21.15.31:0/1161065860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50100ea0 msgr2=0x7fed50198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50100ea0 0x7fed50198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.429+0000 7fed5599a700 1 -- 172.21.15.31:0/1161065860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48009d00 con 0x7fed50102080 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed5599a700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fed4000e970 tx=0x7fed4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:27.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40009790 con 0x7fed50102080 2023-10-06T21:20:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed400044b0 con 0x7fed50102080 2023-10-06T21:20:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501a0f90 con 0x7fed50102080 2023-10-06T21:20:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40003dd0 con 0x7fed50102080 2023-10-06T21:20:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.430+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed501a1460 con 0x7fed50102080 2023-10-06T21:20:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.431+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50193360 con 0x7fed50102080 2023-10-06T21:20:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.433+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fed40003f30 con 0x7fed50102080 2023-10-06T21:20:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.434+0000 7fed477fe700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 0x7fed3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.434+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fed40014070 con 0x7fed50102080 2023-10-06T21:20:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.434+0000 7fed5619b700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 0x7fed3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.435+0000 7fed5619b700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 0x7fed3c0a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005950 tx=0x7fed4c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:27.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.438+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fed40096530 con 0x7fed50102080 2023-10-06T21:20:27.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.651+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed5002d020 con 0x7fed50102080 2023-10-06T21:20:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.652+0000 7fed477fe700 1 -- 172.21.15.31:0/1161065860 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed40099b50 con 0x7fed50102080 2023-10-06T21:20:27.654 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 msgr2=0x7fed3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 0x7fed3c0a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005950 tx=0x7fed4c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 msgr2=0x7fed501993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fed4000e970 tx=0x7fed4000ed30 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 shutdown_connections 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fed3c0a1500 0x7fed3c0a39c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed50102a40 0x7fed501a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50100ea0 0x7fed50198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.654+0000 7fed583ff700 1 --2- 172.21.15.31:0/1161065860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed50102080 0x7fed501993a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.655+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 >> 172.21.15.31:0/1161065860 conn(0x7fed500fc460 msgr2=0x7fed50107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.655+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 shutdown_connections 2023-10-06T21:20:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.655+0000 7fed583ff700 1 -- 172.21.15.31:0/1161065860 wait complete. 2023-10-06T21:20:27.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 -- 172.21.15.31:0/4189060959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec100ea0 msgr2=0x7f9fec1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/4189060959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec100ea0 0x7f9fec1012c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009990 tx=0x7f9fe8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 -- 172.21.15.31:0/4189060959 shutdown_connections 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/4189060959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec102a40 0x7f9fec109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/4189060959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec100ea0 0x7f9fec1012c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/4189060959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 0x7f9fec102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 -- 172.21.15.31:0/4189060959 >> 172.21.15.31:0/4189060959 conn(0x7f9fec0fc460 msgr2=0x7f9fec0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 -- 172.21.15.31:0/4189060959 shutdown_connections 2023-10-06T21:20:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.878+0000 7f9ff39ff700 1 -- 172.21.15.31:0/4189060959 wait complete. 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.879+0000 7f9ff39ff700 1 Processor -- start 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.879+0000 7f9ff39ff700 1 -- start start 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec100ea0 0x7f9fec198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 0x7f9fec1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1a08f0 con 0x7f9fec102a40 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1a0a30 con 0x7f9fec102080 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff39ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1a0bc0 con 0x7f9fec100ea0 2023-10-06T21:20:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46294/0 (socket says 172.21.15.31:46294) 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 -- 172.21.15.31:0/3430408824 learned_addr learned my addr 172.21.15.31:0/3430408824 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff0f9a700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 0x7f9fec1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff179b700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec100ea0 0x7f9fec198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 -- 172.21.15.31:0/3430408824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec100ea0 msgr2=0x7f9fec198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec100ea0 0x7f9fec198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 -- 172.21.15.31:0/3430408824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 msgr2=0x7f9fec1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 0x7f9fec1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.880+0000 7f9ff1f9c700 1 -- 172.21.15.31:0/3430408824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80095f0 con 0x7f9fec102a40 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9ff1f9c700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400ef30 tx=0x7f9fe400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4018070 con 0x7f9fec102a40 2023-10-06T21:20:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec1a0e40 con 0x7f9fec102a40 2023-10-06T21:20:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fe400f040 con 0x7f9fec102a40 2023-10-06T21:20:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400cbc0 con 0x7f9fec102a40 2023-10-06T21:20:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.881+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1a1270 con 0x7f9fec102a40 2023-10-06T21:20:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.882+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec193400 con 0x7f9fec102a40 2023-10-06T21:20:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.884+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9fe400cd80 con 0x7f9fec102a40 2023-10-06T21:20:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.885+0000 7f9fe27fc700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 0x7f9fd80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.885+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f9fe40cc880 con 0x7f9fec102a40 2023-10-06T21:20:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.886+0000 7f9ff179b700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 0x7f9fd80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.886+0000 7f9ff179b700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 0x7f9fd80a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80098e0 tx=0x7f9fe800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:27.889+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fe4095eb0 con 0x7f9fec102a40 2023-10-06T21:20:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:27 smithi031 ceph-mon[98539]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:20:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:27 smithi031 ceph-mon[98539]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-06T21:20:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:27 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1873852982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:27 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1161065860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.103+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fec02d270 con 0x7f9fec102a40 2023-10-06T21:20:28.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.104+0000 7f9fe27fc700 1 -- 172.21.15.31:0/3430408824 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fe40994d0 con 0x7f9fec102a40 2023-10-06T21:20:28.106 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 msgr2=0x7f9fd80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 0x7f9fd80a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80098e0 tx=0x7f9fe800b260 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 msgr2=0x7f9fec1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400ef30 tx=0x7f9fe400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 shutdown_connections 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9fd80a1480 0x7f9fd80a3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fec102a40 0x7f9fec1a0320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec100ea0 0x7f9fec198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 --2- 172.21.15.31:0/3430408824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fec102080 0x7f9fec1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 >> 172.21.15.31:0/3430408824 conn(0x7f9fec0fc460 msgr2=0x7f9fec107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 shutdown_connections 2023-10-06T21:20:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.106+0000 7f9ff39ff700 1 -- 172.21.15.31:0/3430408824 wait complete. 2023-10-06T21:20:28.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:27 smithi146 ceph-mon[102809]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:20:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:27 smithi146 ceph-mon[102809]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-06T21:20:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:27 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1873852982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:27 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1161065860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 -- 172.21.15.31:0/2162340800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c100ea0 msgr2=0x7f914c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 --2- 172.21.15.31:0/2162340800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c100ea0 0x7f914c1012c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f913c009990 tx=0x7f913c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 -- 172.21.15.31:0/2162340800 shutdown_connections 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 --2- 172.21.15.31:0/2162340800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 0x7f914c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 --2- 172.21.15.31:0/2162340800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c100ea0 0x7f914c1012c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 --2- 172.21.15.31:0/2162340800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c102080 0x7f914c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 -- 172.21.15.31:0/2162340800 >> 172.21.15.31:0/2162340800 conn(0x7f914c0fc420 msgr2=0x7f914c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 -- 172.21.15.31:0/2162340800 shutdown_connections 2023-10-06T21:20:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.328+0000 7f91527ff700 1 -- 172.21.15.31:0/2162340800 wait complete. 2023-10-06T21:20:28.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.329+0000 7f91527ff700 1 Processor -- start 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.329+0000 7f91527ff700 1 -- start start 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 0x7f914c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 0x7f914c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a08f0 con 0x7f914c102080 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a0a60 con 0x7f914c102a40 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f91527ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a0c20 con 0x7f914c100ea0 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46320/0 (socket says 172.21.15.31:46320) 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 -- 172.21.15.31:0/2842007108 learned_addr learned my addr 172.21.15.31:0/2842007108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914bfff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 0x7f914c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f9150d9c700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 0x7f914c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 -- 172.21.15.31:0/2842007108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 msgr2=0x7f914c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 0x7f914c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 -- 172.21.15.31:0/2842007108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 msgr2=0x7f914c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 0x7f914c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914b7fe700 1 -- 172.21.15.31:0/2842007108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0095f0 con 0x7f914c102080 2023-10-06T21:20:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.330+0000 7f914bfff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 0x7f914c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f914b7fe700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f914000e970 tx=0x7f914000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140009790 con 0x7f914c102080 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f914000eec0 con 0x7f914c102080 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140003c10 con 0x7f914c102080 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c1a0f00 con 0x7f914c102080 2023-10-06T21:20:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.331+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c1a1320 con 0x7f914c102080 2023-10-06T21:20:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.332+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c193400 con 0x7f914c102080 2023-10-06T21:20:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.334+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9140003db0 con 0x7f914c102080 2023-10-06T21:20:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.335+0000 7f91497fa700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 0x7f91340a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.336+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f9140014070 con 0x7f914c102080 2023-10-06T21:20:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.336+0000 7f914bfff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 0x7f91340a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.336+0000 7f914bfff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 0x7f91340a3a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f913c005910 tx=0x7f913c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:28.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.339+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9140096560 con 0x7f914c102080 2023-10-06T21:20:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:27 smithi142 ceph-mon[102951]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:20:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:27 smithi142 ceph-mon[102951]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-06T21:20:28.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:27 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1873852982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:27 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1161065860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.595+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f914c10c660 con 0x7f914c102080 2023-10-06T21:20:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.596+0000 7f91497fa700 1 -- 172.21.15.31:0/2842007108 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f9140099b80 con 0x7f914c102080 2023-10-06T21:20:28.598 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:28.598 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 msgr2=0x7f91340a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 0x7f91340a3a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f913c005910 tx=0x7f913c00b260 comp rx=0 tx=0).stop 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 msgr2=0x7f914c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f914000e970 tx=0x7f914000ec80 comp rx=0 tx=0).stop 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 shutdown_connections 2023-10-06T21:20:28.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f91340a1550 0x7f91340a3a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a40 0x7f914c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f914c100ea0 0x7f914c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 --2- 172.21.15.31:0/2842007108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f914c102080 0x7f914c1992f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 >> 172.21.15.31:0/2842007108 conn(0x7f914c0fc420 msgr2=0x7f914c107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 shutdown_connections 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.599+0000 7f91527ff700 1 -- 172.21.15.31:0/2842007108 wait complete. 2023-10-06T21:20:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:28.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 -- 172.21.15.31:0/2813079269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c102080 msgr2=0x7fc51c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 --2- 172.21.15.31:0/2813079269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c102080 0x7fc51c102500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009990 tx=0x7fc50c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 -- 172.21.15.31:0/2813079269 shutdown_connections 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 --2- 172.21.15.31:0/2813079269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102a40 0x7fc51c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 --2- 172.21.15.31:0/2813079269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc51c100ea0 0x7fc51c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 --2- 172.21.15.31:0/2813079269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c102080 0x7fc51c102500 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.821+0000 7fc523bff700 1 -- 172.21.15.31:0/2813079269 >> 172.21.15.31:0/2813079269 conn(0x7fc51c0fc420 msgr2=0x7fc51c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.822+0000 7fc523bff700 1 -- 172.21.15.31:0/2813079269 shutdown_connections 2023-10-06T21:20:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.822+0000 7fc523bff700 1 -- 172.21.15.31:0/2813079269 wait complete. 2023-10-06T21:20:28.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.822+0000 7fc523bff700 1 Processor -- start 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 -- start start 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 0x7fc51c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc51c102a40 0x7fc51c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e6f0 con 0x7fc51c100ea0 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e830 con 0x7fc51c102080 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc523bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19e9c0 con 0x7fc51c102a40 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc52199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc52199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46336/0 (socket says 172.21.15.31:46336) 2023-10-06T21:20:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc52199b700 1 -- 172.21.15.31:0/4035726535 learned_addr learned my addr 172.21.15.31:0/4035726535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.823+0000 7fc52119a700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 0x7fc51c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 -- 172.21.15.31:0/4035726535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc51c102a40 msgr2=0x7fc51c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc51c102a40 0x7fc51c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 -- 172.21.15.31:0/4035726535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 msgr2=0x7fc51c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 0x7fc51c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 -- 172.21.15.31:0/4035726535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0095f0 con 0x7fc51c100ea0 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52119a700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 0x7fc51c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc52199b700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc51800e970 tx=0x7fc51800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518009790 con 0x7fc51c100ea0 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5180044b0 con 0x7fc51c100ea0 2023-10-06T21:20:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518003dd0 con 0x7fc51c100ea0 2023-10-06T21:20:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c19eca0 con 0x7fc51c100ea0 2023-10-06T21:20:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.824+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c19f160 con 0x7fc51c100ea0 2023-10-06T21:20:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.825+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c10c610 con 0x7fc51c100ea0 2023-10-06T21:20:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.827+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc518003f30 con 0x7fc51c100ea0 2023-10-06T21:20:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.829+0000 7fc512ffd700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 0x7fc5080a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.829+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fc518014070 con 0x7fc51c100ea0 2023-10-06T21:20:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.829+0000 7fc52119a700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 0x7fc5080a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:28.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.830+0000 7fc52119a700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 0x7fc5080a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005910 tx=0x7fc50c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:28.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:28.833+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5180965d0 con 0x7fc51c100ea0 2023-10-06T21:20:29.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.046+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc51c107ef0 con 0x7fc51c100ea0 2023-10-06T21:20:29.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.048+0000 7fc512ffd700 1 -- 172.21.15.31:0/4035726535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22812 (secure 0 0 0) 0x7fc518099bf0 con 0x7fc51c100ea0 2023-10-06T21:20:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:26.886529+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:29.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 msgr2=0x7fc5080a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 0x7fc5080a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005910 tx=0x7fc50c00b260 comp rx=0 tx=0).stop 2023-10-06T21:20:29.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 msgr2=0x7fc51c196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc51800e970 tx=0x7fc51800ed30 comp rx=0 tx=0).stop 2023-10-06T21:20:29.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 shutdown_connections 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc5080a1550 0x7fc5080a3a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc51c102a40 0x7fc51c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c100ea0 0x7fc51c196b70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 --2- 172.21.15.31:0/4035726535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c102080 0x7fc51c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 >> 172.21.15.31:0/4035726535 conn(0x7fc51c0fc420 msgr2=0x7fc51c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 shutdown_connections 2023-10-06T21:20:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.050+0000 7fc523bff700 1 -- 172.21.15.31:0/4035726535 wait complete. 2023-10-06T21:20:29.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T21:18:54.346752+0000', 'modified': '2023-10-06T21:19:01.835058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24602, 'mds_1': 24565, 'mds_2': 24526, 'mds_3': 24518, 'mds_4': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14694': {'gid': 14694, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/1272583602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 1272583602}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 1272583602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14712': {'gid': 14712, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/847340369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 847340369}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 847340369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/1755294522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 1755294522}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 1755294522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.142:6833/1989266338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 1989266338}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 1989266338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.142:6835/4077995395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 4077995395}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 4077995395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.146:6835/3172848071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 3172848071}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 3172848071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6837/824494679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 824494679}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 824494679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/1999670279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1999670279}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1999670279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.146:6839/1025732473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 1025732473}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 1025732473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2643221655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2643221655}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2643221655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T21:20:29.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-06T21:20:29.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-10-06T21:20:29.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:28 smithi031 ceph-mon[98539]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-06T21:20:29.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:28 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3430408824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:29.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:28 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2842007108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 -- 172.21.15.31:0/3286375828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 msgr2=0x7f57dc1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 --2- 172.21.15.31:0/3286375828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc1024f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009990 tx=0x7f57cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 -- 172.21.15.31:0/3286375828 shutdown_connections 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 --2- 172.21.15.31:0/3286375828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 0x7f57dc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 --2- 172.21.15.31:0/3286375828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 --2- 172.21.15.31:0/3286375828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc1024f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.275+0000 7f57e35ff700 1 -- 172.21.15.31:0/3286375828 >> 172.21.15.31:0/3286375828 conn(0x7f57dc0fc430 msgr2=0x7f57dc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.276+0000 7f57e35ff700 1 -- 172.21.15.31:0/3286375828 shutdown_connections 2023-10-06T21:20:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.276+0000 7f57e35ff700 1 -- 172.21.15.31:0/3286375828 wait complete. 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.276+0000 7f57e35ff700 1 Processor -- start 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 -- start start 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 0x7f57dc19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc19e670 con 0x7f57dc102070 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc19e7b0 con 0x7f57dc102a30 2023-10-06T21:20:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e35ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc19e940 con 0x7f57dc100e90 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46352/0 (socket says 172.21.15.31:46352) 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e0b9a700 1 -- 172.21.15.31:0/1699592843 learned_addr learned my addr 172.21.15.31:0/1699592843 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e139b700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.277+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 0x7f57dc19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 -- 172.21.15.31:0/1699592843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 msgr2=0x7f57dc196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 -- 172.21.15.31:0/1699592843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 msgr2=0x7f57dc19e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 0x7f57dc19e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 -- 172.21.15.31:0/1699592843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0095f0 con 0x7f57dc102070 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e139b700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0098e0 tx=0x7f57cc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57d27fc700 1 -- 172.21.15.31:0/1699592843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc01d070 con 0x7f57dc102070 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc19ebc0 con 0x7f57dc102070 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57d27fc700 1 -- 172.21.15.31:0/1699592843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57cc00f460 con 0x7f57dc102070 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57d27fc700 1 -- 172.21.15.31:0/1699592843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc021670 con 0x7f57dc102070 2023-10-06T21:20:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.278+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc19f040 con 0x7f57dc102070 2023-10-06T21:20:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.282+0000 7f57d27fc700 1 -- 172.21.15.31:0/1699592843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f57cc00f5d0 con 0x7f57dc102070 2023-10-06T21:20:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.283+0000 7f57f6e51b80 1 Processor -- start 2023-10-06T21:20:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 -- start start 2023-10-06T21:20:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82cee3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3f80 0x55f82cee4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57d27fc700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 0x7f57c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ce47410 con 0x55f82cee4e20 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ce38bd0 con 0x55f82cee3f80 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.284+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ce364f0 con 0x55f82cee3620 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57d27fc700 1 -- 172.21.15.31:0/1699592843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f57cc0cdf60 con 0x7f57dc102070 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e139b700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 0x7f57c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46358/0 (socket says 172.21.15.31:46358) 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 -- 172.21.15.31:0/2129138052 learned_addr learned my addr 172.21.15.31:0/2129138052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3f80 0x55f82cee4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82cee3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 -- 172.21.15.31:0/2129138052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 msgr2=0x55f82cee3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82cee3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 -- 172.21.15.31:0/2129138052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3f80 msgr2=0x55f82cee4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3f80 0x55f82cee4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 -- 172.21.15.31:0/2129138052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82cc5cd90 con 0x55f82cee4e20 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e139b700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 0x7f57c80a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57dc101be0 tx=0x7f57d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.285+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57d4013c30 tx=0x7f57d4007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1790b4ebdc97512 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57c77fe700 1 -- 172.21.15.31:0/2129138052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4004bb0 con 0x55f82cee4e20 2023-10-06T21:20:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57c77fe700 1 -- 172.21.15.31:0/2129138052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d400d050 con 0x55f82cee4e20 2023-10-06T21:20:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57c77fe700 1 -- 172.21.15.31:0/2129138052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4015e70 con 0x55f82cee4e20 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57f6e51b80 1 -- 172.21.15.31:0/2129138052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 msgr2=0x55f82cee52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57d4013c30 tx=0x7f57d4007ac0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57f6e51b80 1 -- 172.21.15.31:0/2129138052 shutdown_connections 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.286+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cee52a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82cee3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/2129138052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3f80 0x55f82cee4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 -- 172.21.15.31:0/2129138052 >> 172.21.15.31:0/2129138052 conn(0x55f82ce53210 msgr2=0x55f82ceda0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 -- 172.21.15.31:0/2129138052 shutdown_connections 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 -- 172.21.15.31:0/2129138052 wait complete. 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 -- start start 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.287+0000 7f57f6e51b80 1 Processor -- start 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82ceed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82cef4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ce31de0 con 0x55f82cee3f80 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82cdce6c0 con 0x55f82cee4e20 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82cdce360 con 0x55f82cee3620 2023-10-06T21:20:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57f6e51b80 10 client.0 _ll_register_callbacks cb 0x55f82cc2b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46366/0 (socket says 172.21.15.31:46366) 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 -- 172.21.15.31:0/3247745704 learned_addr learned my addr 172.21.15.31:0/3247745704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e139b700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82ceed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 -- 172.21.15.31:0/3247745704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 msgr2=0x55f82ceed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82ceed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 -- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 msgr2=0x55f82cef4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82cef4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82cef4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e0b9a700 1 -- 172.21.15.31:0/3247745704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d400f040 con 0x55f82cee3f80 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e139b700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82ceed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.288+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82cef4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57cc097750 tx=0x7f57cc00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac34e4d1586416e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc064c60 con 0x55f82cee3f80 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f82ced1ec0 con 0x55f82cee3f80 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f57cc09b950 con 0x55f82cee3f80 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc09b250 con 0x55f82cee3f80 2023-10-06T21:20:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57f6e51b80 10 client.15318 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f82cc45c10 con 0x55f82cee3f80 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57cc0b7020 con 0x55f82cee3f80 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57cc09a740 con 0x55f82cee3f80 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57f6e51b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.289+0000 7f57f6e51b80 10 client.15318 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57f6e51b80 20 client.15318 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cef8ba0 0x55f82cef7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57f6e51b80 4 client.15318 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f82ce2adc0 con 0x55f82cef8ba0 2023-10-06T21:20:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57e139b700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cef8ba0 0x55f82cef7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57e139b700 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cef8ba0 0x55f82cef7850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a474dd27d6f5d9cc server_cookie=a52c47b967831d17 in_seq=0 out_seq=0 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.290+0000 7f57c4ff9700 10 client.15318 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57c4ff9700 1 -- 172.21.15.31:0/3247745704 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f82ce2adc0 con 0x55f82cef8ba0 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57c4ff9700 10 client.15318 handle_command_reply: tid=0 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 client.15318 shutdown 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 msgr2=0x55f82ceed8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57cc097750 tx=0x7f57cc00bd80 comp rx=0 tx=0).stop 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 shutdown_connections 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cef8ba0 0x55f82cef7850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.291+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82cef4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3620 0x55f82ceed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57c4ff9700 0 client.15318 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/3247745704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3f80 0x55f82ceed8e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 >> 172.21.15.31:0/3247745704 conn(0x55f82ce53210 msgr2=0x55f82ced9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 shutdown_connections 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 1 -- 172.21.15.31:0/3247745704 wait complete. 2023-10-06T21:20:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.292+0000 7f57f6e51b80 20 client.15318 trim_cache size 0 max 16384 2023-10-06T21:20:29.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:28 smithi146 ceph-mon[102809]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-06T21:20:29.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:28 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3430408824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:29.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:28 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2842007108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 Processor -- start 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 -- start start 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 0x55f82cee7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cee8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ccb28d0 con 0x55f82cee4e20 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ced42b0 con 0x55f82cee3620 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ce2adc0 con 0x55f82cee3f80 2023-10-06T21:20:29.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.310+0000 7f57e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46376/0 (socket says 172.21.15.31:46376) 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 -- 172.21.15.31:0/1694832235 learned_addr learned my addr 172.21.15.31:0/1694832235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cee8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e139b700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 0x55f82cee7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 -- 172.21.15.31:0/1694832235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 msgr2=0x55f82cee8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cee8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 -- 172.21.15.31:0/1694832235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 msgr2=0x55f82cee7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 0x55f82cee7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 -- 172.21.15.31:0/1694832235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82cc83200 con 0x55f82cee4e20 2023-10-06T21:20:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e139b700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 0x55f82cee7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cee8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57d401b040 tx=0x7f57d4013440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53390b5cbf4f3bd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57c7fff700 1 -- 172.21.15.31:0/1694832235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40077c0 con 0x55f82cee4e20 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57c7fff700 1 -- 172.21.15.31:0/1694832235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d4007920 con 0x55f82cee4e20 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.311+0000 7f57c7fff700 1 -- 172.21.15.31:0/1694832235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4019480 con 0x55f82cee4e20 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 -- 172.21.15.31:0/1694832235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 msgr2=0x55f82cef4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57d401b040 tx=0x7f57d4013440 comp rx=0 tx=0).stop 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 -- 172.21.15.31:0/1694832235 shutdown_connections 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee4e20 0x55f82cef4840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee3620 0x55f82cee7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/1694832235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cee8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 -- 172.21.15.31:0/1694832235 >> 172.21.15.31:0/1694832235 conn(0x55f82ce53210 msgr2=0x55f82ceec060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 -- 172.21.15.31:0/1694832235 shutdown_connections 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.312+0000 7f57f6e51b80 1 -- 172.21.15.31:0/1694832235 wait complete. 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 -- start start 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 Processor -- start 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cef48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82ceef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ceeffe0 con 0x55f82cee3620 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82cf4c5c0 con 0x55f82cee4e20 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82cf4c780 con 0x55f82cee3f80 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57f6e51b80 10 client.0 _ll_register_callbacks cb 0x55f82cc2b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57e139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.313+0000 7f57e139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46392/0 (socket says 172.21.15.31:46392) 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 -- 172.21.15.31:0/463162100 learned_addr learned my addr 172.21.15.31:0/463162100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82ceef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cef48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 -- 172.21.15.31:0/463162100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 msgr2=0x55f82cef48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cef48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 -- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 msgr2=0x55f82ceef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82ceef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 -- 172.21.15.31:0/463162100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f82cc5cd90 con 0x55f82cee3620 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cef48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e1b9c700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82ceef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57e139b700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55f82cef98d0 tx=0x7f57d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2182316751b490 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80265b0 con 0x55f82cee3620 2023-10-06T21:20:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f57d8023040 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.314+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57d400d050 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8004830 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 10 client.15324 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f82cc5b260 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d400d050 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57d8012550 con 0x55f82cee3620 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 10 client.15324 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 20 client.15324 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cf51a90 0x55f82cf4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 4 client.15324 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.315+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f82ce2aae0 con 0x55f82cf51a90 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cf51a90 0x55f82cf4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57e0b9a700 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cf51a90 0x55f82cf4de90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6952b2c1c36264b6 server_cookie=a7a5055867728e3c in_seq=0 out_seq=0 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57c4ff9700 10 client.15324 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57c4ff9700 1 -- 172.21.15.31:0/463162100 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f82ce2aae0 con 0x55f82cf51a90 2023-10-06T21:20:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57c4ff9700 10 client.15324 handle_command_reply: tid=0 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.316+0000 7f57f6e51b80 1 client.15324 shutdown 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 msgr2=0x55f82ceef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55f82cef98d0 tx=0x7f57d80047c0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 shutdown_connections 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f82cf51a90 0x55f82cf4de90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f82cee4e20 0x55f82ceef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f82cee3620 0x55f82ceef370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 --2- 172.21.15.31:0/463162100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f82cee3f80 0x55f82cef48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 >> 172.21.15.31:0/463162100 conn(0x55f82ce53210 msgr2=0x55f82cee7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57c4ff9700 0 client.15324 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 shutdown_connections 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 1 -- 172.21.15.31:0/463162100 wait complete. 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.317+0000 7f57f6e51b80 20 client.15324 trim_cache size 0 max 16384 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2023-10-06T21:20:29.330 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24476, 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 88.485183652000003, 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 132.528862788 2023-10-06T21:20:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:29.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 msgr2=0x7f57c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 0x7f57c80a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57dc101be0 tx=0x7f57d8009260 comp rx=0 tx=0).stop 2023-10-06T21:20:29.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 msgr2=0x7f57dc197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0098e0 tx=0x7f57cc005ef0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 shutdown_connections 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f57c80a1480 0x7f57c80a3940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57dc102a30 0x7f57dc19e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57dc100e90 0x7f57dc196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 --2- 172.21.15.31:0/1699592843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57dc102070 0x7f57dc197030 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 >> 172.21.15.31:0/1699592843 conn(0x7f57dc0fc430 msgr2=0x7f57dc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 shutdown_connections 2023-10-06T21:20:29.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.318+0000 7f57e35ff700 1 -- 172.21.15.31:0/1699592843 wait complete. 2023-10-06T21:20:29.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-06T21:20:29.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:28 smithi142 ceph-mon[102951]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-06T21:20:29.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:28 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3430408824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:29.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:28 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2842007108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 -- 172.21.15.31:0/924036966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140102070 msgr2=0x7f41401024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 --2- 172.21.15.31:0/924036966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140102070 0x7f41401024f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4130009990 tx=0x7f4130009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 -- 172.21.15.31:0/924036966 shutdown_connections 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 --2- 172.21.15.31:0/924036966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f4140109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 --2- 172.21.15.31:0/924036966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4140100e90 0x7f41401012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 --2- 172.21.15.31:0/924036966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140102070 0x7f41401024f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 -- 172.21.15.31:0/924036966 >> 172.21.15.31:0/924036966 conn(0x7f41400fc430 msgr2=0x7f41400fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 -- 172.21.15.31:0/924036966 shutdown_connections 2023-10-06T21:20:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.539+0000 7f41475ff700 1 -- 172.21.15.31:0/924036966 wait complete. 2023-10-06T21:20:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.540+0000 7f41475ff700 1 Processor -- start 2023-10-06T21:20:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 -- start start 2023-10-06T21:20:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4140102070 0x7f4140199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f41401a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0990 con 0x7f4140100e90 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0ad0 con 0x7f4140102a30 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f41475ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0c60 con 0x7f4140102070 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46406/0 (socket says 172.21.15.31:46406) 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f414539b700 1 -- 172.21.15.31:0/2242490107 learned_addr learned my addr 172.21.15.31:0/2242490107 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f4145b9c700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f41401a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.541+0000 7f414539b700 1 -- 172.21.15.31:0/2242490107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4140102070 msgr2=0x7f4140199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f414539b700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4140102070 0x7f4140199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f414539b700 1 -- 172.21.15.31:0/2242490107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 msgr2=0x7f41401a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f414539b700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f41401a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f414539b700 1 -- 172.21.15.31:0/2242490107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300095f0 con 0x7f4140100e90 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f4145b9c700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f41401a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f414539b700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f413c00e970 tx=0x7f413c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f41367fc700 1 -- 172.21.15.31:0/2242490107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c009790 con 0x7f4140100e90 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f41367fc700 1 -- 172.21.15.31:0/2242490107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413c0044b0 con 0x7f4140100e90 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41401a0f40 con 0x7f4140100e90 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f41367fc700 1 -- 172.21.15.31:0/2242490107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c003dd0 con 0x7f4140100e90 2023-10-06T21:20:29.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.542+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a13c0 con 0x7f4140100e90 2023-10-06T21:20:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.547+0000 7f41367fc700 1 -- 172.21.15.31:0/2242490107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f413c004000 con 0x7f4140100e90 2023-10-06T21:20:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 Processor -- start 2023-10-06T21:20:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 -- start start 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db06ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dafd3410 con 0x55e8db06bb70 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dafc4bd0 con 0x55e8db06ca10 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.548+0000 7f415aee9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dafc24f0 con 0x55e8db06b210 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f41367fc700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 0x7f412c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46414/0 (socket says 172.21.15.31:46414) 2023-10-06T21:20:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 -- 172.21.15.31:0/3100563964 learned_addr learned my addr 172.21.15.31:0/3100563964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f4145b9c700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f41367fc700 1 -- 172.21.15.31:0/2242490107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f413c014070 con 0x7f4140100e90 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f4145b9c700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 0x7f412c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f4144b9a700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db06ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 -- 172.21.15.31:0/3100563964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 msgr2=0x55e8db06b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db06b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 -- 172.21.15.31:0/3100563964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 msgr2=0x55e8db06ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db06ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 -- 172.21.15.31:0/3100563964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8dade8d90 con 0x55e8db06bb70 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f4144b9a700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db06ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.549+0000 7f414539b700 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f413c099fb0 tx=0x7f413c06d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7e2d6fe1bbe07e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f4145b9c700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 0x7f412c0a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4138009fd0 tx=0x7f4138006040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f412b7fe700 1 -- 172.21.15.31:0/3100563964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c099df0 con 0x55e8db06bb70 2023-10-06T21:20:29.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f412b7fe700 1 -- 172.21.15.31:0/3100563964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413c099c10 con 0x55e8db06bb70 2023-10-06T21:20:29.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f412b7fe700 1 -- 172.21.15.31:0/3100563964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0a2640 con 0x55e8db06bb70 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 -- 172.21.15.31:0/3100563964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 msgr2=0x55e8db06c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f413c099fb0 tx=0x7f413c06d270 comp rx=0 tx=0).stop 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 -- 172.21.15.31:0/3100563964 shutdown_connections 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db06ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 --2- 172.21.15.31:0/3100563964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db06c430 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.550+0000 7f415aee9b80 1 -- 172.21.15.31:0/3100563964 >> 172.21.15.31:0/3100563964 conn(0x55e8dafdf210 msgr2=0x55e8db0660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.551+0000 7f415aee9b80 1 -- 172.21.15.31:0/3100563964 shutdown_connections 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.551+0000 7f415aee9b80 1 -- 172.21.15.31:0/3100563964 wait complete. 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.551+0000 7f415aee9b80 1 -- start start 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.551+0000 7f415aee9b80 1 Processor -- start 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db075030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06bb70 0x55e8db075570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dafbdde0 con 0x55e8db06ca10 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8daf5a6c0 con 0x55e8db06bb70 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8daf5a360 con 0x55e8db06b210 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f415aee9b80 10 client.0 _ll_register_callbacks cb 0x55e8dadb77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f4145b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f4145b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46426/0 (socket says 172.21.15.31:46426) 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f4145b9c700 1 -- 172.21.15.31:0/1487928855 learned_addr learned my addr 172.21.15.31:0/1487928855 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f4144b9a700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06bb70 0x55e8db075570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f414539b700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db075030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.552+0000 7f4145b9c700 1 -- 172.21.15.31:0/1487928855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 msgr2=0x55e8db075030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4145b9c700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db075030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4145b9c700 1 -- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06bb70 msgr2=0x55e8db075570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4145b9c700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06bb70 0x55e8db075570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4145b9c700 1 -- 172.21.15.31:0/1487928855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a13c0 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4145b9c700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4138009670 tx=0x7f4138004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25915715bb93d68c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4128ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138020070 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f413801ba50 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8db05dec0 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413801bc10 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f415aee9b80 10 client.15336 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.553+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8dadd1c10 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8db05dec0 con 0x55e8db06ca10 2023-10-06T21:20:29.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f41380193d0 con 0x55e8db06ca10 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 10 client.15336 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 20 client.15336 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db080c80 0x55e8db07f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 4 client.15336 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8dafb6dc0 con 0x55e8db080c80 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.554+0000 7f414539b700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db080c80 0x55e8db07f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.555+0000 7f414539b700 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db080c80 0x55e8db07f710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64d0b166ef1cac6e server_cookie=4671655ce3a6f117 in_seq=0 out_seq=0 2023-10-06T21:20:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.555+0000 7f4128ff9700 10 client.15336 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f4128ff9700 1 -- 172.21.15.31:0/1487928855 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8dafb6dc0 con 0x55e8db080c80 2023-10-06T21:20:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f4128ff9700 10 client.15336 handle_command_reply: tid=0 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 client.15336 shutdown 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 msgr2=0x55e8db07c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4138009670 tx=0x7f4138004ad0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 shutdown_connections 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06ca10 0x55e8db07c5b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06b210 0x55e8db075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db080c80 0x55e8db07f710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 --2- 172.21.15.31:0/1487928855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06bb70 0x55e8db075570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 >> 172.21.15.31:0/1487928855 conn(0x55e8dafdf210 msgr2=0x55e8db065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f4128ff9700 0 client.15336 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 shutdown_connections 2023-10-06T21:20:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 1 -- 172.21.15.31:0/1487928855 wait complete. 2023-10-06T21:20:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.556+0000 7f415aee9b80 20 client.15336 trim_cache size 0 max 16384 2023-10-06T21:20:29.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 Processor -- start 2023-10-06T21:20:29.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 -- start start 2023-10-06T21:20:29.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06bb70 0x55e8db06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db07c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8db074c00 con 0x55e8db06b210 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dae3e8d0 con 0x55e8db06ca10 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f415aee9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dafb6df0 con 0x55e8db06bb70 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.575+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46430/0 (socket says 172.21.15.31:46430) 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 -- 172.21.15.31:0/599813982 learned_addr learned my addr 172.21.15.31:0/599813982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f4145b9c700 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db07c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f4144b9a700 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06bb70 0x55e8db06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 -- 172.21.15.31:0/599813982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06bb70 msgr2=0x55e8db06fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06bb70 0x55e8db06fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 -- 172.21.15.31:0/599813982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 msgr2=0x55e8db07c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db07c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 -- 172.21.15.31:0/599813982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8dae0f200 con 0x55e8db06b210 2023-10-06T21:20:29.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f414539b700 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f413c0cae00 tx=0x7f413c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aa7e5203edc4af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f412bfff700 1 -- 172.21.15.31:0/599813982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c09aa30 con 0x55e8db06b210 2023-10-06T21:20:29.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f412bfff700 1 -- 172.21.15.31:0/599813982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413c09a330 con 0x55e8db06b210 2023-10-06T21:20:29.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.576+0000 7f412bfff700 1 -- 172.21.15.31:0/599813982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0983f0 con 0x55e8db06b210 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- 172.21.15.31:0/599813982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 msgr2=0x55e8db06f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f413c0cae00 tx=0x7f413c0046a0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- 172.21.15.31:0/599813982 shutdown_connections 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06ca10 0x55e8db07c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06b210 0x55e8db06f940 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 --2- 172.21.15.31:0/599813982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06bb70 0x55e8db06fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- 172.21.15.31:0/599813982 >> 172.21.15.31:0/599813982 conn(0x55e8dafdf210 msgr2=0x55e8db073c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- 172.21.15.31:0/599813982 shutdown_connections 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- 172.21.15.31:0/599813982 wait complete. 2023-10-06T21:20:29.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.577+0000 7f415aee9b80 1 -- start start 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 Processor -- start 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06b210 0x55e8db0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06ca10 0x55e8db070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8db072540 con 0x55e8db06bb70 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8db0726b0 con 0x55e8db06b210 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8db071430 con 0x55e8db06ca10 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f415aee9b80 10 client.0 _ll_register_callbacks cb 0x7f41380178e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f4144b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f4144b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46436/0 (socket says 172.21.15.31:46436) 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.578+0000 7f4144b9a700 1 -- 172.21.15.31:0/747687775 learned_addr learned my addr 172.21.15.31:0/747687775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 -- 172.21.15.31:0/747687775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06ca10 msgr2=0x55e8db070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06ca10 0x55e8db070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 -- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06b210 msgr2=0x55e8db0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06b210 0x55e8db0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 -- 172.21.15.31:0/747687775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8dade8d90 con 0x55e8db06bb70 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4144b9a700 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4140100bb0 tx=0x7f4130003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a38fabdfcff54c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4128ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000b920 con 0x55e8db06bb70 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f413c099c10 con 0x55e8db06bb70 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f413000ba80 con 0x55e8db06bb70 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413000bc80 con 0x55e8db06bb70 2023-10-06T21:20:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f415aee9b80 10 client.15342 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.579+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8dade7260 con 0x55e8db06bb70 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413c099c10 con 0x55e8db06bb70 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4130023070 con 0x55e8db06bb70 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 10 client.15342 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 20 client.15342 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db0d9410 0x55e8db0d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 4 client.15342 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.580+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8dafb6ae0 con 0x55e8db0d9410 2023-10-06T21:20:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.581+0000 7f414539b700 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db0d9410 0x55e8db0d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.581+0000 7f414539b700 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db0d9410 0x55e8db0d5810 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd10db98c7435ff server_cookie=c2020c5a20ecf782 in_seq=0 out_seq=0 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.581+0000 7f4128ff9700 10 client.15342 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.581+0000 7f4128ff9700 1 -- 172.21.15.31:0/747687775 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e8dafb6ae0 con 0x55e8db0d9410 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.581+0000 7f4128ff9700 10 client.15342 handle_command_reply: tid=0 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 client.15342 shutdown 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 msgr2=0x55e8db07c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4140100bb0 tx=0x7f4130003960 comp rx=0 tx=0).stop 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 shutdown_connections 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8db06ca10 0x55e8db070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55e8db0d9410 0x55e8db0d5810 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8db06b210 0x55e8db0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 --2- 172.21.15.31:0/747687775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e8db06bb70 0x55e8db07c4d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f4128ff9700 0 client.15342 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 >> 172.21.15.31:0/747687775 conn(0x55e8dafdf210 msgr2=0x55e8db06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 shutdown_connections 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 1 -- 172.21.15.31:0/747687775 wait complete. 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.582+0000 7f415aee9b80 20 client.15342 trim_cache size 0 max 16384 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:29.589 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24518, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 89.757898248000004, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 123.245915428 2023-10-06T21:20:29.590 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 msgr2=0x7f412c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 0x7f412c0a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4138009fd0 tx=0x7f4138006040 comp rx=0 tx=0).stop 2023-10-06T21:20:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 msgr2=0x7f4140198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f413c00e970 tx=0x7f413c00ed30 comp rx=0 tx=0).stop 2023-10-06T21:20:29.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 shutdown_connections 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f412c0a1500 0x7f412c0a39c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140102a30 0x7f41401a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140100e90 0x7f4140198e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 --2- 172.21.15.31:0/2242490107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4140102070 0x7f4140199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 >> 172.21.15.31:0/2242490107 conn(0x7f41400fc430 msgr2=0x7f4140107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 shutdown_connections 2023-10-06T21:20:29.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.583+0000 7f41475ff700 1 -- 172.21.15.31:0/2242490107 wait complete. 2023-10-06T21:20:29.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-06T21:20:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 -- 172.21.15.31:0/1594067081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 msgr2=0x7f01a01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 --2- 172.21.15.31:0/1594067081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a01024f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01900098f0 tx=0x7f0190009c00 comp rx=0 tx=0).stop 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 -- 172.21.15.31:0/1594067081 shutdown_connections 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 --2- 172.21.15.31:0/1594067081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0102a30 0x7f01a0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 --2- 172.21.15.31:0/1594067081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0100e90 0x7f01a01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 --2- 172.21.15.31:0/1594067081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a01024f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 -- 172.21.15.31:0/1594067081 >> 172.21.15.31:0/1594067081 conn(0x7f01a00fc430 msgr2=0x7f01a00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 -- 172.21.15.31:0/1594067081 shutdown_connections 2023-10-06T21:20:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.804+0000 7f01a7dff700 1 -- 172.21.15.31:0/1594067081 wait complete. 2023-10-06T21:20:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.805+0000 7f01a7dff700 1 Processor -- start 2023-10-06T21:20:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.805+0000 7f01a7dff700 1 -- start start 2023-10-06T21:20:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0100e90 0x7f01a0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a0960 con 0x7f01a0102a30 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a0aa0 con 0x7f01a0102070 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a7dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a0c30 con 0x7f01a0100e90 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46456/0 (socket says 172.21.15.31:46456) 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 -- 172.21.15.31:0/2159125794 learned_addr learned my addr 172.21.15.31:0/2159125794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0100e90 0x7f01a0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a539a700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 -- 172.21.15.31:0/2159125794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0100e90 msgr2=0x7f01a0198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0100e90 0x7f01a0198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 -- 172.21.15.31:0/2159125794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 msgr2=0x7f01a0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.806+0000 7f01a639c700 1 -- 172.21.15.31:0/2159125794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0095f0 con 0x7f01a0102a30 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f01a639c700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f019800ef30 tx=0x7f019800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f0196ffd700 1 -- 172.21.15.31:0/2159125794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198018070 con 0x7f01a0102a30 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f0196ffd700 1 -- 172.21.15.31:0/2159125794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f019800f040 con 0x7f01a0102a30 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0190009520 con 0x7f01a0102a30 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f0196ffd700 1 -- 172.21.15.31:0/2159125794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800cc70 con 0x7f01a0102a30 2023-10-06T21:20:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.807+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a01a11d0 con 0x7f01a0102a30 2023-10-06T21:20:29.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.810+0000 7f0196ffd700 1 -- 172.21.15.31:0/2159125794 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0198007500 con 0x7f01a0102a30 2023-10-06T21:20:29.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.812+0000 7f01bb775b80 1 Processor -- start 2023-10-06T21:20:29.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 -- start start 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f0196ffd700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 0x7f018c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f0196ffd700 1 -- 172.21.15.31:0/2159125794 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f01980cc8b0 con 0x7f01a0102a30 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34210 0x55b127f34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01a539a700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 0x7f018c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f35e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e9c410 con 0x55b127f34b70 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e8dbd0 con 0x55b127f34210 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01bb775b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e8b4f0 con 0x55b127f35a10 2023-10-06T21:20:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.813+0000 7f01a539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46466/0 (socket says 172.21.15.31:46466) 2023-10-06T21:20:29.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a539a700 1 -- 172.21.15.31:0/2195278785 learned_addr learned my addr 172.21.15.31:0/2195278785 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f35e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34210 0x55b127f34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 -- 172.21.15.31:0/2195278785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34210 msgr2=0x55b127f34630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34210 0x55b127f34630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 -- 172.21.15.31:0/2195278785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 msgr2=0x55b127f35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 -- 172.21.15.31:0/2195278785 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b127cb1d90 con 0x55b127f35a10 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a539a700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a539a700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 0x7f018c0a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f01900098f0 tx=0x7f019000ff60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.814+0000 7f01a639c700 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f35e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0198099b30 tx=0x7f0198099760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b881dad1cc166e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f018bfff700 1 -- 172.21.15.31:0/2195278785 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198012340 con 0x55b127f35a10 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f018bfff700 1 -- 172.21.15.31:0/2195278785 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f019809a2a0 con 0x55b127f35a10 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f018bfff700 1 -- 172.21.15.31:0/2195278785 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800ce50 con 0x55b127f35a10 2023-10-06T21:20:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 -- 172.21.15.31:0/2195278785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 msgr2=0x55b127f35e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f35e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0198099b30 tx=0x7f0198099760 comp rx=0 tx=0).stop 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 -- 172.21.15.31:0/2195278785 shutdown_connections 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f35e90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34210 0x55b127f34630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2195278785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34b70 0x55b127f35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.815+0000 7f01bb775b80 1 -- 172.21.15.31:0/2195278785 >> 172.21.15.31:0/2195278785 conn(0x55b127ea8210 msgr2=0x55b127f2f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.816+0000 7f01bb775b80 1 -- 172.21.15.31:0/2195278785 shutdown_connections 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.816+0000 7f01bb775b80 1 -- 172.21.15.31:0/2195278785 wait complete. 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.816+0000 7f01bb775b80 1 -- start start 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.816+0000 7f01bb775b80 1 Processor -- start 2023-10-06T21:20:29.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 0x55b127f3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f3e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e86de0 con 0x55b127f35a10 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e236c0 con 0x55b127f34b70 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127e23360 con 0x55b127f34210 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01bb775b80 10 client.0 _ll_register_callbacks cb 0x55b127c807b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01a639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01a639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46468/0 (socket says 172.21.15.31:46468) 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01a639c700 1 -- 172.21.15.31:0/245273013 learned_addr learned my addr 172.21.15.31:0/245273013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 0x55b127f3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.817+0000 7f01a539a700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f3e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 -- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 msgr2=0x55b127f3e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 0x55b127f3e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 -- 172.21.15.31:0/245273013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 msgr2=0x55b127f3e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f3e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 -- 172.21.15.31:0/245273013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019001f040 con 0x55b127f35a10 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 0x55b127f3e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a539a700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f3e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01a639c700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f01980090a0 tx=0x7f01980ca260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4be7539150cc3a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198099ba0 con 0x55b127f35a10 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f01980092e0 con 0x55b127f35a10 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b127f26ec0 con 0x55b127f35a10 2023-10-06T21:20:29.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0198097f40 con 0x55b127f35a10 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01bb775b80 10 client.15354 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f01a01a11d0 con 0x55b127f35a10 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.818+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b127f26ec0 con 0x55b127f35a10 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0198078ab0 con 0x55b127f35a10 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 10 client.15354 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 20 client.15354 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127f496b0 0x55b127f484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 4 client.15354 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:29.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.819+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b127e7fdc0 con 0x55b127f496b0 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.820+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127f496b0 0x55b127f484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.820+0000 7f01897fa700 10 client.15354 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.820+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127f496b0 0x55b127f484b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1969ad86ec5899be server_cookie=a456da90f2f616ad in_seq=0 out_seq=0 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.821+0000 7f01897fa700 1 -- 172.21.15.31:0/245273013 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b127e7fdc0 con 0x55b127f496b0 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.821+0000 7f01897fa700 10 client.15354 handle_command_reply: tid=0 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.821+0000 7f01bb775b80 1 client.15354 shutdown 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.821+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 msgr2=0x55b127f45580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.821+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f01980090a0 tx=0x7f01980ca260 comp rx=0 tx=0).stop 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 shutdown_connections 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127f496b0 0x55b127f484b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f35a10 0x55b127f45580 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f34210 0x55b127f3e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 --2- 172.21.15.31:0/245273013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f3e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 >> 172.21.15.31:0/245273013 conn(0x55b127ea8210 msgr2=0x55b127f2ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01897fa700 0 client.15354 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 shutdown_connections 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 1 -- 172.21.15.31:0/245273013 wait complete. 2023-10-06T21:20:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.822+0000 7f01bb775b80 20 client.15354 trim_cache size 0 max 16384 2023-10-06T21:20:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 Processor -- start 2023-10-06T21:20:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 -- start start 2023-10-06T21:20:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127d078d0 con 0x55b127f34210 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127f292b0 con 0x55b127f34b70 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01bb775b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127f3ef00 con 0x55b127f35a10 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46474/0 (socket says 172.21.15.31:46474) 2023-10-06T21:20:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.841+0000 7f01a5b9b700 1 -- 172.21.15.31:0/457765508 learned_addr learned my addr 172.21.15.31:0/457765508 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a639c700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a539a700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 -- 172.21.15.31:0/457765508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 msgr2=0x55b127f2ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f2ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 -- 172.21.15.31:0/457765508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 msgr2=0x55b127f454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 -- 172.21.15.31:0/457765508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b127c9ac10 con 0x55b127f34210 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a639c700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f019c005fd0 tx=0x7f019c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be69e98ebd2480d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.842+0000 7f0194bff700 1 -- 172.21.15.31:0/457765508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c017040 con 0x55b127f34210 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f0194bff700 1 -- 172.21.15.31:0/457765508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f019c018040 con 0x55b127f34210 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f0194bff700 1 -- 172.21.15.31:0/457765508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c004830 con 0x55b127f34210 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 -- 172.21.15.31:0/457765508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 msgr2=0x55b127f3cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f019c005fd0 tx=0x7f019c009310 comp rx=0 tx=0).stop 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 -- 172.21.15.31:0/457765508 shutdown_connections 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f3cc80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 --2- 172.21.15.31:0/457765508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 -- 172.21.15.31:0/457765508 >> 172.21.15.31:0/457765508 conn(0x55b127ea8210 msgr2=0x55b127f2db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 -- 172.21.15.31:0/457765508 shutdown_connections 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.843+0000 7f01bb775b80 1 -- 172.21.15.31:0/457765508 wait complete. 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.844+0000 7f01bb775b80 1 -- start start 2023-10-06T21:20:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.844+0000 7f01bb775b80 1 Processor -- start 2023-10-06T21:20:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127f44c00 con 0x55b127f34210 2023-10-06T21:20:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127f44d70 con 0x55b127f34b70 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b127f44f30 con 0x55b127f35a10 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01bb775b80 10 client.0 _ll_register_callbacks cb 0x7f0190009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46480/0 (socket says 172.21.15.31:46480) 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 -- 172.21.15.31:0/2972303023 learned_addr learned my addr 172.21.15.31:0/2972303023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a639c700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a539a700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 -- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 msgr2=0x55b127f444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 -- 172.21.15.31:0/2972303023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 msgr2=0x55b127f43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f43f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a5b9b700 1 -- 172.21.15.31:0/2972303023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c012590 con 0x55b127f34210 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.845+0000 7f01a539a700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01a5b9b700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f019c009a60 tx=0x7f019c012200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac9065a3c5d99cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c017040 con 0x55b127f34210 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f019809a2a0 con 0x55b127f34210 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f019c0042d0 con 0x55b127f34210 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f019c004490 con 0x55b127f34210 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01bb775b80 10 client.15360 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.846+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b127cd8200 con 0x55b127f34210 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f019809a2a0 con 0x55b127f34210 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f019c0054e0 con 0x55b127f34210 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 10 client.15360 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 20 client.15360 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127fa18c0 0x55b127f9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 4 client.15360 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b127e7fae0 con 0x55b127fa18c0 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.847+0000 7f01a539a700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127fa18c0 0x55b127f9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.848+0000 7f01a539a700 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127fa18c0 0x55b127f9dd00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75a66ef22ce3ea86 server_cookie=52b5a552dc1e286 in_seq=0 out_seq=0 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.848+0000 7f01897fa700 10 client.15360 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.848+0000 7f01897fa700 1 -- 172.21.15.31:0/2972303023 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b127e7fae0 con 0x55b127fa18c0 2023-10-06T21:20:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.848+0000 7f01897fa700 10 client.15360 handle_command_reply: tid=0 2023-10-06T21:20:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.848+0000 7f01bb775b80 1 client.15360 shutdown 2023-10-06T21:20:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 msgr2=0x55b127f439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f019c009a60 tx=0x7f019c012200 comp rx=0 tx=0).stop 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 shutdown_connections 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55b127fa18c0 0x55b127f9dd00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b127f35a10 0x55b127f444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b127f34210 0x55b127f439f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 --2- 172.21.15.31:0/2972303023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b127f34b70 0x55b127f43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01897fa700 0 client.15360 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 >> 172.21.15.31:0/2972303023 conn(0x55b127ea8210 msgr2=0x55b127f2eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 shutdown_connections 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 1 -- 172.21.15.31:0/2972303023 wait complete. 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.849+0000 7f01bb775b80 20 client.15360 trim_cache size 0 max 16384 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24526, 2023-10-06T21:20:29.857 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 91.031282099999999, 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 120.426491132 2023-10-06T21:20:29.858 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 msgr2=0x7f018c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 0x7f018c0a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f01900098f0 tx=0x7f019000ff60 comp rx=0 tx=0).stop 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 msgr2=0x7f01a01a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f019800ef30 tx=0x7f019800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 shutdown_connections 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f018c0a1500 0x7f018c0a39c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a0102a30 0x7f01a01a0390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a0100e90 0x7f01a0198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 --2- 172.21.15.31:0/2159125794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a0102070 0x7f01a0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 >> 172.21.15.31:0/2159125794 conn(0x7f01a00fc430 msgr2=0x7f01a00730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 shutdown_connections 2023-10-06T21:20:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:29.850+0000 7f01a7dff700 1 -- 172.21.15.31:0/2159125794 wait complete. 2023-10-06T21:20:29.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-06T21:20:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/1965796675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102070 msgr2=0x7f0b581024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/1965796675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102070 0x7f0b581024f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009990 tx=0x7f0b48009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/1965796675 shutdown_connections 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/1965796675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102a30 0x7f0b58109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/1965796675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58100e90 0x7f0b581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/1965796675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102070 0x7f0b581024f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/1965796675 >> 172.21.15.31:0/1965796675 conn(0x7f0b580fc430 msgr2=0x7f0b580fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/1965796675 shutdown_connections 2023-10-06T21:20:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.074+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/1965796675 wait complete. 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 Processor -- start 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 -- start start 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58100e90 0x7f0b58196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58102070 0x7f0b58197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819e650 con 0x7f0b58102a30 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819e790 con 0x7f0b58100e90 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.075+0000 7f0b5f3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819e920 con 0x7f0b58102070 2023-10-06T21:20:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46496/0 (socket says 172.21.15.31:46496) 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3113138513 learned_addr learned my addr 172.21.15.31:0/3113138513 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58100e90 0x7f0b58196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58102070 0x7f0b58197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3113138513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58102070 msgr2=0x7f0b58197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58102070 0x7f0b58197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3113138513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58100e90 msgr2=0x7f0b58196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58100e90 0x7f0b58196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3113138513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480095f0 con 0x7f0b58102a30 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.076+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000ef30 tx=0x7f0b5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.077+0000 7f0b4e7fc700 1 -- 172.21.15.31:0/3113138513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50018070 con 0x7f0b58102a30 2023-10-06T21:20:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.077+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5819eba0 con 0x7f0b58102a30 2023-10-06T21:20:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.077+0000 7f0b4e7fc700 1 -- 172.21.15.31:0/3113138513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5000f040 con 0x7f0b58102a30 2023-10-06T21:20:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.077+0000 7f0b4e7fc700 1 -- 172.21.15.31:0/3113138513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000cbc0 con 0x7f0b58102a30 2023-10-06T21:20:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.077+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5819f0b0 con 0x7f0b58102a30 2023-10-06T21:20:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.080+0000 7f0b4e7fc700 1 -- 172.21.15.31:0/3113138513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0b50007500 con 0x7f0b58102a30 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.081+0000 7f0b72de9b80 1 Processor -- start 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.081+0000 7f0b4e7fc700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 0x7f0b440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.081+0000 7f0b72de9b80 1 -- start start 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b4e7fc700 1 -- 172.21.15.31:0/3113138513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f0b500cc810 con 0x7f0b58102a30 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 0x7f0b440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135447410 con 0x55c1354e4e20 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135438bd0 con 0x55c1354e3f80 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1354364f0 con 0x55c1354e3620 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46512/0 (socket says 172.21.15.31:46512) 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5d99c700 1 -- 172.21.15.31:0/402027105 learned_addr learned my addr 172.21.15.31:0/402027105 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.082+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 -- 172.21.15.31:0/402027105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 msgr2=0x55c1354e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 -- 172.21.15.31:0/402027105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 msgr2=0x55c1354e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 -- 172.21.15.31:0/402027105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c13525cd90 con 0x55c1354e4e20 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 0x7f0b440a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005910 tx=0x7f0b4800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0b50096570 tx=0x7f0b50099710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7003953f366a1c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b437fe700 1 -- 172.21.15.31:0/402027105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50084040 con 0x55c1354e4e20 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b437fe700 1 -- 172.21.15.31:0/402027105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5009a200 con 0x55c1354e4e20 2023-10-06T21:20:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.083+0000 7f0b437fe700 1 -- 172.21.15.31:0/402027105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50097cd0 con 0x55c1354e4e20 2023-10-06T21:20:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 -- 172.21.15.31:0/402027105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 msgr2=0x55c1354e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0b50096570 tx=0x7f0b50099710 comp rx=0 tx=0).stop 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 -- 172.21.15.31:0/402027105 shutdown_connections 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354e52a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/402027105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 -- 172.21.15.31:0/402027105 >> 172.21.15.31:0/402027105 conn(0x55c135453210 msgr2=0x55c1354da0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 -- 172.21.15.31:0/402027105 shutdown_connections 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.084+0000 7f0b72de9b80 1 -- 172.21.15.31:0/402027105 wait complete. 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.085+0000 7f0b72de9b80 1 -- start start 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.085+0000 7f0b72de9b80 1 Processor -- start 2023-10-06T21:20:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3f80 0x55c1354ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135431de0 con 0x55c1354e4e20 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1353ce6c0 con 0x55c1354e3620 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1353ce360 con 0x55c1354e3f80 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b72de9b80 10 client.0 _ll_register_callbacks cb 0x55c13522b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:51060/0 (socket says 172.21.15.31:51060) 2023-10-06T21:20:30.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 -- 172.21.15.31:0/3540359220 learned_addr learned my addr 172.21.15.31:0/3540359220 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46522/0 (socket says 172.21.15.31:46522) 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3f80 0x55c1354ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 -- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3f80 msgr2=0x55c1354ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3f80 0x55c1354ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 -- 172.21.15.31:0/3540359220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 msgr2=0x55c1354f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d19b700 1 -- 172.21.15.31:0/3540359220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5819f0b0 con 0x55c1354e3620 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.086+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b54012ac0 tx=0x7f0b54012dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4d204fc5d5bcd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54010cd0 con 0x55c1354e3620 2023-10-06T21:20:30.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f0b54010e30 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1354d1ec0 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5400ae70 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b72de9b80 10 client.25172 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.087+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c135245c10 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b54028020 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b54024480 con 0x55c1354e3620 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 10 client.25172 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 10 client.25172 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 20 client.25172 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c1354f90b0 0x55c1354f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 4 client.25172 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:30.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.088+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c13542adc0 con 0x55c1354f90b0 2023-10-06T21:20:30.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.089+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c1354f90b0 0x55c1354f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.089+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c1354f90b0 0x55c1354f7ba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41704f79e3236f57 server_cookie=8a7824390145fd0e in_seq=0 out_seq=0 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.089+0000 7f0b40ff9700 10 client.25172 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b40ff9700 1 -- 172.21.15.31:0/3540359220 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c13542adc0 con 0x55c1354f90b0 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b40ff9700 10 client.25172 handle_command_reply: tid=0 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b72de9b80 1 client.25172 shutdown 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 msgr2=0x55c1354ed470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b54012ac0 tx=0x7f0b54012dd0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.090+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 shutdown_connections 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c1354f90b0 0x55c1354f7ba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b40ff9700 0 client.25172 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3620 0x55c1354ed470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3540359220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3f80 0x55c1354ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 >> 172.21.15.31:0/3540359220 conn(0x55c135453210 msgr2=0x55c1354d9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 shutdown_connections 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3540359220 wait complete. 2023-10-06T21:20:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.091+0000 7f0b72de9b80 20 client.25172 trim_cache size 0 max 16384 2023-10-06T21:20:30.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 Processor -- start 2023-10-06T21:20:30.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 -- start start 2023-10-06T21:20:30.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1352b28d0 con 0x55c1354e4e20 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1354e46d0 con 0x55c1354e3f80 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.109+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1353d84d0 con 0x55c1354e3620 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46534/0 (socket says 172.21.15.31:46534) 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3442514816 learned_addr learned my addr 172.21.15.31:0/3442514816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3442514816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 msgr2=0x55c1354ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3442514816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 msgr2=0x55c1354f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 -- 172.21.15.31:0/3442514816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c135283200 con 0x55c1354e4e20 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b50096000 tx=0x7f0b50009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739acf1322b85351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.110+0000 7f0b43fff700 1 -- 172.21.15.31:0/3442514816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50084040 con 0x55c1354e4e20 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b43fff700 1 -- 172.21.15.31:0/3442514816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5009dc50 con 0x55c1354e4e20 2023-10-06T21:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b43fff700 1 -- 172.21.15.31:0/3442514816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500b21a0 con 0x55c1354e4e20 2023-10-06T21:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3442514816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 msgr2=0x55c1354eb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b50096000 tx=0x7f0b50009ed0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3442514816 shutdown_connections 2023-10-06T21:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e4e20 0x55c1354eb240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/3442514816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e3f80 0x55c1354f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3442514816 >> 172.21.15.31:0/3442514816 conn(0x55c135453210 msgr2=0x55c1354d8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3442514816 shutdown_connections 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.111+0000 7f0b72de9b80 1 -- 172.21.15.31:0/3442514816 wait complete. 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 -- start start 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 Processor -- start 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 0x55c1354f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1354f4570 con 0x55c1354e3f80 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1354f46e0 con 0x55c1354e4e20 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1354f48a0 con 0x55c1354e3620 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b72de9b80 10 client.0 _ll_register_callbacks cb 0x7f0b5400ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b5c99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b5c99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46548/0 (socket says 172.21.15.31:46548) 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.112+0000 7f0b5c99a700 1 -- 172.21.15.31:0/1465516371 learned_addr learned my addr 172.21.15.31:0/1465516371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 0x55c1354f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 -- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 msgr2=0x55c1354ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 -- 172.21.15.31:0/1465516371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 msgr2=0x55c1354f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 0x55c1354f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 -- 172.21.15.31:0/1465516371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c13525cd90 con 0x55c1354e3f80 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5d99c700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 0x55c1354f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b5c99a700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006010 tx=0x7f0b48009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ce1a1f32f3b9a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.113+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48003d60 con 0x55c1354e3f80 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f0b48034c90 con 0x55c1354e3f80 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b5009a200 con 0x55c1354e3f80 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b48034e50 con 0x55c1354e3f80 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 10 client.15378 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:30.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c13525b260 con 0x55c1354e3f80 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5009a200 con 0x55c1354e3f80 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b4800bce0 con 0x55c1354e3f80 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 10 client.15378 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 20 client.15378 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c135551590 0x55c13554d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 4 client.15378 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.114+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c13542aae0 con 0x55c135551590 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c135551590 0x55c13554d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b5d19b700 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c135551590 0x55c13554d9d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b69bfd989bf25bc server_cookie=fe43c1f42bda0520 in_seq=0 out_seq=0 2023-10-06T21:20:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b40ff9700 10 client.15378 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b40ff9700 1 -- 172.21.15.31:0/1465516371 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c13542aae0 con 0x55c135551590 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b40ff9700 10 client.15378 handle_command_reply: tid=0 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.115+0000 7f0b72de9b80 1 client.15378 shutdown 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 msgr2=0x55c1354f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006010 tx=0x7f0b48009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 shutdown_connections 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55c135551590 0x55c13554d9d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1354e4e20 0x55c1354f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c1354e3620 0x55c1354ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 --2- 172.21.15.31:0/1465516371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c1354e3f80 0x55c1354f3950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b40ff9700 0 client.15378 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 >> 172.21.15.31:0/1465516371 conn(0x55c135453210 msgr2=0x55c1354ec090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 shutdown_connections 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 1 -- 172.21.15.31:0/1465516371 wait complete. 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.116+0000 7f0b72de9b80 20 client.15378 trim_cache size 0 max 16384 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24565, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 92.231350117999995, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 111.271026447 2023-10-06T21:20:30.126 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 msgr2=0x7f0b440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 0x7f0b440a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005910 tx=0x7f0b4800b260 comp rx=0 tx=0).stop 2023-10-06T21:20:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 msgr2=0x7f0b5819e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000ef30 tx=0x7f0b5000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 shutdown_connections 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f0b440a1500 0x7f0b440a39c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b58102a30 0x7f0b5819e080 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58100e90 0x7f0b58196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 --2- 172.21.15.31:0/3113138513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b58102070 0x7f0b58197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 >> 172.21.15.31:0/3113138513 conn(0x7f0b580fc430 msgr2=0x7f0b580fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 shutdown_connections 2023-10-06T21:20:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.117+0000 7f0b5f3ff700 1 -- 172.21.15.31:0/3113138513 wait complete. 2023-10-06T21:20:30.134 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:29 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4035726535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:30.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T21:20:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:29 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4035726535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:30.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 -- 172.21.15.31:0/155390214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80073910 msgr2=0x7f8e80073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 --2- 172.21.15.31:0/155390214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80073910 0x7f8e80073d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009990 tx=0x7f8e70009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 -- 172.21.15.31:0/155390214 shutdown_connections 2023-10-06T21:20:30.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 --2- 172.21.15.31:0/155390214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80108da0 0x7f8e8010b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 --2- 172.21.15.31:0/155390214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80074ef0 0x7f8e80073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 --2- 172.21.15.31:0/155390214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80073910 0x7f8e80073d90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 -- 172.21.15.31:0/155390214 >> 172.21.15.31:0/155390214 conn(0x7f8e800fc430 msgr2=0x7f8e800fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 -- 172.21.15.31:0/155390214 shutdown_connections 2023-10-06T21:20:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.339+0000 7f8e885ff700 1 -- 172.21.15.31:0/155390214 wait complete. 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.340+0000 7f8e885ff700 1 Processor -- start 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 -- start start 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 0x7f8e80198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 0x7f8e801a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a08d0 con 0x7f8e80074ef0 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a0a10 con 0x7f8e80073910 2023-10-06T21:20:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e885ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a0ba0 con 0x7f8e80108da0 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e85b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e85b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46570/0 (socket says 172.21.15.31:46570) 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.341+0000 7f8e85b9a700 1 -- 172.21.15.31:0/4208681067 learned_addr learned my addr 172.21.15.31:0/4208681067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e8639b700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 0x7f8e80198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 0x7f8e801a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 -- 172.21.15.31:0/4208681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 msgr2=0x7f8e801a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 0x7f8e801a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 -- 172.21.15.31:0/4208681067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 msgr2=0x7f8e80198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 0x7f8e80198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 -- 172.21.15.31:0/4208681067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700095f0 con 0x7f8e80074ef0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e8639b700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 0x7f8e80198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 0x7f8e801a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e700098e0 tx=0x7f8e70009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e777fe700 1 -- 172.21.15.31:0/4208681067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001d070 con 0x7f8e80074ef0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e777fe700 1 -- 172.21.15.31:0/4208681067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e70022470 con 0x7f8e80074ef0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e801a0e20 con 0x7f8e80074ef0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.342+0000 7f8e777fe700 1 -- 172.21.15.31:0/4208681067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000f6c0 con 0x7f8e80074ef0 2023-10-06T21:20:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.343+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a11e0 con 0x7f8e80074ef0 2023-10-06T21:20:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.346+0000 7f8e777fe700 1 -- 172.21.15.31:0/4208681067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8e70022890 con 0x7f8e80074ef0 2023-10-06T21:20:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.347+0000 7f8e9be4db80 1 Processor -- start 2023-10-06T21:20:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.347+0000 7f8e9be4db80 1 -- start start 2023-10-06T21:20:30.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e777fe700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 0x7f8e6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f8ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948ef1410 con 0x557948f89210 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948ee2bd0 con 0x557948f89b70 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948ee04f0 con 0x557948f8aa10 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e777fe700 1 -- 172.21.15.31:0/4208681067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8e700ce3d0 con 0x7f8e80074ef0 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e86b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e8639b700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 0x7f8e6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e86b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f89630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46572/0 (socket says 172.21.15.31:46572) 2023-10-06T21:20:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e86b9c700 1 -- 172.21.15.31:0/1903859575 learned_addr learned my addr 172.21.15.31:0/1903859575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.348+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f8ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 -- 172.21.15.31:0/1903859575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 msgr2=0x557948f8ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f8ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 -- 172.21.15.31:0/1903859575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 msgr2=0x557948f89630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f89630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 -- 172.21.15.31:0/1903859575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557948d06d90 con 0x557948f89b70 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e8639b700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 0x7f8e6c0a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009cb0 tx=0x7f8e7c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f8a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e70005f50 tx=0x7f8e70005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8731810369566bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e74bfe700 1 -- 172.21.15.31:0/1903859575 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700a0550 con 0x557948f89b70 2023-10-06T21:20:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e74bfe700 1 -- 172.21.15.31:0/1903859575 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e70022e90 con 0x557948f89b70 2023-10-06T21:20:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.349+0000 7f8e74bfe700 1 -- 172.21.15.31:0/1903859575 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700a18d0 con 0x557948f89b70 2023-10-06T21:20:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 -- 172.21.15.31:0/1903859575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 msgr2=0x557948f8a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f8a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e70005f50 tx=0x7f8e70005fd0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 -- 172.21.15.31:0/1903859575 shutdown_connections 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f8ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f89630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/1903859575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f8a430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 -- 172.21.15.31:0/1903859575 >> 172.21.15.31:0/1903859575 conn(0x557948efd210 msgr2=0x557948f840f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 -- 172.21.15.31:0/1903859575 shutdown_connections 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.350+0000 7f8e9be4db80 1 -- 172.21.15.31:0/1903859575 wait complete. 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 -- start start 2023-10-06T21:20:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 Processor -- start 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f93210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f9a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948edbde0 con 0x557948f89210 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948e786c0 con 0x557948f89b70 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948e78360 con 0x557948f8aa10 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.351+0000 7f8e9be4db80 10 client.0 _ll_register_callbacks cb 0x557948cd57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46584/0 (socket says 172.21.15.31:46584) 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 -- 172.21.15.31:0/7515156 learned_addr learned my addr 172.21.15.31:0/7515156 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f9a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f93210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 -- 172.21.15.31:0/7515156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 msgr2=0x557948f9a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f9a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 -- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 msgr2=0x557948f93210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f93210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 -- 172.21.15.31:0/7515156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a11e0 con 0x557948f89210 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e8639b700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004480 tx=0x7f8e7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b6d7f16b5e6e11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.352+0000 7f8e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c003e20 con 0x557948f89210 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557948f7bec0 con 0x557948f89210 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8e7c018070 con 0x557948f89210 2023-10-06T21:20:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c012450 con 0x557948f89210 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 10 client.15390 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557948cefc10 con 0x557948f89210 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557948f7bec0 con 0x557948f89210 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e7c027060 con 0x557948f89210 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 20 client.15390 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948f9e920 0x557948f9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 4 client.15390 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.353+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557948ed4dc0 con 0x557948f9e920 2023-10-06T21:20:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.354+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948f9e920 0x557948f9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.354+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948f9e920 0x557948f9d3b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcf3306250380cc server_cookie=c1f4730b278b399e in_seq=0 out_seq=0 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.354+0000 7f8e69ffb700 10 client.15390 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e69ffb700 1 -- 172.21.15.31:0/7515156 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557948ed4dc0 con 0x557948f9e920 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e69ffb700 10 client.15390 handle_command_reply: tid=0 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 client.15390 shutdown 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 msgr2=0x557948f92cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004480 tx=0x7f8e7c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 shutdown_connections 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f9a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948f9e920 0x557948f9d3b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f92cd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/7515156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f93210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 >> 172.21.15.31:0/7515156 conn(0x557948efd210 msgr2=0x557948f83f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.355+0000 7f8e69ffb700 0 client.15390 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.356+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 shutdown_connections 2023-10-06T21:20:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.356+0000 7f8e9be4db80 1 -- 172.21.15.31:0/7515156 wait complete. 2023-10-06T21:20:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.356+0000 7f8e9be4db80 20 client.15390 trim_cache size 0 max 16384 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 Processor -- start 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 -- start start 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 0x557948f9aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f8aa10 0x557948f8d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948f926e0 con 0x557948f89b70 2023-10-06T21:20:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948d5c8d0 con 0x557948f8aa10 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.374+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948f83290 con 0x557948f89210 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46588/0 (socket says 172.21.15.31:46588) 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 -- 172.21.15.31:0/772053213 learned_addr learned my addr 172.21.15.31:0/772053213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e8639b700 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 0x557948f9aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 -- 172.21.15.31:0/772053213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 msgr2=0x557948f9aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 0x557948f9aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 -- 172.21.15.31:0/772053213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f8aa10 msgr2=0x557948f8d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f8aa10 0x557948f8d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 -- 172.21.15.31:0/772053213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557948d2d200 con 0x557948f89b70 2023-10-06T21:20:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e8639b700 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 0x557948f9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8e7009f9d0 tx=0x7f8e70004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28202a6b0f01b063 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e753ff700 1 -- 172.21.15.31:0/772053213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7009b740 con 0x557948f89b70 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e753ff700 1 -- 172.21.15.31:0/772053213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7009bdc0 con 0x557948f89b70 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.375+0000 7f8e753ff700 1 -- 172.21.15.31:0/772053213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70099b30 con 0x557948f89b70 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- 172.21.15.31:0/772053213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 msgr2=0x557948f9a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8e7009f9d0 tx=0x7f8e70004320 comp rx=0 tx=0).stop 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- 172.21.15.31:0/772053213 shutdown_connections 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f8aa10 0x557948f8d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f89210 0x557948f9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/772053213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89b70 0x557948f9a210 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- 172.21.15.31:0/772053213 >> 172.21.15.31:0/772053213 conn(0x557948efd210 msgr2=0x557948fa7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- 172.21.15.31:0/772053213 shutdown_connections 2023-10-06T21:20:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- 172.21.15.31:0/772053213 wait complete. 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.376+0000 7f8e9be4db80 1 -- start start 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 Processor -- start 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948f99b10 con 0x557948f89210 2023-10-06T21:20:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948f99c80 con 0x557948f89b70 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557948f99e40 con 0x557948f8aa10 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e9be4db80 10 client.0 _ll_register_callbacks cb 0x7f8e7c004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e8639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e8639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46592/0 (socket says 172.21.15.31:46592) 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e8639b700 1 -- 172.21.15.31:0/10673634 learned_addr learned my addr 172.21.15.31:0/10673634 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.377+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 -- 172.21.15.31:0/10673634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 msgr2=0x557948f995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 -- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 msgr2=0x557948f965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 -- 172.21.15.31:0/10673634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557948d06d90 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e86b9c700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e8639b700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0039e0 tx=0x7f8e7c029a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc32e17af020d95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004610 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e70022e90 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8e7c018050 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c033470 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.378+0000 7f8e9be4db80 10 client.15396 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557948d05260 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e70022e90 con 0x557948f89210 2023-10-06T21:20:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e7c027020 con 0x557948f89210 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 10 client.15396 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 20 client.15396 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948ff6ac0 0x557948ff2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 4 client.15396 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:20:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557948ed4ae0 con 0x557948ff6ac0 2023-10-06T21:20:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.379+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948ff6ac0 0x557948ff2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e85b9a700 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948ff6ac0 0x557948ff2f20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14924bbe4f0cb752 server_cookie=3881f0415381b4b2 in_seq=0 out_seq=0 2023-10-06T21:20:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e69ffb700 10 client.15396 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e69ffb700 1 -- 172.21.15.31:0/10673634 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557948ed4ae0 con 0x557948ff6ac0 2023-10-06T21:20:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e69ffb700 10 client.15396 handle_command_reply: tid=0 2023-10-06T21:20:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 client.15396 shutdown 2023-10-06T21:20:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 msgr2=0x557948f96070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0039e0 tx=0x7f8e7c029a50 comp rx=0 tx=0).stop 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 shutdown_connections 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557948f8aa10 0x557948f995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557948ff6ac0 0x557948ff2f20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557948f89210 0x557948f96070 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e69ffb700 0 client.15396 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 --2- 172.21.15.31:0/10673634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557948f89b70 0x557948f965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.380+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 >> 172.21.15.31:0/10673634 conn(0x557948efd210 msgr2=0x557948f9aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 shutdown_connections 2023-10-06T21:20:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e9be4db80 1 -- 172.21.15.31:0/10673634 wait complete. 2023-10-06T21:20:30.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e9be4db80 20 client.15396 trim_cache size 0 max 16384 2023-10-06T21:20:30.392 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9b5d7b0a-648c-11ee-8db5-212e2dc638e7", 2023-10-06T21:20:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T21:20:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24602, 2023-10-06T21:20:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T21:20:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T21:20:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 95.520134655000007, 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 86, 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 102.246773687 2023-10-06T21:20:30.394 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 msgr2=0x7f8e6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 0x7f8e6c0a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009cb0 tx=0x7f8e7c009260 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 msgr2=0x7f8e80199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e700098e0 tx=0x7f8e70009570 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 shutdown_connections 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8e6c0a1500 0x7f8e6c0a39c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e80108da0 0x7f8e801a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e80073910 0x7f8e80198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 --2- 172.21.15.31:0/4208681067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80074ef0 0x7f8e80199290 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.381+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 >> 172.21.15.31:0/4208681067 conn(0x7f8e800fc430 msgr2=0x7f8e800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.382+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 shutdown_connections 2023-10-06T21:20:30.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:30.382+0000 7f8e885ff700 1 -- 172.21.15.31:0/4208681067 wait complete. 2023-10-06T21:20:30.402 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T21:20:30.723 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T21:20:30.723 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-10-06T21:20:30.723 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2023-10-06T21:20:30.723 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-06T21:20:30.724 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:30.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:30.724 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T21:20:30.725 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:29 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4035726535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:30.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:30.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T21:20:30.889 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T21:20:30.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:20:30.893 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T21:20:30.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-06T21:20:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:30 smithi031 ceph-mon[98539]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s 2023-10-06T21:20:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:30 smithi146 ceph-mon[102809]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s 2023-10-06T21:20:31.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:30 smithi142 ceph-mon[102951]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s 2023-10-06T21:20:31.632 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:20:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 -- 172.21.15.31:0/741217164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48100fa0 msgr2=0x7f3d48101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 --2- 172.21.15.31:0/741217164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48100fa0 0x7f3d48101380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009990 tx=0x7f3d44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 -- 172.21.15.31:0/741217164 shutdown_connections 2023-10-06T21:20:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 --2- 172.21.15.31:0/741217164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48106080 0x7f3d4810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 --2- 172.21.15.31:0/741217164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48100fa0 0x7f3d48101380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 --2- 172.21.15.31:0/741217164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48101950 0x7f3d481059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 -- 172.21.15.31:0/741217164 >> 172.21.15.31:0/741217164 conn(0x7f3d480fc8b0 msgr2=0x7f3d480fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 -- 172.21.15.31:0/741217164 shutdown_connections 2023-10-06T21:20:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.121+0000 7f3d4e919700 1 -- 172.21.15.31:0/741217164 wait complete. 2023-10-06T21:20:33.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.122+0000 7f3d4e919700 1 Processor -- start 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 -- start start 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48100fa0 0x7f3d48196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106080 0x7f3d4819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4819bd50 con 0x7f3d48101950 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4819bec0 con 0x7f3d48106080 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4e919700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4819c080 con 0x7f3d48100fa0 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4d116700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:33.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4d116700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46292/0 (socket says 172.21.15.31:46292) 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.123+0000 7f3d4d116700 1 -- 172.21.15.31:0/1474524450 learned_addr learned my addr 172.21.15.31:0/1474524450 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4e118700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106080 0x7f3d4819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d917700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48100fa0 0x7f3d48196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 -- 172.21.15.31:0/1474524450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48100fa0 msgr2=0x7f3d48196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48100fa0 0x7f3d48196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 -- 172.21.15.31:0/1474524450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106080 msgr2=0x7f3d4819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106080 0x7f3d4819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 -- 172.21.15.31:0/1474524450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440095f0 con 0x7f3d48101950 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4d116700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800d980 tx=0x7f3d3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800f890 con 0x7f3d48101950 2023-10-06T21:20:33.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4819c360 con 0x7f3d48101950 2023-10-06T21:20:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d3800fe80 con 0x7f3d48101950 2023-10-06T21:20:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.124+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e3b0 con 0x7f3d48101950 2023-10-06T21:20:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.125+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d481a3bc0 con 0x7f3d48101950 2023-10-06T21:20:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.126+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4804f400 con 0x7f3d48101950 2023-10-06T21:20:33.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.128+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3d3800f9f0 con 0x7f3d48101950 2023-10-06T21:20:33.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.129+0000 7f3d3effd700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 0x7f3d340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:33.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.129+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f3d380cc800 con 0x7f3d48101950 2023-10-06T21:20:33.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.129+0000 7f3d4d917700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 0x7f3d340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:33.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.130+0000 7f3d4d917700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 0x7f3d340a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400b5c0 tx=0x7f3d44009f00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:33.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.132+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d38095da0 con 0x7f3d48101950 2023-10-06T21:20:33.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:32 smithi031 ceph-mon[98539]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:33.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:32 smithi146 ceph-mon[102809]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.386+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d48110c90 con 0x7f3d48101950 2023-10-06T21:20:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.387+0000 7f3d3effd700 1 -- 172.21.15.31:0/1474524450 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3d38020070 con 0x7f3d48101950 2023-10-06T21:20:33.389 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T21:20:33.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 msgr2=0x7f3d340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:33.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 0x7f3d340a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400b5c0 tx=0x7f3d44009f00 comp rx=0 tx=0).stop 2023-10-06T21:20:33.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 msgr2=0x7f3d481972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800d980 tx=0x7f3d3800dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 shutdown_connections 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3d340a1480 0x7f3d340a3940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106080 0x7f3d4819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d48100fa0 0x7f3d48196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 --2- 172.21.15.31:0/1474524450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48101950 0x7f3d481972d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 >> 172.21.15.31:0/1474524450 conn(0x7f3d480fc8b0 msgr2=0x7f3d480fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 shutdown_connections 2023-10-06T21:20:33.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:33.389+0000 7f3d4e919700 1 -- 172.21.15.31:0/1474524450 wait complete. 2023-10-06T21:20:33.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:32 smithi142 ceph-mon[102951]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:33.814 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T21:20:33.814 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T21:20:33.815 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T21:20:33.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:33.815 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T21:20:33.863 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T21:20:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T21:20:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T21:20:33.864 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41508sec preferred_lft 41508sec 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T21:20:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T21:20:33.866 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T21:20:33.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:33.866 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T21:20:33.866 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T21:20:33.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-10-06T21:20:33.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-10-06T21:20:33.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-10-06T21:20:33.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T21:20:33.867 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T21:20:34.244 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:33 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1474524450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:20:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:33 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1474524450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:20:34.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:34.342 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T21:20:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:33 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1474524450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:20:34.418 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T21:20:34.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:34.421 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T21:20:34.481 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T21:20:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T21:20:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T21:20:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T21:20:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T21:20:34.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:34.486 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T21:20:34.487 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T21:20:34.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T21:20:34.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T21:20:34.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T21:20:34.487 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T21:20:35.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:34 smithi031 ceph-mon[98539]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:35.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:35.152 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T21:20:35.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:35.198 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-10-06T21:20:35.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:35.281 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T21:20:35.281 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T21:20:35.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:35.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T21:20:35.281 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T21:20:35.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:34 smithi146 ceph-mon[102809]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:35.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:34 smithi142 ceph-mon[102951]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2023-10-06T21:20:35.467 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T21:20:35.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T21:20:35.468 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T21:20:35.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:35.745 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T21:20:35.745 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T21:20:35.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T21:20:35.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T21:20:35.746 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T21:20:35.923 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2023-10-06T21:20:35.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T21:20:35.926 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:35.971 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T21:20:35.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T21:20:35.971 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:36.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2023-10-06T21:20:36.167 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2023-10-06T21:20:36.211 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2023-10-06T21:20:36.211 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/systemd 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/memory 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-06T21:20:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/blkio 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/hugetlb 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/perf_event 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/freezer 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpuset 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/devices 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/pids 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:36.213 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b502686b82c397a1c262f826c8b7bbc71a393d11c8e842eebfbe0ed5ea2b747b/merged 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/229b3b8e4133e1e7296c030ba61790632246fedd0a1ce94cc404301380946a03/merged 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2023-10-06T21:20:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/da80753a4ccd3bf1b7a0efa93fd3bc86d0cea3d3b00deeeb32cbc7be6e04c656/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/400906468a420ffdc73cbfd2a95464224a92dcb6dc08ad49170edcc181d552f4/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1fccb6d9708c8d00c4bac5f9cdfe479b2f7e692af1f7ccdd2a23f44585250b93/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/682f9ebe6b5d4201c4bdfc1c36221ade0c69549540c1b39ecb7cad45f13926df/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7f121299d7ca98db2497718ca2c5385267875c763affaca3e9ea46b5b6316b72/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5e588a7da8ea7c4a8e5b1838d35ed3a1cb2537d2b82c0bb6963187884ec647f4/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/330f786f27db8498c389bb73abde40d76f7cbec883b07eca0ab91637bbaee443/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/252b17b98cb3bbb0467e5440af681c88ff68d991cdb004fc19de2e0b99c0b9ca/merged 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:36.215 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:36.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:36.216 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2023-10-06T21:20:36.300 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T21:20:36.301 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a --client_fs=cephfs) 2023-10-06T21:20:36.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2023-10-06T21:20:36.418 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2023-10-06T21:20:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2023-10-06T21:20:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2023-10-06T21:20:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2023-10-06T21:20:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2023-10-06T21:20:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/systemd 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/memory 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/blkio 2023-10-06T21:20:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/hugetlb 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/perf_event 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/freezer 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpuset 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/devices 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/pids 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2023-10-06T21:20:36.467 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b502686b82c397a1c262f826c8b7bbc71a393d11c8e842eebfbe0ed5ea2b747b/merged 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/229b3b8e4133e1e7296c030ba61790632246fedd0a1ce94cc404301380946a03/merged 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/da80753a4ccd3bf1b7a0efa93fd3bc86d0cea3d3b00deeeb32cbc7be6e04c656/merged 2023-10-06T21:20:36.468 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/400906468a420ffdc73cbfd2a95464224a92dcb6dc08ad49170edcc181d552f4/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1fccb6d9708c8d00c4bac5f9cdfe479b2f7e692af1f7ccdd2a23f44585250b93/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/682f9ebe6b5d4201c4bdfc1c36221ade0c69549540c1b39ecb7cad45f13926df/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7f121299d7ca98db2497718ca2c5385267875c763affaca3e9ea46b5b6316b72/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5e588a7da8ea7c4a8e5b1838d35ed3a1cb2537d2b82c0bb6963187884ec647f4/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/330f786f27db8498c389bb73abde40d76f7cbec883b07eca0ab91637bbaee443/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/252b17b98cb3bbb0467e5440af681c88ff68d991cdb004fc19de2e0b99c0b9ca/merged 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:36.469 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:36.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:36.470 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2023-10-06T21:20:36.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2023-10-06T21:20:36.538+0000 7fe79716d5c0 -1 init, newargv = 0x55b5987edf20 newargc=15 2023-10-06T21:20:36.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[160790]: starting ceph client 2023-10-06T21:20:36.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[160790]: starting fuse 2023-10-06T21:20:37.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:36 smithi031 ceph-mon[98539]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 514 B/s wr, 10 op/s 2023-10-06T21:20:37.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:36 smithi146 ceph-mon[102809]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 514 B/s wr, 10 op/s 2023-10-06T21:20:37.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:36 smithi142 ceph-mon[102951]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 514 B/s wr, 10 op/s 2023-10-06T21:20:37.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2023-10-06T21:20:37.695 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2023-10-06T21:20:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/systemd 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/memory 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/blkio 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/hugetlb 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/perf_event 2023-10-06T21:20:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/freezer 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/cpuset 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/devices 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup/pids 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2023-10-06T21:20:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b502686b82c397a1c262f826c8b7bbc71a393d11c8e842eebfbe0ed5ea2b747b/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/229b3b8e4133e1e7296c030ba61790632246fedd0a1ce94cc404301380946a03/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/da80753a4ccd3bf1b7a0efa93fd3bc86d0cea3d3b00deeeb32cbc7be6e04c656/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/400906468a420ffdc73cbfd2a95464224a92dcb6dc08ad49170edcc181d552f4/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/1fccb6d9708c8d00c4bac5f9cdfe479b2f7e692af1f7ccdd2a23f44585250b93/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/682f9ebe6b5d4201c4bdfc1c36221ade0c69549540c1b39ecb7cad45f13926df/merged 2023-10-06T21:20:37.744 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/7f121299d7ca98db2497718ca2c5385267875c763affaca3e9ea46b5b6316b72/merged 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5e588a7da8ea7c4a8e5b1838d35ed3a1cb2537d2b82c0bb6963187884ec647f4/merged 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/330f786f27db8498c389bb73abde40d76f7cbec883b07eca0ab91637bbaee443/merged 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/252b17b98cb3bbb0467e5440af681c88ff68d991cdb004fc19de2e0b99c0b9ca/merged 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:20:37.745 INFO:teuthology.orchestra.run.smithi031.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:37.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:37.746 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2023-10-06T21:20:37.790 INFO:teuthology.orchestra.run.smithi031.stdout:121 2023-10-06T21:20:37.791 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> import re 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> import os 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> 2023-10-06T21:20:37.791 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T21:20:37.792 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> return f 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2023-10-06T21:20:37.793 DEBUG:teuthology.orchestra.run.smithi031:> ' 2023-10-06T21:20:37.978 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.160790.asok 2023-10-06T21:20:37.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-06T21:20:37.991 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.160790.asok 2023-10-06T21:20:37.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:37.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.160790.asok status 2023-10-06T21:20:38.223 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "14d8d032507701c76ca224e769ee65a96b660a8c", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "160790", 2023-10-06T21:20:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a" 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 15408, 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2023-10-06T21:20:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 15408 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2425227403 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2023-10-06T21:20:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2425227403 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.15408 192.168.0.1:0/2425227403", 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2425227403", 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 5, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 27, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 90, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2023-10-06T21:20:38.227 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:38.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:38.238 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2023-10-06T21:20:38.238 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:38.283 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2023-10-06T21:20:38.283 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:38.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:20:38.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:38.395 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-06T21:20:38.411 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-06T21:20:38.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 -- 172.21.15.31:0/3310384117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0102a40 msgr2=0x7f3ea0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/3310384117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0102a40 0x7f3ea0109360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800b0a0 tx=0x7f3e9800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 -- 172.21.15.31:0/3310384117 shutdown_connections 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/3310384117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0102a40 0x7f3ea0109360 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/3310384117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0100ea0 0x7f3ea01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/3310384117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102080 0x7f3ea0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 -- 172.21.15.31:0/3310384117 >> 172.21.15.31:0/3310384117 conn(0x7f3ea00fc420 msgr2=0x7f3ea00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 -- 172.21.15.31:0/3310384117 shutdown_connections 2023-10-06T21:20:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.621+0000 7f3ea4dff700 1 -- 172.21.15.31:0/3310384117 wait complete. 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.622+0000 7f3ea4dff700 1 Processor -- start 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 -- start start 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 0x7f3ea0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102a40 0x7f3ea01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a09c0 con 0x7f3ea0100ea0 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a0b00 con 0x7f3ea0102a40 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3ea4dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a0c90 con 0x7f3ea0102080 2023-10-06T21:20:38.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46300/0 (socket says 172.21.15.31:46300) 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3e9e59c700 1 -- 172.21.15.31:0/2154886085 learned_addr learned my addr 172.21.15.31:0/2154886085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.623+0000 7f3e9ed9d700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102a40 0x7f3ea01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9dd9b700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 0x7f3ea0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 -- 172.21.15.31:0/2154886085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 msgr2=0x7f3ea0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 0x7f3ea0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 -- 172.21.15.31:0/2154886085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102a40 msgr2=0x7f3ea01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102a40 0x7f3ea01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 -- 172.21.15.31:0/2154886085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98009d00 con 0x7f3ea0100ea0 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9dd9b700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 0x7f3ea0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e9e59c700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000d980 tx=0x7f3e9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004020 con 0x7f3ea0100ea0 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e90009730 con 0x7f3ea0100ea0 2023-10-06T21:20:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01a0f70 con 0x7f3ea0100ea0 2023-10-06T21:20:38.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90003a00 con 0x7f3ea0100ea0 2023-10-06T21:20:38.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.624+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea01a13a0 con 0x7f3ea0100ea0 2023-10-06T21:20:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.625+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea010c610 con 0x7f3ea0100ea0 2023-10-06T21:20:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.627+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3e900098a0 con 0x7f3ea0100ea0 2023-10-06T21:20:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.629+0000 7f3e8f7fe700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 0x7f3e880a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.629+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f3e900ccd70 con 0x7f3ea0100ea0 2023-10-06T21:20:38.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.629+0000 7f3e9dd9b700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 0x7f3e880a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.630+0000 7f3e9dd9b700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 0x7f3e880a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e94005fd0 tx=0x7f3e94005f00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.631+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e90096390 con 0x7f3ea0100ea0 2023-10-06T21:20:38.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.843+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea010c850 con 0x7f3ea0100ea0 2023-10-06T21:20:38.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.845+0000 7f3e8f7fe700 1 -- 172.21.15.31:0/2154886085 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22812 (secure 0 0 0) 0x7f3e90017090 con 0x7f3ea0100ea0 2023-10-06T21:20:38.849 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:38.849 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:26.886529+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 msgr2=0x7f3e880a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 0x7f3e880a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e94005fd0 tx=0x7f3e94005f00 comp rx=0 tx=0).stop 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 msgr2=0x7f3ea0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000d980 tx=0x7f3e9000dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 shutdown_connections 2023-10-06T21:20:38.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f3e880a1550 0x7f3e880a3a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea0102a40 0x7f3ea01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ea0100ea0 0x7f3ea0198e40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 --2- 172.21.15.31:0/2154886085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea0102080 0x7f3ea0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 >> 172.21.15.31:0/2154886085 conn(0x7f3ea00fc420 msgr2=0x7f3ea00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 shutdown_connections 2023-10-06T21:20:38.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:38.848+0000 7f3ea4dff700 1 -- 172.21.15.31:0/2154886085 wait complete. 2023-10-06T21:20:38.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:20:38.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:20:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 -- 172.21.15.31:0/1281418418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4108db0 msgr2=0x7f22f410b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 --2- 172.21.15.31:0/1281418418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4108db0 0x7f22f410b240 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00b0a0 tx=0x7f22ec00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 -- 172.21.15.31:0/1281418418 shutdown_connections 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 --2- 172.21.15.31:0/1281418418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4108db0 0x7f22f410b240 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 --2- 172.21.15.31:0/1281418418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4074ef0 0x7f22f4073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 --2- 172.21.15.31:0/1281418418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 -- 172.21.15.31:0/1281418418 >> 172.21.15.31:0/1281418418 conn(0x7f22f40fc460 msgr2=0x7f22f40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 -- 172.21.15.31:0/1281418418 shutdown_connections 2023-10-06T21:20:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.079+0000 7f22f8bff700 1 -- 172.21.15.31:0/1281418418 wait complete. 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 Processor -- start 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 -- start start 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4108db0 0x7f22f41a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a0970 con 0x7f22f4074ef0 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a0ab0 con 0x7f22f4108db0 2023-10-06T21:20:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.080+0000 7f22f8bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a0c40 con 0x7f22f4073910 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46322/0 (socket says 172.21.15.31:46322) 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 -- 172.21.15.31:0/3860234218 learned_addr learned my addr 172.21.15.31:0/3860234218 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f2d9d700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4108db0 0x7f22f41a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f259c700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 -- 172.21.15.31:0/3860234218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 msgr2=0x7f22f4198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 -- 172.21.15.31:0/3860234218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4108db0 msgr2=0x7f22f41a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4108db0 0x7f22f41a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 -- 172.21.15.31:0/3860234218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009d00 con 0x7f22f4074ef0 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f259c700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22f1d9b700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d980 tx=0x7f22e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.081+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8004020 con 0x7f22f4074ef0 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.082+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f41a0f20 con 0x7f22f4074ef0 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.082+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22e8009730 con 0x7f22f4074ef0 2023-10-06T21:20:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.082+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e80039c0 con 0x7f22f4074ef0 2023-10-06T21:20:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.082+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f41a13a0 con 0x7f22f4074ef0 2023-10-06T21:20:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.083+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f4107460 con 0x7f22f4074ef0 2023-10-06T21:20:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.085+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f22e80098a0 con 0x7f22f4074ef0 2023-10-06T21:20:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.086+0000 7f22e37fe700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 0x7f22dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.086+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f22e80ccab0 con 0x7f22f4074ef0 2023-10-06T21:20:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.087+0000 7f22f259c700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 0x7f22dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.087+0000 7f22f259c700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 0x7f22dc0a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22e4005fd0 tx=0x7f22e4005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.091+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f22e8096050 con 0x7f22f4074ef0 2023-10-06T21:20:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:38 smithi031 ceph-mon[98539]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 467 B/s wr, 10 op/s 2023-10-06T21:20:39.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:38 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2154886085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:38 smithi146 ceph-mon[102809]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 467 B/s wr, 10 op/s 2023-10-06T21:20:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:38 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2154886085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:39.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.302+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22f402d020 con 0x7f22f4074ef0 2023-10-06T21:20:39.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.302+0000 7f22e37fe700 1 -- 172.21.15.31:0/3860234218 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22e8099670 con 0x7f22f4074ef0 2023-10-06T21:20:39.305 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:20:39.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 msgr2=0x7f22dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 0x7f22dc0a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22e4005fd0 tx=0x7f22e4005ee0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 msgr2=0x7f22f4199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f22e800d980 tx=0x7f22e800dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 shutdown_connections 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f22dc0a1480 0x7f22dc0a3940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4108db0 0x7f22f41a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22f4073910 0x7f22f4198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 --2- 172.21.15.31:0/3860234218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4074ef0 0x7f22f4199380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 >> 172.21.15.31:0/3860234218 conn(0x7f22f40fc460 msgr2=0x7f22f40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 shutdown_connections 2023-10-06T21:20:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.305+0000 7f22f8bff700 1 -- 172.21.15.31:0/3860234218 wait complete. 2023-10-06T21:20:39.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:20:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:38 smithi142 ceph-mon[102951]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 467 B/s wr, 10 op/s 2023-10-06T21:20:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:38 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2154886085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.529+0000 7f9e225ff700 1 -- 172.21.15.31:0/3365422013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102070 msgr2=0x7f9e1c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 --2- 172.21.15.31:0/3365422013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102070 0x7f9e1c1024f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009990 tx=0x7f9e10009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 -- 172.21.15.31:0/3365422013 shutdown_connections 2023-10-06T21:20:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 --2- 172.21.15.31:0/3365422013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102a30 0x7f9e1c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 --2- 172.21.15.31:0/3365422013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c100e90 0x7f9e1c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 --2- 172.21.15.31:0/3365422013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c102070 0x7f9e1c1024f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 -- 172.21.15.31:0/3365422013 >> 172.21.15.31:0/3365422013 conn(0x7f9e1c0fc430 msgr2=0x7f9e1c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 -- 172.21.15.31:0/3365422013 shutdown_connections 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.530+0000 7f9e225ff700 1 -- 172.21.15.31:0/3365422013 wait complete. 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 Processor -- start 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 -- start start 2023-10-06T21:20:39.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102070 0x7f9e1c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c102a30 0x7f9e1c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a08f0 con 0x7f9e1c100e90 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a0a30 con 0x7f9e1c102070 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e225ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a0bc0 con 0x7f9e1c102a30 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.531+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46340/0 (socket says 172.21.15.31:46340) 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 -- 172.21.15.31:0/1289784887 learned_addr learned my addr 172.21.15.31:0/1289784887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1b7fe700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102070 0x7f9e1c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e20b9c700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c102a30 0x7f9e1c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 -- 172.21.15.31:0/1289784887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c102a30 msgr2=0x7f9e1c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c102a30 0x7f9e1c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 -- 172.21.15.31:0/1289784887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102070 msgr2=0x7f9e1c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102070 0x7f9e1c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 -- 172.21.15.31:0/1289784887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100095f0 con 0x7f9e1c100e90 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e1bfff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d980 tx=0x7f9e0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c004020 con 0x7f9e1c100e90 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e0c009730 con 0x7f9e1c100e90 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.532+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1c1a0ea0 con 0x7f9e1c100e90 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.533+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c003a00 con 0x7f9e1c100e90 2023-10-06T21:20:39.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.533+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c1a12d0 con 0x7f9e1c100e90 2023-10-06T21:20:39.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.534+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1c10c600 con 0x7f9e1c100e90 2023-10-06T21:20:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.536+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9e0c0098a0 con 0x7f9e1c100e90 2023-10-06T21:20:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.537+0000 7f9e197fa700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 0x7f9e040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.537+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f9e0c0cca00 con 0x7f9e1c100e90 2023-10-06T21:20:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.537+0000 7f9e1b7fe700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 0x7f9e040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.538+0000 7f9e1b7fe700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 0x7f9e040a39c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e1000b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.542+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e0c095fa0 con 0x7f9e1c100e90 2023-10-06T21:20:39.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.754+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e1c1a1520 con 0x7f9e1c100e90 2023-10-06T21:20:39.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.754+0000 7f9e197fa700 1 -- 172.21.15.31:0/1289784887 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e0c0995c0 con 0x7f9e1c100e90 2023-10-06T21:20:39.757 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:39.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 msgr2=0x7f9e040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 0x7f9e040a39c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e1000b560 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 msgr2=0x7f9e1c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d980 tx=0x7f9e0c00dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 shutdown_connections 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f9e040a1500 0x7f9e040a39c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e1c102a30 0x7f9e1c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e1c100e90 0x7f9e1c198d70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 --2- 172.21.15.31:0/1289784887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c102070 0x7f9e1c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.756+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 >> 172.21.15.31:0/1289784887 conn(0x7f9e1c0fc430 msgr2=0x7f9e1c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.757+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 shutdown_connections 2023-10-06T21:20:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.757+0000 7f9e225ff700 1 -- 172.21.15.31:0/1289784887 wait complete. 2023-10-06T21:20:39.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 -- 172.21.15.31:0/2244269209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68100ea0 msgr2=0x7fbe681012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/2244269209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68100ea0 0x7fbe681012c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009990 tx=0x7fbe64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 -- 172.21.15.31:0/2244269209 shutdown_connections 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/2244269209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe68109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/2244269209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68100ea0 0x7fbe681012c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/2244269209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68102080 0x7fbe68102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 -- 172.21.15.31:0/2244269209 >> 172.21.15.31:0/2244269209 conn(0x7fbe680fc460 msgr2=0x7fbe680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 -- 172.21.15.31:0/2244269209 shutdown_connections 2023-10-06T21:20:39.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.986+0000 7fbe6fdff700 1 -- 172.21.15.31:0/2244269209 wait complete. 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 Processor -- start 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 -- start start 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 0x7fbe68198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe681a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe681a0820 con 0x7fbe68102080 2023-10-06T21:20:39.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe681a0990 con 0x7fbe68100ea0 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.987+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe681a0b50 con 0x7fbe68102a40 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46354/0 (socket says 172.21.15.31:46354) 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 -- 172.21.15.31:0/960309213 learned_addr learned my addr 172.21.15.31:0/960309213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6e39c700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe681a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6db9b700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 0x7fbe68198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 -- 172.21.15.31:0/960309213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 msgr2=0x7fbe681a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe681a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 -- 172.21.15.31:0/960309213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 msgr2=0x7fbe68198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 0x7fbe68198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 -- 172.21.15.31:0/960309213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640095f0 con 0x7fbe68102080 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6db9b700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 0x7fbe68198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6e39c700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe681a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:39.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.988+0000 7fbe6d39a700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c990 tx=0x7fbe5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.989+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007b00 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.989+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe681a0e30 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.989+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe58007c60 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.989+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58018660 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.989+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe681a12b0 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.990+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe681933b0 con 0x7fbe68102080 2023-10-06T21:20:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.992+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbe5800f450 con 0x7fbe68102080 2023-10-06T21:20:39.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.993+0000 7fbe5effd700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 0x7fbe540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:39.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.993+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fbe580cce40 con 0x7fbe68102080 2023-10-06T21:20:39.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.994+0000 7fbe6db9b700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 0x7fbe540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:39.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.994+0000 7fbe6db9b700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 0x7fbe540a3940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe64000c00 tx=0x7fbe6400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:39.995+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbe58096470 con 0x7fbe68102080 2023-10-06T21:20:40.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.204+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe6810c810 con 0x7fbe68102080 2023-10-06T21:20:40.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.205+0000 7fbe5effd700 1 -- 172.21.15.31:0/960309213 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe58099a90 con 0x7fbe68102080 2023-10-06T21:20:40.208 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:40.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 msgr2=0x7fbe540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 0x7fbe540a3940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe64000c00 tx=0x7fbe6400b540 comp rx=0 tx=0).stop 2023-10-06T21:20:40.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 msgr2=0x7fbe681991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c990 tx=0x7fbe5800cd50 comp rx=0 tx=0).stop 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 shutdown_connections 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbe540a1480 0x7fbe540a3940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe68102a40 0x7fbe681a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68100ea0 0x7fbe68198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 --2- 172.21.15.31:0/960309213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe68102080 0x7fbe681991f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 >> 172.21.15.31:0/960309213 conn(0x7fbe680fc460 msgr2=0x7fbe68107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 shutdown_connections 2023-10-06T21:20:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.207+0000 7fbe6fdff700 1 -- 172.21.15.31:0/960309213 wait complete. 2023-10-06T21:20:40.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:39 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3860234218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:39 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1289784887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:39 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3860234218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:39 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1289784887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:39 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3860234218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:39 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1289784887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1033092914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4102080 msgr2=0x7f97e4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1033092914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4102080 0x7f97e4102500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009990 tx=0x7f97d8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1033092914 shutdown_connections 2023-10-06T21:20:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1033092914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 0x7f97e4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1033092914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4100ea0 0x7f97e41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1033092914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4102080 0x7f97e4102500 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1033092914 >> 172.21.15.31:0/1033092914 conn(0x7f97e40fc420 msgr2=0x7f97e40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1033092914 shutdown_connections 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.430+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1033092914 wait complete. 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 Processor -- start 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 -- start start 2023-10-06T21:20:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4102080 0x7f97e40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 0x7f97e40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e4073f80 con 0x7f97e4100ea0 2023-10-06T21:20:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e40740f0 con 0x7f97e4102080 2023-10-06T21:20:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.431+0000 7f97ea9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e41a0b60 con 0x7f97e4102a40 2023-10-06T21:20:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46374/0 (socket says 172.21.15.31:46374) 2023-10-06T21:20:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 -- 172.21.15.31:0/1206673743 learned_addr learned my addr 172.21.15.31:0/1206673743 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e8f9c700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 0x7f97e40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e37fe700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4102080 0x7f97e40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 -- 172.21.15.31:0/1206673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 msgr2=0x7f97e40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 0x7f97e40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 -- 172.21.15.31:0/1206673743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4102080 msgr2=0x7f97e40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4102080 0x7f97e40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 -- 172.21.15.31:0/1206673743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80095f0 con 0x7f97e4100ea0 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.432+0000 7f97e3fff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e970 tx=0x7f97d400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.433+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4009790 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.433+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97d400c7e0 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.433+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4003bd0 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.433+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e41a0e40 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.433+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e41a11b0 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.434+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e410c610 con 0x7f97e4100ea0 2023-10-06T21:20:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.436+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f97d400c950 con 0x7f97e4100ea0 2023-10-06T21:20:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.437+0000 7f97e17fa700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 0x7f97cc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.437+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f97d4014070 con 0x7f97e4100ea0 2023-10-06T21:20:40.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.438+0000 7f97e37fe700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 0x7f97cc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.438+0000 7f97e37fe700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 0x7f97cc0a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97d8005910 tx=0x7f97d800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.441+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f97d4096500 con 0x7f97e4100ea0 2023-10-06T21:20:40.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.654+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97e410c850 con 0x7f97e4100ea0 2023-10-06T21:20:40.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.656+0000 7f97e17fa700 1 -- 172.21.15.31:0/1206673743 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22812 (secure 0 0 0) 0x7f97d4099b20 con 0x7f97e4100ea0 2023-10-06T21:20:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:40.659 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:26.886529+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:40.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 msgr2=0x7f97cc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 0x7f97cc0a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97d8005910 tx=0x7f97d800b260 comp rx=0 tx=0).stop 2023-10-06T21:20:40.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 msgr2=0x7f97e4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e970 tx=0x7f97d400ec80 comp rx=0 tx=0).stop 2023-10-06T21:20:40.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 shutdown_connections 2023-10-06T21:20:40.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f97cc0a14d0 0x7f97cc0a3990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97e4102a40 0x7f97e40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e4100ea0 0x7f97e4074c60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 --2- 172.21.15.31:0/1206673743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97e4102080 0x7f97e40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 >> 172.21.15.31:0/1206673743 conn(0x7f97e40fc420 msgr2=0x7f97e40fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:40.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 shutdown_connections 2023-10-06T21:20:40.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.658+0000 7f97ea9ff700 1 -- 172.21.15.31:0/1206673743 wait complete. 2023-10-06T21:20:40.681 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:20:40.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1832533254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 msgr2=0x7f59e4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1832533254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e4109360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b0a0 tx=0x7f59dc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1832533254 shutdown_connections 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1832533254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e4109360 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1832533254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4100ea0 0x7f59e41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1832533254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4102080 0x7f59e4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1832533254 >> 172.21.15.31:0/1832533254 conn(0x7f59e40fc460 msgr2=0x7f59e40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:40.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1832533254 shutdown_connections 2023-10-06T21:20:40.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.885+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1832533254 wait complete. 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.886+0000 7f59eb7ff700 1 Processor -- start 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.886+0000 7f59eb7ff700 1 -- start start 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 0x7f59e4196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4102080 0x7f59e41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419e710 con 0x7f59e4102a40 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419e850 con 0x7f59e4100ea0 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59eb7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419e9e0 con 0x7f59e4102080 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59e959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 0x7f59e4196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46394/0 (socket says 172.21.15.31:46394) 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59e9d9c700 1 -- 172.21.15.31:0/1872263504 learned_addr learned my addr 172.21.15.31:0/1872263504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.887+0000 7f59e8d9a700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4102080 0x7f59e41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 -- 172.21.15.31:0/1872263504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4102080 msgr2=0x7f59e41970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4102080 0x7f59e41970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 -- 172.21.15.31:0/1872263504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 msgr2=0x7f59e4196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 0x7f59e4196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 -- 172.21.15.31:0/1872263504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc009d00 con 0x7f59e4102a40 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e959b700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 0x7f59e4196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59e9d9c700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b070 tx=0x7f59dc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00e040 con 0x7f59e4102a40 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59dc003e90 con 0x7f59e4102a40 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e419ec60 con 0x7f59e4102a40 2023-10-06T21:20:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc004020 con 0x7f59e4102a40 2023-10-06T21:20:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.888+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e419f130 con 0x7f59e4102a40 2023-10-06T21:20:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.889+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e4107e00 con 0x7f59e4102a40 2023-10-06T21:20:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.891+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f59dc003a00 con 0x7f59e4102a40 2023-10-06T21:20:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.893+0000 7f59da7fc700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 0x7f59d00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.893+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f59dc0ce310 con 0x7f59e4102a40 2023-10-06T21:20:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.893+0000 7f59e959b700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 0x7f59d00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.893+0000 7f59e959b700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 0x7f59d00a39c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59e4101bf0 tx=0x7f59e000a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:40.896+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59dc097940 con 0x7f59e4102a40 2023-10-06T21:20:41.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.106+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59e402d020 con 0x7f59e4102a40 2023-10-06T21:20:41.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.107+0000 7f59da7fc700 1 -- 172.21.15.31:0/1872263504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59dc09af60 con 0x7f59e4102a40 2023-10-06T21:20:41.111 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:20:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.109+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 msgr2=0x7f59d00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.109+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 0x7f59d00a39c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59e4101bf0 tx=0x7f59e000a3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.109+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 msgr2=0x7f59e419e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.109+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b070 tx=0x7f59dc004c80 comp rx=0 tx=0).stop 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 shutdown_connections 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f59d00a1500 0x7f59d00a39c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59e4102a40 0x7f59e419e140 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e4100ea0 0x7f59e4196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 --2- 172.21.15.31:0/1872263504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59e4102080 0x7f59e41970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 >> 172.21.15.31:0/1872263504 conn(0x7f59e40fc460 msgr2=0x7f59e40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:41.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 shutdown_connections 2023-10-06T21:20:41.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.110+0000 7f59eb7ff700 1 -- 172.21.15.31:0/1872263504 wait complete. 2023-10-06T21:20:41.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:20:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:40 smithi031 ceph-mon[98539]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-06T21:20:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/960309213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:41.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:40 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1206673743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:40 smithi146 ceph-mon[102809]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-06T21:20:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:40 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/960309213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:41.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:40 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1206673743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:41.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.334+0000 7f90e07ff700 1 -- 172.21.15.31:0/1409942092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 msgr2=0x7f90d81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.334+0000 7f90e07ff700 1 --2- 172.21.15.31:0/1409942092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81024f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009990 tx=0x7f90c8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 -- 172.21.15.31:0/1409942092 shutdown_connections 2023-10-06T21:20:41.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 --2- 172.21.15.31:0/1409942092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 --2- 172.21.15.31:0/1409942092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 0x7f90d81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 --2- 172.21.15.31:0/1409942092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81024f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 -- 172.21.15.31:0/1409942092 >> 172.21.15.31:0/1409942092 conn(0x7f90d80fc430 msgr2=0x7f90d80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:41.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 -- 172.21.15.31:0/1409942092 shutdown_connections 2023-10-06T21:20:41.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.335+0000 7f90e07ff700 1 -- 172.21.15.31:0/1409942092 wait complete. 2023-10-06T21:20:41.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 Processor -- start 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 -- start start 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 0x7f90d8196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819e6f0 con 0x7f90d8102070 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819e830 con 0x7f90d8102a30 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90e07ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819e9c0 con 0x7f90d8100e90 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90ddd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90ddd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46408/0 (socket says 172.21.15.31:46408) 2023-10-06T21:20:41.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.336+0000 7f90ddd9a700 1 -- 172.21.15.31:0/2460900220 learned_addr learned my addr 172.21.15.31:0/2460900220 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ded9c700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 -- 172.21.15.31:0/2460900220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 msgr2=0x7f90d8196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90de59b700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 0x7f90d8196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 0x7f90d8196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 -- 172.21.15.31:0/2460900220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 msgr2=0x7f90d819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d819e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 -- 172.21.15.31:0/2460900220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80095f0 con 0x7f90d8102070 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ded9c700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90ddd9a700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004730 tx=0x7f90c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c801d070 con 0x7f90d8102070 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90c8004ca0 con 0x7f90d8102070 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d819ec40 con 0x7f90d8102070 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.337+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8003d90 con 0x7f90d8102070 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.338+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d819f0c0 con 0x7f90d8102070 2023-10-06T21:20:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.339+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d8107f80 con 0x7f90d8102070 2023-10-06T21:20:41.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.340+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f90c8004810 con 0x7f90d8102070 2023-10-06T21:20:41.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.342+0000 7f90cf7fe700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 0x7f90c40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.342+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f90c80cde80 con 0x7f90d8102070 2023-10-06T21:20:41.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.342+0000 7f90de59b700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 0x7f90c40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.343+0000 7f90de59b700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 0x7f90c40a39c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f90d4005950 tx=0x7f90d40058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:41.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.345+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f90c80974b0 con 0x7f90d8102070 2023-10-06T21:20:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:40 smithi142 ceph-mon[102951]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-06T21:20:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:40 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/960309213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:41.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:40 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1206673743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.556+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90d802d1a0 con 0x7f90d8102070 2023-10-06T21:20:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.556+0000 7f90cf7fe700 1 -- 172.21.15.31:0/2460900220 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90c809aad0 con 0x7f90d8102070 2023-10-06T21:20:41.560 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 msgr2=0x7f90c40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 0x7f90c40a39c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f90d4005950 tx=0x7f90d40058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 msgr2=0x7f90d81970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004730 tx=0x7f90c8005e80 comp rx=0 tx=0).stop 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 shutdown_connections 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f90c40a1500 0x7f90c40a39c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d8102a30 0x7f90d819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d8100e90 0x7f90d8196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 --2- 172.21.15.31:0/2460900220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90d8102070 0x7f90d81970b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 >> 172.21.15.31:0/2460900220 conn(0x7f90d80fc430 msgr2=0x7f90d80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 shutdown_connections 2023-10-06T21:20:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.559+0000 7f90e07ff700 1 -- 172.21.15.31:0/2460900220 wait complete. 2023-10-06T21:20:41.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 -- 172.21.15.31:0/1040309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 msgr2=0x7f4e90106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 --2- 172.21.15.31:0/1040309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90106660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009990 tx=0x7f4e84009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 -- 172.21.15.31:0/1040309754 shutdown_connections 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 --2- 172.21.15.31:0/1040309754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90106ba0 0x7f4e90109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 --2- 172.21.15.31:0/1040309754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90101940 0x7f4e90103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 --2- 172.21.15.31:0/1040309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90106660 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 -- 172.21.15.31:0/1040309754 >> 172.21.15.31:0/1040309754 conn(0x7f4e900fb330 msgr2=0x7f4e900fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 -- 172.21.15.31:0/1040309754 shutdown_connections 2023-10-06T21:20:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.784+0000 7f4e951ff700 1 -- 172.21.15.31:0/1040309754 wait complete. 2023-10-06T21:20:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.785+0000 7f4e951ff700 1 Processor -- start 2023-10-06T21:20:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.785+0000 7f4e951ff700 1 -- start start 2023-10-06T21:20:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 0x7f4e90196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90106ba0 0x7f4e9019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019e690 con 0x7f4e90104270 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019e7d0 con 0x7f4e90101940 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e951ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019e960 con 0x7f4e90106ba0 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 0x7f4e90196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46434/0 (socket says 172.21.15.31:46434) 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 -- 172.21.15.31:0/3408034633 learned_addr learned my addr 172.21.15.31:0/3408034633 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8f59e700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90106ba0 0x7f4e9019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 -- 172.21.15.31:0/3408034633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90106ba0 msgr2=0x7f4e9019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90106ba0 0x7f4e9019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 -- 172.21.15.31:0/3408034633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 msgr2=0x7f4e90196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 0x7f4e90196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.786+0000 7f4e8e59c700 1 -- 172.21.15.31:0/3408034633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e840095f0 con 0x7f4e90104270 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e8ed9d700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 0x7f4e90196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e8e59c700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005910 tx=0x7f4e84005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8401d070 con 0x7f4e90104270 2023-10-06T21:20:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8400be70 con 0x7f4e90104270 2023-10-06T21:20:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9019ebe0 con 0x7f4e90104270 2023-10-06T21:20:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84022e60 con 0x7f4e90104270 2023-10-06T21:20:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.787+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9019f000 con 0x7f4e90104270 2023-10-06T21:20:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.788+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e900fceb0 con 0x7f4e90104270 2023-10-06T21:20:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.790+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4e84022600 con 0x7f4e90104270 2023-10-06T21:20:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.792+0000 7f4e77fff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 0x7f4e780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.792+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f4e840ce620 con 0x7f4e90104270 2023-10-06T21:20:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.792+0000 7f4e8ed9d700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 0x7f4e780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.792+0000 7f4e8ed9d700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 0x7f4e780a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011490 tx=0x7f4e80009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:41.794+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e84097c50 con 0x7f4e90104270 2023-10-06T21:20:42.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.007+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e9002cf70 con 0x7f4e90104270 2023-10-06T21:20:42.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.007+0000 7f4e77fff700 1 -- 172.21.15.31:0/3408034633 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e8409b270 con 0x7f4e90104270 2023-10-06T21:20:42.011 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:42.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 msgr2=0x7f4e780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 0x7f4e780a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011490 tx=0x7f4e80009260 comp rx=0 tx=0).stop 2023-10-06T21:20:42.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 msgr2=0x7f4e90197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005910 tx=0x7f4e84005740 comp rx=0 tx=0).stop 2023-10-06T21:20:42.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 shutdown_connections 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e780a1480 0x7f4e780a3940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e90106ba0 0x7f4e9019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e90101940 0x7f4e90196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 --2- 172.21.15.31:0/3408034633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e90104270 0x7f4e90197050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 >> 172.21.15.31:0/3408034633 conn(0x7f4e900fb330 msgr2=0x7f4e901025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 shutdown_connections 2023-10-06T21:20:42.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.010+0000 7f4e951ff700 1 -- 172.21.15.31:0/3408034633 wait complete. 2023-10-06T21:20:42.031 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-06T21:20:42.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/2537895535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 msgr2=0x7f4b54109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/2537895535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b54109360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b0a0 tx=0x7f4b4c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/2537895535 shutdown_connections 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/2537895535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b54109360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/2537895535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54100ea0 0x7f4b541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/2537895535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54102080 0x7f4b54102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/2537895535 >> 172.21.15.31:0/2537895535 conn(0x7f4b540fc420 msgr2=0x7f4b540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/2537895535 shutdown_connections 2023-10-06T21:20:42.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.233+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/2537895535 wait complete. 2023-10-06T21:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.234+0000 7f4b5b3ff700 1 Processor -- start 2023-10-06T21:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.234+0000 7f4b5b3ff700 1 -- start start 2023-10-06T21:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 0x7f4b54196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 0x7f4b54197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419e760 con 0x7f4b54102a40 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419e8a0 con 0x7f4b54102080 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419ea30 con 0x7f4b54100ea0 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46450/0 (socket says 172.21.15.31:46450) 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 -- 172.21.15.31:0/186683760 learned_addr learned my addr 172.21.15.31:0/186683760 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5919b700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 0x7f4b54196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5899a700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 0x7f4b54197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 -- 172.21.15.31:0/186683760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 msgr2=0x7f4b54196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 0x7f4b54196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 -- 172.21.15.31:0/186683760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 msgr2=0x7f4b54197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 0x7f4b54197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5999c700 1 -- 172.21.15.31:0/186683760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009d00 con 0x7f4b54102a40 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5899a700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 0x7f4b54197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.235+0000 7f4b5919b700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 0x7f4b54196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b5999c700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00d010 tx=0x7f4b4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e040 con 0x7f4b54102a40 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b4c003e10 con 0x7f4b54102a40 2023-10-06T21:20:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5419ecb0 con 0x7f4b54102a40 2023-10-06T21:20:42.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c005310 con 0x7f4b54102a40 2023-10-06T21:20:42.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.236+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5419f0d0 con 0x7f4b54102a40 2023-10-06T21:20:42.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.237+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b541084a0 con 0x7f4b54102a40 2023-10-06T21:20:42.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.239+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4b4c01daa0 con 0x7f4b54102a40 2023-10-06T21:20:42.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.241+0000 7f4b4a7fc700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 0x7f4b400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.241+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f4b4c0ce5d0 con 0x7f4b54102a40 2023-10-06T21:20:42.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.241+0000 7f4b5919b700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 0x7f4b400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.241+0000 7f4b5919b700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 0x7f4b400a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b54101bf0 tx=0x7f4b50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:42.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.242+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4b4c097c00 con 0x7f4b54102a40 2023-10-06T21:20:42.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:41 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1872263504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:41 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2460900220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:41 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1872263504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:41 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2460900220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:41 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1872263504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:41 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2460900220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:42.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.497+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b5402d090 con 0x7f4b54102a40 2023-10-06T21:20:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.498+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/186683760 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f4b4c09b220 con 0x7f4b54102a40 2023-10-06T21:20:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:42.501 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 msgr2=0x7f4b400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 0x7f4b400a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b54101bf0 tx=0x7f4b50009260 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 msgr2=0x7f4b5419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00d010 tx=0x7f4b4c00bd70 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 shutdown_connections 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4b400a14d0 0x7f4b400a3990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102a40 0x7f4b5419e190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b54100ea0 0x7f4b54196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 --2- 172.21.15.31:0/186683760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b54102080 0x7f4b54197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 >> 172.21.15.31:0/186683760 conn(0x7f4b540fc420 msgr2=0x7f4b541055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 shutdown_connections 2023-10-06T21:20:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.500+0000 7f4b5b3ff700 1 -- 172.21.15.31:0/186683760 wait complete. 2023-10-06T21:20:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:42.521 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-06T21:20:42.522 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-06T21:20:42.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:20:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.721+0000 7fb5a7dff700 1 -- 172.21.15.31:0/710076577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0100ea0 msgr2=0x7fb5a01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.721+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/710076577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0100ea0 0x7fb5a01012c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009990 tx=0x7fb59c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 -- 172.21.15.31:0/710076577 shutdown_connections 2023-10-06T21:20:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/710076577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0102a40 0x7fb5a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/710076577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0100ea0 0x7fb5a01012c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/710076577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a0102080 0x7fb5a0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 -- 172.21.15.31:0/710076577 >> 172.21.15.31:0/710076577 conn(0x7fb5a00fc420 msgr2=0x7fb5a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:42.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 -- 172.21.15.31:0/710076577 shutdown_connections 2023-10-06T21:20:42.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.722+0000 7fb5a7dff700 1 -- 172.21.15.31:0/710076577 wait complete. 2023-10-06T21:20:42.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 Processor -- start 2023-10-06T21:20:42.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 -- start start 2023-10-06T21:20:42.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0100ea0 0x7fb5a0196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a0102a40 0x7fb5a019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019e630 con 0x7fb5a0102080 2023-10-06T21:20:42.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019e770 con 0x7fb5a0102a40 2023-10-06T21:20:42.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a7dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019e900 con 0x7fb5a0100ea0 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46470/0 (socket says 172.21.15.31:46470) 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.723+0000 7fb5a539a700 1 -- 172.21.15.31:0/1644099126 learned_addr learned my addr 172.21.15.31:0/1644099126 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a5b9b700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0100ea0 0x7fb5a0196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 -- 172.21.15.31:0/1644099126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0100ea0 msgr2=0x7fb5a0196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0100ea0 0x7fb5a0196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 -- 172.21.15.31:0/1644099126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a0102a40 msgr2=0x7fb5a019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a0102a40 0x7fb5a019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 -- 172.21.15.31:0/1644099126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0095f0 con 0x7fb5a0102080 2023-10-06T21:20:42.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a539a700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb59000d980 tx=0x7fb59000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:42.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000b950 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb590007620 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a019ebe0 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.724+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000f3c0 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.725+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a019f0f0 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.726+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a0108500 con 0x7fb5a0102080 2023-10-06T21:20:42.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.728+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb59000bab0 con 0x7fb5a0102080 2023-10-06T21:20:42.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.729+0000 7fb596ffd700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 0x7fb58c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:42.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.729+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fb5900ccf70 con 0x7fb5a0102080 2023-10-06T21:20:42.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.729+0000 7fb5a5b9b700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 0x7fb58c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:42.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.730+0000 7fb5a5b9b700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 0x7fb58c0a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b5c0 tx=0x7fb59c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:42.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.733+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb590096510 con 0x7fb5a0102080 2023-10-06T21:20:42.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.943+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5a002d2b0 con 0x7fb58c0a14d0 2023-10-06T21:20:42.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.957+0000 7fb596ffd700 1 -- 172.21.15.31:0/1644099126 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343375 (secure 0 0 0) 0x7fb5a002d2b0 con 0x7fb58c0a14d0 2023-10-06T21:20:42.964 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 msgr2=0x7fb58c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 0x7fb58c0a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b5c0 tx=0x7fb59c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 msgr2=0x7fb5a0196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb59000d980 tx=0x7fb59000dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 shutdown_connections 2023-10-06T21:20:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fb58c0a14d0 0x7fb58c0a3990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a0102a40 0x7fb5a019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5a0100ea0 0x7fb5a0196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 --2- 172.21.15.31:0/1644099126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a0102080 0x7fb5a0196ff0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 >> 172.21.15.31:0/1644099126 conn(0x7fb5a00fc420 msgr2=0x7fb5a0105640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 shutdown_connections 2023-10-06T21:20:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:42.962+0000 7fb5a7dff700 1 -- 172.21.15.31:0/1644099126 wait complete. 2023-10-06T21:20:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:20:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2023-10-06T21:20:41.836505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659292,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1181,"num_read_kb":717,"num_write":432,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336300,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605524,"statfs":{"total":1151940427776,"available":1151596056576,"internally_reserved":0,"allocated":12357632,"data_stored":7697490,"data_compressed":251907,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2495,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002684"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908715+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:20:25.908715+0000","last_peered":"2023-10-06T21:20:25.908715+0000","last_clean":"2023-10-06T21:20:25.908715+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:20:25.908715+0000","last_undegraded":"2023-10-06T21:20:25.908715+0000","last_fullsized":"2023-10-06T21:20:25.908715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502125+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:20:26.502125+0000","last_peered":"2023-10-06T21:20:26.502125+0000","last_clean":"2023-10-06T21:20:26.502125+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:20:26.502125+0000","last_undegraded":"2023-10-06T21:20:26.502125+0000","last_fullsized":"2023-10-06T21:20:26.502125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502218+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:20:26.502218+0000","last_peered":"2023-10-06T21:20:26.502218+0000","last_clean":"2023-10-06T21:20:26.502218+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:20:26.502218+0000","last_undegraded":"2023-10-06T21:20:26.502218+0000","last_fullsized":"2023-10-06T21:20:26.502218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889201+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:20:25.889201+0000","last_peered":"2023-10-06T21:20:25.889201+0000","last_clean":"2023-10-06T21:20:25.889201+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:20:25.889201+0000","last_undegraded":"2023-10-06T21:20:25.889201+0000","last_fullsized":"2023-10-06T21:20:25.889201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898528+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:20:25.898528+0000","last_peered":"2023-10-06T21:20:25.898528+0000","last_clean":"2023-10-06T21:20:25.898528+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:20:25.898528+0000","last_undegraded":"2023-10-06T21:20:25.898528+0000","last_fullsized":"2023-10-06T21:20:25.898528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908557+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:20:25.908557+0000","last_peered":"2023-10-06T21:20:25.908557+0000","last_clean":"2023-10-06T21:20:25.908557+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:20:25.908557+0000","last_undegraded":"2023-10-06T21:20:25.908557+0000","last_fullsized":"2023-10-06T21:20:25.908557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890967+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:20:25.890967+0000","last_peered":"2023-10-06T21:20:25.890967+0000","last_clean":"2023-10-06T21:20:25.890967+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:20:25.890967+0000","last_undegraded":"2023-10-06T21:20:25.890967+0000","last_fullsized":"2023-10-06T21:20:25.890967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894114+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:20:25.894114+0000","last_peered":"2023-10-06T21:20:25.894114+0000","last_clean":"2023-10-06T21:20:25.894114+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:20:25.894114+0000","last_undegraded":"2023-10-06T21:20:25.894114+0000","last_fullsized":"2023-10-06T21:20:25.894114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894063+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:20:25.894063+0000","last_peered":"2023-10-06T21:20:25.894063+0000","last_clean":"2023-10-06T21:20:25.894063+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:20:25.894063+0000","last_undegraded":"2023-10-06T21:20:25.894063+0000","last_fullsized":"2023-10-06T21:20:25.894063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908894+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:20:25.908894+0000","last_peered":"2023-10-06T21:20:25.908894+0000","last_clean":"2023-10-06T21:20:25.908894+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:20:25.908894+0000","last_undegraded":"2023-10-06T21:20:25.908894+0000","last_fullsized":"2023-10-06T21:20:25.908894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891256+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:20:25.891256+0000","last_peered":"2023-10-06T21:20:25.891256+0000","last_clean":"2023-10-06T21:20:25.891256+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:20:25.891256+0000","last_undegraded":"2023-10-06T21:20:25.891256+0000","last_fullsized":"2023-10-06T21:20:25.891256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889121+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:20:25.889121+0000","last_peered":"2023-10-06T21:20:25.889121+0000","last_clean":"2023-10-06T21:20:25.889121+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:20:25.889121+0000","last_undegraded":"2023-10-06T21:20:25.889121+0000","last_fullsized":"2023-10-06T21:20:25.889121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893926+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:20:25.893926+0000","last_peered":"2023-10-06T21:20:25.893926+0000","last_clean":"2023-10-06T21:20:25.893926+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:20:25.893926+0000","last_undegraded":"2023-10-06T21:20:25.893926+0000","last_fullsized":"2023-10-06T21:20:25.893926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501878+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:20:26.501878+0000","last_peered":"2023-10-06T21:20:26.501878+0000","last_clean":"2023-10-06T21:20:26.501878+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:20:26.501878+0000","last_undegraded":"2023-10-06T21:20:26.501878+0000","last_fullsized":"2023-10-06T21:20:26.501878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501915+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:20:26.501915+0000","last_peered":"2023-10-06T21:20:26.501915+0000","last_clean":"2023-10-06T21:20:26.501915+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:20:26.501915+0000","last_undegraded":"2023-10-06T21:20:26.501915+0000","last_fullsized":"2023-10-06T21:20:26.501915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893547+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:20:25.893547+0000","last_peered":"2023-10-06T21:20:25.893547+0000","last_clean":"2023-10-06T21:20:25.893547+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:20:25.893547+0000","last_undegraded":"2023-10-06T21:20:25.893547+0000","last_fullsized":"2023-10-06T21:20:25.893547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908765+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:20:25.908765+0000","last_peered":"2023-10-06T21:20:25.908765+0000","last_clean":"2023-10-06T21:20:25.908765+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:20:25.908765+0000","last_undegraded":"2023-10-06T21:20:25.908765+0000","last_fullsized":"2023-10-06T21:20:25.908765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889141+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:20:25.889141+0000","last_peered":"2023-10-06T21:20:25.889141+0000","last_clean":"2023-10-06T21:20:25.889141+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:20:25.889141+0000","last_undegraded":"2023-10-06T21:20:25.889141+0000","last_fullsized":"2023-10-06T21:20:25.889141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889176+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:20:25.889176+0000","last_peered":"2023-10-06T21:20:25.889176+0000","last_clean":"2023-10-06T21:20:25.889176+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:20:25.889176+0000","last_undegraded":"2023-10-06T21:20:25.889176+0000","last_fullsized":"2023-10-06T21:20:25.889176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:36.565843+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:20:36.565843+0000","last_peered":"2023-10-06T21:20:36.565843+0000","last_clean":"2023-10-06T21:20:36.565843+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:20:36.565843+0000","last_undegraded":"2023-10-06T21:20:36.565843+0000","last_fullsized":"2023-10-06T21:20:36.565843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894964+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:20:25.894964+0000","last_peered":"2023-10-06T21:20:25.894964+0000","last_clean":"2023-10-06T21:20:25.894964+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:20:25.894964+0000","last_undegraded":"2023-10-06T21:20:25.894964+0000","last_fullsized":"2023-10-06T21:20:25.894964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895174+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:20:25.895174+0000","last_peered":"2023-10-06T21:20:25.895174+0000","last_clean":"2023-10-06T21:20:25.895174+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:20:25.895174+0000","last_undegraded":"2023-10-06T21:20:25.895174+0000","last_fullsized":"2023-10-06T21:20:25.895174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502253+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:20:26.502253+0000","last_peered":"2023-10-06T21:20:26.502253+0000","last_clean":"2023-10-06T21:20:26.502253+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:20:26.502253+0000","last_undegraded":"2023-10-06T21:20:26.502253+0000","last_fullsized":"2023-10-06T21:20:26.502253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898484+0000","last_change":"2023-10-06T21:19:26.863512+0000","last_active":"2023-10-06T21:20:25.898484+0000","last_peered":"2023-10-06T21:20:25.898484+0000","last_clean":"2023-10-06T21:20:25.898484+0000","last_became_active":"2023-10-06T21:19:26.845274+0000","last_became_peered":"2023-10-06T21:19:26.845274+0000","last_unstale":"2023-10-06T21:20:25.898484+0000","last_undegraded":"2023-10-06T21:20:25.898484+0000","last_fullsized":"2023-10-06T21:20:25.898484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:16.751914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893672+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:20:25.893672+0000","last_peered":"2023-10-06T21:20:25.893672+0000","last_clean":"2023-10-06T21:20:25.893672+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:20:25.893672+0000","last_undegraded":"2023-10-06T21:20:25.893672+0000","last_fullsized":"2023-10-06T21:20:25.893672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898508+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:20:25.898508+0000","last_peered":"2023-10-06T21:20:25.898508+0000","last_clean":"2023-10-06T21:20:25.898508+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:20:25.898508+0000","last_undegraded":"2023-10-06T21:20:25.898508+0000","last_fullsized":"2023-10-06T21:20:25.898508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897746+0000","last_change":"2023-10-06T21:19:26.991142+0000","last_active":"2023-10-06T21:20:25.897746+0000","last_peered":"2023-10-06T21:20:25.897746+0000","last_clean":"2023-10-06T21:20:25.897746+0000","last_became_active":"2023-10-06T21:19:26.990765+0000","last_became_peered":"2023-10-06T21:19:26.990765+0000","last_unstale":"2023-10-06T21:20:25.897746+0000","last_undegraded":"2023-10-06T21:20:25.897746+0000","last_fullsized":"2023-10-06T21:20:25.897746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:35:44.692174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:39.987358+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:20:39.987358+0000","last_peered":"2023-10-06T21:20:39.987358+0000","last_clean":"2023-10-06T21:20:39.987358+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:20:39.987358+0000","last_undegraded":"2023-10-06T21:20:39.987358+0000","last_fullsized":"2023-10-06T21:20:39.987358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1121,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897701+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:20:25.897701+0000","last_peered":"2023-10-06T21:20:25.897701+0000","last_clean":"2023-10-06T21:20:25.897701+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:20:25.897701+0000","last_undegraded":"2023-10-06T21:20:25.897701+0000","last_fullsized":"2023-10-06T21:20:25.897701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908501+0000","last_change":"2023-10-06T21:19:26.998407+0000","last_active":"2023-10-06T21:20:25.908501+0000","last_peered":"2023-10-06T21:20:25.908501+0000","last_clean":"2023-10-06T21:20:25.908501+0000","last_became_active":"2023-10-06T21:19:26.991918+0000","last_became_peered":"2023-10-06T21:19:26.991918+0000","last_unstale":"2023-10-06T21:20:25.908501+0000","last_undegraded":"2023-10-06T21:20:25.908501+0000","last_fullsized":"2023-10-06T21:20:25.908501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:05.903796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899375+0000","last_change":"2023-10-06T21:20:26.898661+0000","last_active":"2023-10-06T21:20:26.899375+0000","last_peered":"2023-10-06T21:20:26.899375+0000","last_clean":"2023-10-06T21:20:26.899375+0000","last_became_active":"2023-10-06T21:20:26.898216+0000","last_became_peered":"2023-10-06T21:20:26.898216+0000","last_unstale":"2023-10-06T21:20:26.899375+0000","last_undegraded":"2023-10-06T21:20:26.899375+0000","last_fullsized":"2023-10-06T21:20:26.899375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:55:14.473221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893480+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:20:25.893480+0000","last_peered":"2023-10-06T21:20:25.893480+0000","last_clean":"2023-10-06T21:20:25.893480+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:20:25.893480+0000","last_undegraded":"2023-10-06T21:20:25.893480+0000","last_fullsized":"2023-10-06T21:20:25.893480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894320+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:20:25.894320+0000","last_peered":"2023-10-06T21:20:25.894320+0000","last_clean":"2023-10-06T21:20:25.894320+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:20:25.894320+0000","last_undegraded":"2023-10-06T21:20:25.894320+0000","last_fullsized":"2023-10-06T21:20:25.894320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897315+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:20:25.897315+0000","last_peered":"2023-10-06T21:20:25.897315+0000","last_clean":"2023-10-06T21:20:25.897315+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:20:25.897315+0000","last_undegraded":"2023-10-06T21:20:25.897315+0000","last_fullsized":"2023-10-06T21:20:25.897315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888820+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:20:25.888820+0000","last_peered":"2023-10-06T21:20:25.888820+0000","last_clean":"2023-10-06T21:20:25.888820+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:20:25.888820+0000","last_undegraded":"2023-10-06T21:20:25.888820+0000","last_fullsized":"2023-10-06T21:20:25.888820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":226,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988250+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:20:40.988250+0000","last_peered":"2023-10-06T21:20:40.988250+0000","last_clean":"2023-10-06T21:20:40.988250+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:20:40.988250+0000","last_undegraded":"2023-10-06T21:20:40.988250+0000","last_fullsized":"2023-10-06T21:20:40.988250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":203,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894451+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:20:25.894451+0000","last_peered":"2023-10-06T21:20:25.894451+0000","last_clean":"2023-10-06T21:20:25.894451+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:20:25.894451+0000","last_undegraded":"2023-10-06T21:20:25.894451+0000","last_fullsized":"2023-10-06T21:20:25.894451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891255+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:20:25.891255+0000","last_peered":"2023-10-06T21:20:25.891255+0000","last_clean":"2023-10-06T21:20:25.891255+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:20:25.891255+0000","last_undegraded":"2023-10-06T21:20:25.891255+0000","last_fullsized":"2023-10-06T21:20:25.891255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502189+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:20:26.502189+0000","last_peered":"2023-10-06T21:20:26.502189+0000","last_clean":"2023-10-06T21:20:26.502189+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:20:26.502189+0000","last_undegraded":"2023-10-06T21:20:26.502189+0000","last_fullsized":"2023-10-06T21:20:26.502189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897849+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:20:25.897849+0000","last_peered":"2023-10-06T21:20:25.897849+0000","last_clean":"2023-10-06T21:20:25.897849+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:20:25.897849+0000","last_undegraded":"2023-10-06T21:20:25.897849+0000","last_fullsized":"2023-10-06T21:20:25.897849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893778+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:20:25.893778+0000","last_peered":"2023-10-06T21:20:25.893778+0000","last_clean":"2023-10-06T21:20:25.893778+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:20:25.893778+0000","last_undegraded":"2023-10-06T21:20:25.893778+0000","last_fullsized":"2023-10-06T21:20:25.893778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988126+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:20:40.988126+0000","last_peered":"2023-10-06T21:20:40.988126+0000","last_clean":"2023-10-06T21:20:40.988126+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:20:40.988126+0000","last_undegraded":"2023-10-06T21:20:40.988126+0000","last_fullsized":"2023-10-06T21:20:40.988126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59729,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":37,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898054+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:20:25.898054+0000","last_peered":"2023-10-06T21:20:25.898054+0000","last_clean":"2023-10-06T21:20:25.898054+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:20:25.898054+0000","last_undegraded":"2023-10-06T21:20:25.898054+0000","last_fullsized":"2023-10-06T21:20:25.898054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889077+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:20:25.889077+0000","last_peered":"2023-10-06T21:20:25.889077+0000","last_clean":"2023-10-06T21:20:25.889077+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:20:25.889077+0000","last_undegraded":"2023-10-06T21:20:25.889077+0000","last_fullsized":"2023-10-06T21:20:25.889077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893449+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:20:25.893449+0000","last_peered":"2023-10-06T21:20:25.893449+0000","last_clean":"2023-10-06T21:20:25.893449+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:20:25.893449+0000","last_undegraded":"2023-10-06T21:20:25.893449+0000","last_fullsized":"2023-10-06T21:20:25.893449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897611+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:20:25.897611+0000","last_peered":"2023-10-06T21:20:25.897611+0000","last_clean":"2023-10-06T21:20:25.897611+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:20:25.897611+0000","last_undegraded":"2023-10-06T21:20:25.897611+0000","last_fullsized":"2023-10-06T21:20:25.897611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.896400+0000","last_change":"2023-10-06T21:20:26.896400+0000","last_active":"2023-10-06T21:20:26.896400+0000","last_peered":"2023-10-06T21:20:26.896400+0000","last_clean":"2023-10-06T21:20:26.896400+0000","last_became_active":"2023-10-06T21:20:26.896036+0000","last_became_peered":"2023-10-06T21:20:26.896036+0000","last_unstale":"2023-10-06T21:20:26.896400+0000","last_undegraded":"2023-10-06T21:20:26.896400+0000","last_fullsized":"2023-10-06T21:20:26.896400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:30:04.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.030597+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:20:40.030597+0000","last_peered":"2023-10-06T21:20:40.030597+0000","last_clean":"2023-10-06T21:20:40.030597+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:20:40.030597+0000","last_undegraded":"2023-10-06T21:20:40.030597+0000","last_fullsized":"2023-10-06T21:20:40.030597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891007+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:20:25.891007+0000","last_peered":"2023-10-06T21:20:25.891007+0000","last_clean":"2023-10-06T21:20:25.891007+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:20:25.891007+0000","last_undegraded":"2023-10-06T21:20:25.891007+0000","last_fullsized":"2023-10-06T21:20:25.891007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908465+0000","last_change":"2023-10-06T21:19:26.862156+0000","last_active":"2023-10-06T21:20:25.908465+0000","last_peered":"2023-10-06T21:20:25.908465+0000","last_clean":"2023-10-06T21:20:25.908465+0000","last_became_active":"2023-10-06T21:19:26.844987+0000","last_became_peered":"2023-10-06T21:19:26.844987+0000","last_unstale":"2023-10-06T21:20:25.908465+0000","last_undegraded":"2023-10-06T21:20:25.908465+0000","last_fullsized":"2023-10-06T21:20:25.908465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:12:56.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897320+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:20:25.897320+0000","last_peered":"2023-10-06T21:20:25.897320+0000","last_clean":"2023-10-06T21:20:25.897320+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:20:25.897320+0000","last_undegraded":"2023-10-06T21:20:25.897320+0000","last_fullsized":"2023-10-06T21:20:25.897320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897952+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:20:25.897952+0000","last_peered":"2023-10-06T21:20:25.897952+0000","last_clean":"2023-10-06T21:20:25.897952+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:20:25.897952+0000","last_undegraded":"2023-10-06T21:20:25.897952+0000","last_fullsized":"2023-10-06T21:20:25.897952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908587+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:20:25.908587+0000","last_peered":"2023-10-06T21:20:25.908587+0000","last_clean":"2023-10-06T21:20:25.908587+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:20:25.908587+0000","last_undegraded":"2023-10-06T21:20:25.908587+0000","last_fullsized":"2023-10-06T21:20:25.908587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889118+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:20:25.889118+0000","last_peered":"2023-10-06T21:20:25.889118+0000","last_clean":"2023-10-06T21:20:25.889118+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:20:25.889118+0000","last_undegraded":"2023-10-06T21:20:25.889118+0000","last_fullsized":"2023-10-06T21:20:25.889118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908797+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:20:25.908797+0000","last_peered":"2023-10-06T21:20:25.908797+0000","last_clean":"2023-10-06T21:20:25.908797+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:20:25.908797+0000","last_undegraded":"2023-10-06T21:20:25.908797+0000","last_fullsized":"2023-10-06T21:20:25.908797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.006783+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:20:37.006783+0000","last_peered":"2023-10-06T21:20:37.006783+0000","last_clean":"2023-10-06T21:20:37.006783+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:20:37.006783+0000","last_undegraded":"2023-10-06T21:20:37.006783+0000","last_fullsized":"2023-10-06T21:20:37.006783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908595+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:20:25.908595+0000","last_peered":"2023-10-06T21:20:25.908595+0000","last_clean":"2023-10-06T21:20:25.908595+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:20:25.908595+0000","last_undegraded":"2023-10-06T21:20:25.908595+0000","last_fullsized":"2023-10-06T21:20:25.908595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897427+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:20:25.897427+0000","last_peered":"2023-10-06T21:20:25.897427+0000","last_clean":"2023-10-06T21:20:25.897427+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:20:25.897427+0000","last_undegraded":"2023-10-06T21:20:25.897427+0000","last_fullsized":"2023-10-06T21:20:25.897427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.023875+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:20:37.023875+0000","last_peered":"2023-10-06T21:20:37.023875+0000","last_clean":"2023-10-06T21:20:37.023875+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:20:37.023875+0000","last_undegraded":"2023-10-06T21:20:37.023875+0000","last_fullsized":"2023-10-06T21:20:37.023875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894894+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:20:25.894894+0000","last_peered":"2023-10-06T21:20:25.894894+0000","last_clean":"2023-10-06T21:20:25.894894+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:20:25.894894+0000","last_undegraded":"2023-10-06T21:20:25.894894+0000","last_fullsized":"2023-10-06T21:20:25.894894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889049+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:20:25.889049+0000","last_peered":"2023-10-06T21:20:25.889049+0000","last_clean":"2023-10-06T21:20:25.889049+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:20:25.889049+0000","last_undegraded":"2023-10-06T21:20:25.889049+0000","last_fullsized":"2023-10-06T21:20:25.889049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":323,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891086+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:20:25.891086+0000","last_peered":"2023-10-06T21:20:25.891086+0000","last_clean":"2023-10-06T21:20:25.891086+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:20:25.891086+0000","last_undegraded":"2023-10-06T21:20:25.891086+0000","last_fullsized":"2023-10-06T21:20:25.891086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893217+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:20:25.893217+0000","last_peered":"2023-10-06T21:20:25.893217+0000","last_clean":"2023-10-06T21:20:25.893217+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:20:25.893217+0000","last_undegraded":"2023-10-06T21:20:25.893217+0000","last_fullsized":"2023-10-06T21:20:25.893217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908323+0000","last_change":"2023-10-06T21:19:26.864852+0000","last_active":"2023-10-06T21:20:25.908323+0000","last_peered":"2023-10-06T21:20:25.908323+0000","last_clean":"2023-10-06T21:20:25.908323+0000","last_became_active":"2023-10-06T21:19:26.846795+0000","last_became_peered":"2023-10-06T21:19:26.846795+0000","last_unstale":"2023-10-06T21:20:25.908323+0000","last_undegraded":"2023-10-06T21:20:25.908323+0000","last_fullsized":"2023-10-06T21:20:25.908323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:15:22.977641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898390+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:20:25.898390+0000","last_peered":"2023-10-06T21:20:25.898390+0000","last_clean":"2023-10-06T21:20:25.898390+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:20:25.898390+0000","last_undegraded":"2023-10-06T21:20:25.898390+0000","last_fullsized":"2023-10-06T21:20:25.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893558+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:20:25.893558+0000","last_peered":"2023-10-06T21:20:25.893558+0000","last_clean":"2023-10-06T21:20:25.893558+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:20:25.893558+0000","last_undegraded":"2023-10-06T21:20:25.893558+0000","last_fullsized":"2023-10-06T21:20:25.893558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.893269+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:20:26.893269+0000","last_peered":"2023-10-06T21:20:26.893269+0000","last_clean":"2023-10-06T21:20:26.893269+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:20:26.893269+0000","last_undegraded":"2023-10-06T21:20:26.893269+0000","last_fullsized":"2023-10-06T21:20:26.893269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895118+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:20:25.895118+0000","last_peered":"2023-10-06T21:20:25.895118+0000","last_clean":"2023-10-06T21:20:25.895118+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:20:25.895118+0000","last_undegraded":"2023-10-06T21:20:25.895118+0000","last_fullsized":"2023-10-06T21:20:25.895118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889178+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:20:25.889178+0000","last_peered":"2023-10-06T21:20:25.889178+0000","last_clean":"2023-10-06T21:20:25.889178+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:20:25.889178+0000","last_undegraded":"2023-10-06T21:20:25.889178+0000","last_fullsized":"2023-10-06T21:20:25.889178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502163+0000","last_change":"2023-10-06T21:19:26.841631+0000","last_active":"2023-10-06T21:20:26.502163+0000","last_peered":"2023-10-06T21:20:26.502163+0000","last_clean":"2023-10-06T21:20:26.502163+0000","last_became_active":"2023-10-06T21:19:26.841352+0000","last_became_peered":"2023-10-06T21:19:26.841352+0000","last_unstale":"2023-10-06T21:20:26.502163+0000","last_undegraded":"2023-10-06T21:20:26.502163+0000","last_fullsized":"2023-10-06T21:20:26.502163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:26:07.482368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893584+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:20:25.893584+0000","last_peered":"2023-10-06T21:20:25.893584+0000","last_clean":"2023-10-06T21:20:25.893584+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:20:25.893584+0000","last_undegraded":"2023-10-06T21:20:25.893584+0000","last_fullsized":"2023-10-06T21:20:25.893584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897765+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:20:25.897765+0000","last_peered":"2023-10-06T21:20:25.897765+0000","last_clean":"2023-10-06T21:20:25.897765+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:20:25.897765+0000","last_undegraded":"2023-10-06T21:20:25.897765+0000","last_fullsized":"2023-10-06T21:20:25.897765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894378+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:20:25.894378+0000","last_peered":"2023-10-06T21:20:25.894378+0000","last_clean":"2023-10-06T21:20:25.894378+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:20:25.894378+0000","last_undegraded":"2023-10-06T21:20:25.894378+0000","last_fullsized":"2023-10-06T21:20:25.894378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908495+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:20:25.908495+0000","last_peered":"2023-10-06T21:20:25.908495+0000","last_clean":"2023-10-06T21:20:25.908495+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:20:25.908495+0000","last_undegraded":"2023-10-06T21:20:25.908495+0000","last_fullsized":"2023-10-06T21:20:25.908495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889012+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:20:25.889012+0000","last_peered":"2023-10-06T21:20:25.889012+0000","last_clean":"2023-10-06T21:20:25.889012+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:20:25.889012+0000","last_undegraded":"2023-10-06T21:20:25.889012+0000","last_fullsized":"2023-10-06T21:20:25.889012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894653+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:20:25.894653+0000","last_peered":"2023-10-06T21:20:25.894653+0000","last_clean":"2023-10-06T21:20:25.894653+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:20:25.894653+0000","last_undegraded":"2023-10-06T21:20:25.894653+0000","last_fullsized":"2023-10-06T21:20:25.894653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898404+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:20:25.898404+0000","last_peered":"2023-10-06T21:20:25.898404+0000","last_clean":"2023-10-06T21:20:25.898404+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:20:25.898404+0000","last_undegraded":"2023-10-06T21:20:25.898404+0000","last_fullsized":"2023-10-06T21:20:25.898404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891356+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:20:25.891356+0000","last_peered":"2023-10-06T21:20:25.891356+0000","last_clean":"2023-10-06T21:20:25.891356+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:20:25.891356+0000","last_undegraded":"2023-10-06T21:20:25.891356+0000","last_fullsized":"2023-10-06T21:20:25.891356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.020044+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:20:37.020044+0000","last_peered":"2023-10-06T21:20:37.020044+0000","last_clean":"2023-10-06T21:20:37.020044+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:20:37.020044+0000","last_undegraded":"2023-10-06T21:20:37.020044+0000","last_fullsized":"2023-10-06T21:20:37.020044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1669,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891410+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:20:25.891410+0000","last_peered":"2023-10-06T21:20:25.891410+0000","last_clean":"2023-10-06T21:20:25.891410+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:20:25.891410+0000","last_undegraded":"2023-10-06T21:20:25.891410+0000","last_fullsized":"2023-10-06T21:20:25.891410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897736+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:20:25.897736+0000","last_peered":"2023-10-06T21:20:25.897736+0000","last_clean":"2023-10-06T21:20:25.897736+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:20:25.897736+0000","last_undegraded":"2023-10-06T21:20:25.897736+0000","last_fullsized":"2023-10-06T21:20:25.897736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891052+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:20:25.891052+0000","last_peered":"2023-10-06T21:20:25.891052+0000","last_clean":"2023-10-06T21:20:25.891052+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:20:25.891052+0000","last_undegraded":"2023-10-06T21:20:25.891052+0000","last_fullsized":"2023-10-06T21:20:25.891052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893514+0000","last_change":"2023-10-06T21:19:26.851061+0000","last_active":"2023-10-06T21:20:25.893514+0000","last_peered":"2023-10-06T21:20:25.893514+0000","last_clean":"2023-10-06T21:20:25.893514+0000","last_became_active":"2023-10-06T21:19:26.845797+0000","last_became_peered":"2023-10-06T21:19:26.845797+0000","last_unstale":"2023-10-06T21:20:25.893514+0000","last_undegraded":"2023-10-06T21:20:25.893514+0000","last_fullsized":"2023-10-06T21:20:25.893514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:48.212793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891426+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:20:25.891426+0000","last_peered":"2023-10-06T21:20:25.891426+0000","last_clean":"2023-10-06T21:20:25.891426+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:20:25.891426+0000","last_undegraded":"2023-10-06T21:20:25.891426+0000","last_fullsized":"2023-10-06T21:20:25.891426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908593+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:20:25.908593+0000","last_peered":"2023-10-06T21:20:25.908593+0000","last_clean":"2023-10-06T21:20:25.908593+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:20:25.908593+0000","last_undegraded":"2023-10-06T21:20:25.908593+0000","last_fullsized":"2023-10-06T21:20:25.908593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888944+0000","last_change":"2023-10-06T21:19:26.844504+0000","last_active":"2023-10-06T21:20:25.888944+0000","last_peered":"2023-10-06T21:20:25.888944+0000","last_clean":"2023-10-06T21:20:25.888944+0000","last_became_active":"2023-10-06T21:19:26.844105+0000","last_became_peered":"2023-10-06T21:19:26.844105+0000","last_unstale":"2023-10-06T21:20:25.888944+0000","last_undegraded":"2023-10-06T21:20:25.888944+0000","last_fullsized":"2023-10-06T21:20:25.888944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:08.655868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888876+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:20:25.888876+0000","last_peered":"2023-10-06T21:20:25.888876+0000","last_clean":"2023-10-06T21:20:25.888876+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:20:25.888876+0000","last_undegraded":"2023-10-06T21:20:25.888876+0000","last_fullsized":"2023-10-06T21:20:25.888876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502036+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:20:26.502036+0000","last_peered":"2023-10-06T21:20:26.502036+0000","last_clean":"2023-10-06T21:20:26.502036+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:20:26.502036+0000","last_undegraded":"2023-10-06T21:20:26.502036+0000","last_fullsized":"2023-10-06T21:20:26.502036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893458+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:20:25.893458+0000","last_peered":"2023-10-06T21:20:25.893458+0000","last_clean":"2023-10-06T21:20:25.893458+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:20:25.893458+0000","last_undegraded":"2023-10-06T21:20:25.893458+0000","last_fullsized":"2023-10-06T21:20:25.893458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908546+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:20:25.908546+0000","last_peered":"2023-10-06T21:20:25.908546+0000","last_clean":"2023-10-06T21:20:25.908546+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:20:25.908546+0000","last_undegraded":"2023-10-06T21:20:25.908546+0000","last_fullsized":"2023-10-06T21:20:25.908546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889295+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:20:25.889295+0000","last_peered":"2023-10-06T21:20:25.889295+0000","last_clean":"2023-10-06T21:20:25.889295+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:20:25.889295+0000","last_undegraded":"2023-10-06T21:20:25.889295+0000","last_fullsized":"2023-10-06T21:20:25.889295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895021+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:20:25.895021+0000","last_peered":"2023-10-06T21:20:25.895021+0000","last_clean":"2023-10-06T21:20:25.895021+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:20:25.895021+0000","last_undegraded":"2023-10-06T21:20:25.895021+0000","last_fullsized":"2023-10-06T21:20:25.895021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890975+0000","last_change":"2023-10-06T21:19:26.847714+0000","last_active":"2023-10-06T21:20:25.890975+0000","last_peered":"2023-10-06T21:20:25.890975+0000","last_clean":"2023-10-06T21:20:25.890975+0000","last_became_active":"2023-10-06T21:19:26.847302+0000","last_became_peered":"2023-10-06T21:19:26.847302+0000","last_unstale":"2023-10-06T21:20:25.890975+0000","last_undegraded":"2023-10-06T21:20:25.890975+0000","last_fullsized":"2023-10-06T21:20:25.890975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:10:24.079203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894276+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:20:25.894276+0000","last_peered":"2023-10-06T21:20:25.894276+0000","last_clean":"2023-10-06T21:20:25.894276+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:20:25.894276+0000","last_undegraded":"2023-10-06T21:20:25.894276+0000","last_fullsized":"2023-10-06T21:20:25.894276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899339+0000","last_change":"2023-10-06T21:20:26.897996+0000","last_active":"2023-10-06T21:20:26.899339+0000","last_peered":"2023-10-06T21:20:26.899339+0000","last_clean":"2023-10-06T21:20:26.899339+0000","last_became_active":"2023-10-06T21:20:26.897555+0000","last_became_peered":"2023-10-06T21:20:26.897555+0000","last_unstale":"2023-10-06T21:20:26.899339+0000","last_undegraded":"2023-10-06T21:20:26.899339+0000","last_fullsized":"2023-10-06T21:20:26.899339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:59:28.968861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893964+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:20:25.893964+0000","last_peered":"2023-10-06T21:20:25.893964+0000","last_clean":"2023-10-06T21:20:25.893964+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:20:25.893964+0000","last_undegraded":"2023-10-06T21:20:25.893964+0000","last_fullsized":"2023-10-06T21:20:25.893964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908609+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:20:25.908609+0000","last_peered":"2023-10-06T21:20:25.908609+0000","last_clean":"2023-10-06T21:20:25.908609+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:20:25.908609+0000","last_undegraded":"2023-10-06T21:20:25.908609+0000","last_fullsized":"2023-10-06T21:20:25.908609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68678,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":598,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206034,"data_compressed":5511,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478543,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3819999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.59699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.038,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.087},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.59799999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93999999999999995,"5min":1.536,"15min":1.536},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.016,"5min":1.23,"15min":1.23},"last":1.052},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.083,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.022}]},{"osd":5,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73199999999999998,"15min":0.747},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86199999999999999,"5min":2.036,"15min":2.036},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85799999999999998},"min":{"1min":0.68600000000000005,"5min":0.626,"15min":0.626},"max":{"1min":1.0600000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":1.631,"15min":1.631},"last":0.88500000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.034},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.143}]},{"osd":9,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.71099999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2949999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.65300000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.302,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.1679999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93600000000000005,"15min":0.93999999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.165,"5min":1.671,"15min":1.671},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.96799999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2749999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0760000000000001}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1129395,"data_compressed":49169,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.177,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.256,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.57199999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.024,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.111,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.64300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.288,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.87}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.71099999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3129999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79300000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.1279999999999999,"15min":1.0940000000000001},"min":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3440000000000001,"5min":10.827999999999999,"15min":10.827999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.63500000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.0089999999999999,"5min":10.808999999999999,"15min":10.808999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0620000000000001,"15min":1.0820000000000001},"min":{"1min":0.70899999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.155,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89500000000000002,"15min":0.88800000000000001},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.143,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71699999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":477862,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":0.749,"15min":0.749},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.131,"15min":1.131},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74299999999999999,"15min":0.76500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.5},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.67000000000000004,"15min":0.71799999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.21,"15min":1.21},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.879,"15min":0.92900000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.242,"15min":1.242},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.85299999999999998,"15min":0.89500000000000002},"min":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.222,"15min":1.222},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.88100000000000001,"15min":0.91900000000000004},"min":{"1min":0.63,"5min":0.62,"15min":0.62},"max":{"1min":1.262,"5min":1.288,"15min":1.288},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96899999999999997,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.91900000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.88100000000000001,"15min":0.93899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0609999999999999,"5min":1.282,"15min":1.282},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98199999999999998,"15min":1.042},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1180000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":1.022},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0980000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.751}]},{"osd":8,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.242,"5min":1.379,"15min":1.379},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":1.044},"min":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1859999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.93999999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1779999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.0369999999999999,"5min":1.575,"15min":1.575},"last":0.877}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477797,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.435}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.749,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.196,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.60199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.167,"5min":0.92100000000000004,"15min":0.89000000000000001},"min":{"1min":0.58399999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.63100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2450000000000001,"5min":1.468,"15min":1.468},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.89800000000000002,"15min":0.88900000000000001},"min":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.266,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.97499999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.85499999999999998},"min":{"1min":0.65400000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.171,"5min":0.96399999999999997,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88400000000000001,"15min":0.85199999999999998},"min":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94999999999999996,"15min":0.91100000000000003},"min":{"1min":0.65400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.94699999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.54600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1072070,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.58399999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.57299999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":1.044,"15min":1.044},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66300000000000003},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.62},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.59799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0309999999999999,"5min":1.073,"15min":1.073},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.82999999999999996,"15min":0.77700000000000002},"min":{"1min":0.65100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95299999999999996,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91800000000000004,"5min":1.444,"15min":1.444},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98199999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":1.0469999999999999},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.119,"15min":1.119},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.878,"15min":0.88400000000000001},"min":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.198,"15min":1.198},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.85799999999999998,"15min":0.85399999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0049999999999999,"5min":1.034,"15min":1.034},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.91000000000000003,"15min":0.92600000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1819999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 21:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0069999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478084,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.871},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.80900000000000005,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.66100000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.84099999999999997,"5min":1.887,"15min":1.887},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":0.84999999999999998,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.752},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.097,"5min":0.86199999999999999,"15min":0.81299999999999994},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.441},"max":{"1min":5.6870000000000003,"5min":5.6870000000000003,"15min":5.6870000000000003},"last":0.99299999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.53300000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.568,"5min":10.568,"15min":10.568},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.827,"15min":1.827},"last":0.75}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.54900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":0.96199999999999997,"15min":0.89000000000000001},"min":{"1min":0.55600000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.551,"5min":10.551,"15min":10.551},"last":0.95699999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.399,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.997,"5min":1.905,"15min":1.905},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.337,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.024},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.93700000000000006,"5min":1.232,"15min":1.232},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.0640000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":538720,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.67300000000000004,"15min":0.629},"min":{"1min":0.5,"5min":0.373,"15min":0.373},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.78900000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":15.016999999999999,"5min":15.016999999999999,"15min":15.016999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72799999999999998,"15min":0.65600000000000003},"min":{"1min":0.57299999999999995,"5min":0.39300000000000002,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69699999999999995,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98699999999999999,"5min":1.619,"15min":1.619},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":6.3419999999999996,"5min":6.3419999999999996,"15min":6.3419999999999996},"last":0.87},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64800000000000002,"15min":0.57199999999999995},"min":{"1min":0.54200000000000004,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.63400000000000001,"15min":0.55400000000000005},"min":{"1min":0.53700000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.66100000000000003},"min":{"1min":0.59599999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.079,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.79100000000000004,"15min":0.68000000000000005},"min":{"1min":0.69399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":3.149,"15min":3.149},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82299999999999995,"15min":0.76100000000000001},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.0800000000000001,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.64400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.9410000000000001,"5min":3.218,"15min":3.218},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94099999999999995,"5min":8.7620000000000005,"15min":8.7620000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0660000000000001,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":0.629}]},{"osd":8,"last update":"Fri Oct 6 21:19:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.83499999999999996,"15min":0.78900000000000003},"min":{"1min":0.73999999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.91800000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.92300000000000004,"15min":0.84499999999999997},"min":{"1min":0.82599999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2989999999999999,"5min":7.1760000000000002,"15min":7.1760000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.96999999999999997,"15min":0.89500000000000002},"min":{"1min":0.80400000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":0.93799999999999994,"15min":0.89400000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":6.0419999999999998,"5min":6.0419999999999998,"15min":6.0419999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.016,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.95799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":479417,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":2.6030000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71499999999999997,"15min":0.72799999999999998},"min":{"1min":0.51400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93100000000000005,"5min":1.204,"15min":2.5419999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.749},"min":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":1.1759999999999999,"5min":1.228,"15min":2.5150000000000001},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.74299999999999999,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74099999999999999,"15min":0.69799999999999995},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96699999999999997,"5min":1.139,"15min":1.139},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.71599999999999997,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.85799999999999998,"15min":0.83499999999999996},"min":{"1min":0.65500000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.159,"5min":1.228,"15min":1.228},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.89900000000000002,"15min":0.91000000000000003},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.163,"5min":1.286,"15min":1.286},"last":0.872},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.98999999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0329999999999999,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.94899999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.71799999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.127,"5min":1.302,"15min":1.302},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.89500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":539166,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.56799999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0620000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.57499999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84399999999999997,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.627},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.137,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68899999999999995,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.048,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.56899999999999995,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1140000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":2.5899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.123,"5min":1.298,"15min":1.298},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1220000000000001,"5min":1.266,"15min":1.266},"last":0.68300000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90000000000000002,"15min":0.88200000000000001},"min":{"1min":0.73899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.85799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.194},"last":0.505}]},{"osd":10,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.93500000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2490000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.88900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477416,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45000000000000001,"15min":0.40999999999999998},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71299999999999997,"15min":0.65400000000000003},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":1.0009999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.75,"15min":0.67200000000000004},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.1799999999999997,"5min":5.1799999999999997,"15min":5.1799999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72099999999999997,"15min":0.67900000000000005},"min":{"1min":0.51000000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.181,"5min":1.206,"15min":1.206},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98799999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.125,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92700000000000005,"5min":1.26,"15min":1.26},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.254,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96499999999999997,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.60699999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.88100000000000001,"15min":0.90700000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.47099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.93400000000000005,"15min":0.94099999999999995},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.113,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.75},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72499999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478470,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.40100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":0.874,"15min":0.874},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.5,"15min":0.43099999999999999},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97699999999999998,"5min":1.601,"15min":1.601},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.874},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.74399999999999999,"15min":0.66400000000000003},"min":{"1min":0.53900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.53300000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.173,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81100000000000005,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0720000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.73799999999999999},"min":{"1min":0.59199999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.55200000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.758,"15min":1.758},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":1.018},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0640000000000001,"5min":1.127,"15min":1.127},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83599999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0629999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86799999999999999,"15min":0.84999999999999998},"min":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.86699999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.044,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.92300000000000004},"min":{"1min":0.72699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.081,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.89800000000000002},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.66800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.042,"5min":1.093,"15min":1.093},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85799999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1070550,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.66800000000000004},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1799999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.45900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.093}]},{"osd":3,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.151},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.61299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.999,"5min":1.036,"15min":1.036},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77700000000000002,"15min":0.71599999999999997},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1040000000000001,"5min":1.196,"15min":1.196},"last":1.258}]},{"osd":6,"last update":"Fri Oct 6 21:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.78000000000000003,"15min":0.753},"min":{"1min":0.64800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.347},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.72899999999999998,"15min":0.70399999999999996},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.196}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.86199999999999999,"15min":0.84199999999999997},"min":{"1min":0.77600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.103,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.401},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80500000000000005,"15min":0.77300000000000002},"min":{"1min":0.69099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.232,"15min":1.232},"last":1.329}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":1.383},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.4830000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85899999999999999,"15min":0.84099999999999997},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.873,"15min":0.877},"min":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.143,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.306}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.91000000000000003,"15min":0.87},"min":{"1min":0.88700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.82399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.3660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61980,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62699,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62495,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:20:42.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:20:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 -- 172.21.15.31:0/1136060498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 msgr2=0x7fd688102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1136060498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd688102500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd678009990 tx=0x7fd678009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 -- 172.21.15.31:0/1136060498 shutdown_connections 2023-10-06T21:20:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1136060498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd688109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1136060498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 0x7fd6881012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1136060498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd688102500 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 -- 172.21.15.31:0/1136060498 >> 172.21.15.31:0/1136060498 conn(0x7fd6880fc420 msgr2=0x7fd6880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:43.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 -- 172.21.15.31:0/1136060498 shutdown_connections 2023-10-06T21:20:43.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.223+0000 7fd6905ff700 1 -- 172.21.15.31:0/1136060498 wait complete. 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 Processor -- start 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 -- start start 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 0x7fd688074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd6880737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd688073f80 con 0x7fd688102080 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880740f0 con 0x7fd688102a40 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.224+0000 7fd6905ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6881a0bb0 con 0x7fd688100ea0 2023-10-06T21:20:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56750/0 (socket says 172.21.15.31:56750) 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 -- 172.21.15.31:0/1817443961 learned_addr learned my addr 172.21.15.31:0/1817443961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68e39b700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 0x7fd688074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68eb9c700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd6880737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 -- 172.21.15.31:0/1817443961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 msgr2=0x7fd688074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 0x7fd688074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 -- 172.21.15.31:0/1817443961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 msgr2=0x7fd6880737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd6880737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 -- 172.21.15.31:0/1817443961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780095f0 con 0x7fd688102080 2023-10-06T21:20:43.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68eb9c700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd6880737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd68db9a700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd678004730 tx=0x7fd678005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.225+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67801d070 con 0x7fd688102080 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.226+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6881a0e30 con 0x7fd688102080 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.226+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd678004ca0 con 0x7fd688102080 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.226+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678003d90 con 0x7fd688102080 2023-10-06T21:20:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.226+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6881a11f0 con 0x7fd688102080 2023-10-06T21:20:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.227+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68810c610 con 0x7fd688102080 2023-10-06T21:20:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.229+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd678004810 con 0x7fd688102080 2023-10-06T21:20:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.230+0000 7fd67f7fe700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 0x7fd6740a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.230+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fd6780ce1e0 con 0x7fd688102080 2023-10-06T21:20:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.230+0000 7fd68e39b700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 0x7fd6740a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.231+0000 7fd68e39b700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 0x7fd6740a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd684005950 tx=0x7fd6840058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.232+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd678097810 con 0x7fd688102080 2023-10-06T21:20:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:42 smithi031 ceph-mon[98539]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T21:20:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3408034633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:43.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:42 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/186683760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:42 smithi146 ceph-mon[102809]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T21:20:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3408034633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:43.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:42 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/186683760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:42 smithi142 ceph-mon[102951]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T21:20:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3408034633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:43.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:42 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/186683760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:43.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.436+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6880612f0 con 0x7fd6740a1550 2023-10-06T21:20:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.447+0000 7fd67f7fe700 1 -- 172.21.15.31:0/1817443961 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343375 (secure 0 0 0) 0x7fd6880612f0 con 0x7fd6740a1550 2023-10-06T21:20:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 msgr2=0x7fd6740a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 0x7fd6740a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd684005950 tx=0x7fd6840058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 msgr2=0x7fd6880732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd678004730 tx=0x7fd678005e80 comp rx=0 tx=0).stop 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 shutdown_connections 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd6740a1550 0x7fd6740a3a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd688102a40 0x7fd6880737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd688100ea0 0x7fd688074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 --2- 172.21.15.31:0/1817443961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688102080 0x7fd6880732a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 >> 172.21.15.31:0/1817443961 conn(0x7fd6880fc420 msgr2=0x7fd6880fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 shutdown_connections 2023-10-06T21:20:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.452+0000 7fd6905ff700 1 -- 172.21.15.31:0/1817443961 wait complete. 2023-10-06T21:20:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:20:43.473 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2023-10-06T21:20:41.836505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659292,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1181,"num_read_kb":717,"num_write":432,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336300,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605524,"statfs":{"total":1151940427776,"available":1151596056576,"internally_reserved":0,"allocated":12357632,"data_stored":7697490,"data_compressed":251907,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2495,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002684"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908715+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:20:25.908715+0000","last_peered":"2023-10-06T21:20:25.908715+0000","last_clean":"2023-10-06T21:20:25.908715+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:20:25.908715+0000","last_undegraded":"2023-10-06T21:20:25.908715+0000","last_fullsized":"2023-10-06T21:20:25.908715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502125+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:20:26.502125+0000","last_peered":"2023-10-06T21:20:26.502125+0000","last_clean":"2023-10-06T21:20:26.502125+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:20:26.502125+0000","last_undegraded":"2023-10-06T21:20:26.502125+0000","last_fullsized":"2023-10-06T21:20:26.502125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502218+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:20:26.502218+0000","last_peered":"2023-10-06T21:20:26.502218+0000","last_clean":"2023-10-06T21:20:26.502218+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:20:26.502218+0000","last_undegraded":"2023-10-06T21:20:26.502218+0000","last_fullsized":"2023-10-06T21:20:26.502218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889201+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:20:25.889201+0000","last_peered":"2023-10-06T21:20:25.889201+0000","last_clean":"2023-10-06T21:20:25.889201+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:20:25.889201+0000","last_undegraded":"2023-10-06T21:20:25.889201+0000","last_fullsized":"2023-10-06T21:20:25.889201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898528+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:20:25.898528+0000","last_peered":"2023-10-06T21:20:25.898528+0000","last_clean":"2023-10-06T21:20:25.898528+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:20:25.898528+0000","last_undegraded":"2023-10-06T21:20:25.898528+0000","last_fullsized":"2023-10-06T21:20:25.898528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908557+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:20:25.908557+0000","last_peered":"2023-10-06T21:20:25.908557+0000","last_clean":"2023-10-06T21:20:25.908557+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:20:25.908557+0000","last_undegraded":"2023-10-06T21:20:25.908557+0000","last_fullsized":"2023-10-06T21:20:25.908557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890967+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:20:25.890967+0000","last_peered":"2023-10-06T21:20:25.890967+0000","last_clean":"2023-10-06T21:20:25.890967+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:20:25.890967+0000","last_undegraded":"2023-10-06T21:20:25.890967+0000","last_fullsized":"2023-10-06T21:20:25.890967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894114+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:20:25.894114+0000","last_peered":"2023-10-06T21:20:25.894114+0000","last_clean":"2023-10-06T21:20:25.894114+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:20:25.894114+0000","last_undegraded":"2023-10-06T21:20:25.894114+0000","last_fullsized":"2023-10-06T21:20:25.894114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894063+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:20:25.894063+0000","last_peered":"2023-10-06T21:20:25.894063+0000","last_clean":"2023-10-06T21:20:25.894063+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:20:25.894063+0000","last_undegraded":"2023-10-06T21:20:25.894063+0000","last_fullsized":"2023-10-06T21:20:25.894063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908894+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:20:25.908894+0000","last_peered":"2023-10-06T21:20:25.908894+0000","last_clean":"2023-10-06T21:20:25.908894+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:20:25.908894+0000","last_undegraded":"2023-10-06T21:20:25.908894+0000","last_fullsized":"2023-10-06T21:20:25.908894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891256+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:20:25.891256+0000","last_peered":"2023-10-06T21:20:25.891256+0000","last_clean":"2023-10-06T21:20:25.891256+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:20:25.891256+0000","last_undegraded":"2023-10-06T21:20:25.891256+0000","last_fullsized":"2023-10-06T21:20:25.891256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889121+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:20:25.889121+0000","last_peered":"2023-10-06T21:20:25.889121+0000","last_clean":"2023-10-06T21:20:25.889121+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:20:25.889121+0000","last_undegraded":"2023-10-06T21:20:25.889121+0000","last_fullsized":"2023-10-06T21:20:25.889121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893926+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:20:25.893926+0000","last_peered":"2023-10-06T21:20:25.893926+0000","last_clean":"2023-10-06T21:20:25.893926+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:20:25.893926+0000","last_undegraded":"2023-10-06T21:20:25.893926+0000","last_fullsized":"2023-10-06T21:20:25.893926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501878+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:20:26.501878+0000","last_peered":"2023-10-06T21:20:26.501878+0000","last_clean":"2023-10-06T21:20:26.501878+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:20:26.501878+0000","last_undegraded":"2023-10-06T21:20:26.501878+0000","last_fullsized":"2023-10-06T21:20:26.501878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501915+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:20:26.501915+0000","last_peered":"2023-10-06T21:20:26.501915+0000","last_clean":"2023-10-06T21:20:26.501915+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:20:26.501915+0000","last_undegraded":"2023-10-06T21:20:26.501915+0000","last_fullsized":"2023-10-06T21:20:26.501915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893547+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:20:25.893547+0000","last_peered":"2023-10-06T21:20:25.893547+0000","last_clean":"2023-10-06T21:20:25.893547+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:20:25.893547+0000","last_undegraded":"2023-10-06T21:20:25.893547+0000","last_fullsized":"2023-10-06T21:20:25.893547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908765+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:20:25.908765+0000","last_peered":"2023-10-06T21:20:25.908765+0000","last_clean":"2023-10-06T21:20:25.908765+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:20:25.908765+0000","last_undegraded":"2023-10-06T21:20:25.908765+0000","last_fullsized":"2023-10-06T21:20:25.908765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889141+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:20:25.889141+0000","last_peered":"2023-10-06T21:20:25.889141+0000","last_clean":"2023-10-06T21:20:25.889141+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:20:25.889141+0000","last_undegraded":"2023-10-06T21:20:25.889141+0000","last_fullsized":"2023-10-06T21:20:25.889141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889176+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:20:25.889176+0000","last_peered":"2023-10-06T21:20:25.889176+0000","last_clean":"2023-10-06T21:20:25.889176+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:20:25.889176+0000","last_undegraded":"2023-10-06T21:20:25.889176+0000","last_fullsized":"2023-10-06T21:20:25.889176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:36.565843+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:20:36.565843+0000","last_peered":"2023-10-06T21:20:36.565843+0000","last_clean":"2023-10-06T21:20:36.565843+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:20:36.565843+0000","last_undegraded":"2023-10-06T21:20:36.565843+0000","last_fullsized":"2023-10-06T21:20:36.565843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894964+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:20:25.894964+0000","last_peered":"2023-10-06T21:20:25.894964+0000","last_clean":"2023-10-06T21:20:25.894964+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:20:25.894964+0000","last_undegraded":"2023-10-06T21:20:25.894964+0000","last_fullsized":"2023-10-06T21:20:25.894964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895174+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:20:25.895174+0000","last_peered":"2023-10-06T21:20:25.895174+0000","last_clean":"2023-10-06T21:20:25.895174+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:20:25.895174+0000","last_undegraded":"2023-10-06T21:20:25.895174+0000","last_fullsized":"2023-10-06T21:20:25.895174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502253+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:20:26.502253+0000","last_peered":"2023-10-06T21:20:26.502253+0000","last_clean":"2023-10-06T21:20:26.502253+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:20:26.502253+0000","last_undegraded":"2023-10-06T21:20:26.502253+0000","last_fullsized":"2023-10-06T21:20:26.502253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898484+0000","last_change":"2023-10-06T21:19:26.863512+0000","last_active":"2023-10-06T21:20:25.898484+0000","last_peered":"2023-10-06T21:20:25.898484+0000","last_clean":"2023-10-06T21:20:25.898484+0000","last_became_active":"2023-10-06T21:19:26.845274+0000","last_became_peered":"2023-10-06T21:19:26.845274+0000","last_unstale":"2023-10-06T21:20:25.898484+0000","last_undegraded":"2023-10-06T21:20:25.898484+0000","last_fullsized":"2023-10-06T21:20:25.898484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:16.751914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893672+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:20:25.893672+0000","last_peered":"2023-10-06T21:20:25.893672+0000","last_clean":"2023-10-06T21:20:25.893672+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:20:25.893672+0000","last_undegraded":"2023-10-06T21:20:25.893672+0000","last_fullsized":"2023-10-06T21:20:25.893672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898508+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:20:25.898508+0000","last_peered":"2023-10-06T21:20:25.898508+0000","last_clean":"2023-10-06T21:20:25.898508+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:20:25.898508+0000","last_undegraded":"2023-10-06T21:20:25.898508+0000","last_fullsized":"2023-10-06T21:20:25.898508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897746+0000","last_change":"2023-10-06T21:19:26.991142+0000","last_active":"2023-10-06T21:20:25.897746+0000","last_peered":"2023-10-06T21:20:25.897746+0000","last_clean":"2023-10-06T21:20:25.897746+0000","last_became_active":"2023-10-06T21:19:26.990765+0000","last_became_peered":"2023-10-06T21:19:26.990765+0000","last_unstale":"2023-10-06T21:20:25.897746+0000","last_undegraded":"2023-10-06T21:20:25.897746+0000","last_fullsized":"2023-10-06T21:20:25.897746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:35:44.692174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:39.987358+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:20:39.987358+0000","last_peered":"2023-10-06T21:20:39.987358+0000","last_clean":"2023-10-06T21:20:39.987358+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:20:39.987358+0000","last_undegraded":"2023-10-06T21:20:39.987358+0000","last_fullsized":"2023-10-06T21:20:39.987358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1121,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897701+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:20:25.897701+0000","last_peered":"2023-10-06T21:20:25.897701+0000","last_clean":"2023-10-06T21:20:25.897701+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:20:25.897701+0000","last_undegraded":"2023-10-06T21:20:25.897701+0000","last_fullsized":"2023-10-06T21:20:25.897701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908501+0000","last_change":"2023-10-06T21:19:26.998407+0000","last_active":"2023-10-06T21:20:25.908501+0000","last_peered":"2023-10-06T21:20:25.908501+0000","last_clean":"2023-10-06T21:20:25.908501+0000","last_became_active":"2023-10-06T21:19:26.991918+0000","last_became_peered":"2023-10-06T21:19:26.991918+0000","last_unstale":"2023-10-06T21:20:25.908501+0000","last_undegraded":"2023-10-06T21:20:25.908501+0000","last_fullsized":"2023-10-06T21:20:25.908501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:05.903796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899375+0000","last_change":"2023-10-06T21:20:26.898661+0000","last_active":"2023-10-06T21:20:26.899375+0000","last_peered":"2023-10-06T21:20:26.899375+0000","last_clean":"2023-10-06T21:20:26.899375+0000","last_became_active":"2023-10-06T21:20:26.898216+0000","last_became_peered":"2023-10-06T21:20:26.898216+0000","last_unstale":"2023-10-06T21:20:26.899375+0000","last_undegraded":"2023-10-06T21:20:26.899375+0000","last_fullsized":"2023-10-06T21:20:26.899375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:55:14.473221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893480+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:20:25.893480+0000","last_peered":"2023-10-06T21:20:25.893480+0000","last_clean":"2023-10-06T21:20:25.893480+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:20:25.893480+0000","last_undegraded":"2023-10-06T21:20:25.893480+0000","last_fullsized":"2023-10-06T21:20:25.893480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894320+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:20:25.894320+0000","last_peered":"2023-10-06T21:20:25.894320+0000","last_clean":"2023-10-06T21:20:25.894320+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:20:25.894320+0000","last_undegraded":"2023-10-06T21:20:25.894320+0000","last_fullsized":"2023-10-06T21:20:25.894320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897315+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:20:25.897315+0000","last_peered":"2023-10-06T21:20:25.897315+0000","last_clean":"2023-10-06T21:20:25.897315+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:20:25.897315+0000","last_undegraded":"2023-10-06T21:20:25.897315+0000","last_fullsized":"2023-10-06T21:20:25.897315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888820+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:20:25.888820+0000","last_peered":"2023-10-06T21:20:25.888820+0000","last_clean":"2023-10-06T21:20:25.888820+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:20:25.888820+0000","last_undegraded":"2023-10-06T21:20:25.888820+0000","last_fullsized":"2023-10-06T21:20:25.888820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":226,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988250+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:20:40.988250+0000","last_peered":"2023-10-06T21:20:40.988250+0000","last_clean":"2023-10-06T21:20:40.988250+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:20:40.988250+0000","last_undegraded":"2023-10-06T21:20:40.988250+0000","last_fullsized":"2023-10-06T21:20:40.988250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":203,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894451+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:20:25.894451+0000","last_peered":"2023-10-06T21:20:25.894451+0000","last_clean":"2023-10-06T21:20:25.894451+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:20:25.894451+0000","last_undegraded":"2023-10-06T21:20:25.894451+0000","last_fullsized":"2023-10-06T21:20:25.894451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891255+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:20:25.891255+0000","last_peered":"2023-10-06T21:20:25.891255+0000","last_clean":"2023-10-06T21:20:25.891255+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:20:25.891255+0000","last_undegraded":"2023-10-06T21:20:25.891255+0000","last_fullsized":"2023-10-06T21:20:25.891255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502189+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:20:26.502189+0000","last_peered":"2023-10-06T21:20:26.502189+0000","last_clean":"2023-10-06T21:20:26.502189+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:20:26.502189+0000","last_undegraded":"2023-10-06T21:20:26.502189+0000","last_fullsized":"2023-10-06T21:20:26.502189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897849+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:20:25.897849+0000","last_peered":"2023-10-06T21:20:25.897849+0000","last_clean":"2023-10-06T21:20:25.897849+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:20:25.897849+0000","last_undegraded":"2023-10-06T21:20:25.897849+0000","last_fullsized":"2023-10-06T21:20:25.897849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893778+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:20:25.893778+0000","last_peered":"2023-10-06T21:20:25.893778+0000","last_clean":"2023-10-06T21:20:25.893778+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:20:25.893778+0000","last_undegraded":"2023-10-06T21:20:25.893778+0000","last_fullsized":"2023-10-06T21:20:25.893778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988126+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:20:40.988126+0000","last_peered":"2023-10-06T21:20:40.988126+0000","last_clean":"2023-10-06T21:20:40.988126+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:20:40.988126+0000","last_undegraded":"2023-10-06T21:20:40.988126+0000","last_fullsized":"2023-10-06T21:20:40.988126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59729,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":37,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898054+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:20:25.898054+0000","last_peered":"2023-10-06T21:20:25.898054+0000","last_clean":"2023-10-06T21:20:25.898054+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:20:25.898054+0000","last_undegraded":"2023-10-06T21:20:25.898054+0000","last_fullsized":"2023-10-06T21:20:25.898054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889077+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:20:25.889077+0000","last_peered":"2023-10-06T21:20:25.889077+0000","last_clean":"2023-10-06T21:20:25.889077+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:20:25.889077+0000","last_undegraded":"2023-10-06T21:20:25.889077+0000","last_fullsized":"2023-10-06T21:20:25.889077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893449+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:20:25.893449+0000","last_peered":"2023-10-06T21:20:25.893449+0000","last_clean":"2023-10-06T21:20:25.893449+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:20:25.893449+0000","last_undegraded":"2023-10-06T21:20:25.893449+0000","last_fullsized":"2023-10-06T21:20:25.893449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897611+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:20:25.897611+0000","last_peered":"2023-10-06T21:20:25.897611+0000","last_clean":"2023-10-06T21:20:25.897611+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:20:25.897611+0000","last_undegraded":"2023-10-06T21:20:25.897611+0000","last_fullsized":"2023-10-06T21:20:25.897611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.896400+0000","last_change":"2023-10-06T21:20:26.896400+0000","last_active":"2023-10-06T21:20:26.896400+0000","last_peered":"2023-10-06T21:20:26.896400+0000","last_clean":"2023-10-06T21:20:26.896400+0000","last_became_active":"2023-10-06T21:20:26.896036+0000","last_became_peered":"2023-10-06T21:20:26.896036+0000","last_unstale":"2023-10-06T21:20:26.896400+0000","last_undegraded":"2023-10-06T21:20:26.896400+0000","last_fullsized":"2023-10-06T21:20:26.896400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:30:04.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.030597+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:20:40.030597+0000","last_peered":"2023-10-06T21:20:40.030597+0000","last_clean":"2023-10-06T21:20:40.030597+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:20:40.030597+0000","last_undegraded":"2023-10-06T21:20:40.030597+0000","last_fullsized":"2023-10-06T21:20:40.030597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891007+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:20:25.891007+0000","last_peered":"2023-10-06T21:20:25.891007+0000","last_clean":"2023-10-06T21:20:25.891007+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:20:25.891007+0000","last_undegraded":"2023-10-06T21:20:25.891007+0000","last_fullsized":"2023-10-06T21:20:25.891007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908465+0000","last_change":"2023-10-06T21:19:26.862156+0000","last_active":"2023-10-06T21:20:25.908465+0000","last_peered":"2023-10-06T21:20:25.908465+0000","last_clean":"2023-10-06T21:20:25.908465+0000","last_became_active":"2023-10-06T21:19:26.844987+0000","last_became_peered":"2023-10-06T21:19:26.844987+0000","last_unstale":"2023-10-06T21:20:25.908465+0000","last_undegraded":"2023-10-06T21:20:25.908465+0000","last_fullsized":"2023-10-06T21:20:25.908465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:12:56.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897320+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:20:25.897320+0000","last_peered":"2023-10-06T21:20:25.897320+0000","last_clean":"2023-10-06T21:20:25.897320+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:20:25.897320+0000","last_undegraded":"2023-10-06T21:20:25.897320+0000","last_fullsized":"2023-10-06T21:20:25.897320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897952+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:20:25.897952+0000","last_peered":"2023-10-06T21:20:25.897952+0000","last_clean":"2023-10-06T21:20:25.897952+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:20:25.897952+0000","last_undegraded":"2023-10-06T21:20:25.897952+0000","last_fullsized":"2023-10-06T21:20:25.897952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908587+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:20:25.908587+0000","last_peered":"2023-10-06T21:20:25.908587+0000","last_clean":"2023-10-06T21:20:25.908587+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:20:25.908587+0000","last_undegraded":"2023-10-06T21:20:25.908587+0000","last_fullsized":"2023-10-06T21:20:25.908587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889118+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:20:25.889118+0000","last_peered":"2023-10-06T21:20:25.889118+0000","last_clean":"2023-10-06T21:20:25.889118+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:20:25.889118+0000","last_undegraded":"2023-10-06T21:20:25.889118+0000","last_fullsized":"2023-10-06T21:20:25.889118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908797+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:20:25.908797+0000","last_peered":"2023-10-06T21:20:25.908797+0000","last_clean":"2023-10-06T21:20:25.908797+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:20:25.908797+0000","last_undegraded":"2023-10-06T21:20:25.908797+0000","last_fullsized":"2023-10-06T21:20:25.908797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.006783+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:20:37.006783+0000","last_peered":"2023-10-06T21:20:37.006783+0000","last_clean":"2023-10-06T21:20:37.006783+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:20:37.006783+0000","last_undegraded":"2023-10-06T21:20:37.006783+0000","last_fullsized":"2023-10-06T21:20:37.006783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908595+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:20:25.908595+0000","last_peered":"2023-10-06T21:20:25.908595+0000","last_clean":"2023-10-06T21:20:25.908595+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:20:25.908595+0000","last_undegraded":"2023-10-06T21:20:25.908595+0000","last_fullsized":"2023-10-06T21:20:25.908595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897427+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:20:25.897427+0000","last_peered":"2023-10-06T21:20:25.897427+0000","last_clean":"2023-10-06T21:20:25.897427+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:20:25.897427+0000","last_undegraded":"2023-10-06T21:20:25.897427+0000","last_fullsized":"2023-10-06T21:20:25.897427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.023875+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:20:37.023875+0000","last_peered":"2023-10-06T21:20:37.023875+0000","last_clean":"2023-10-06T21:20:37.023875+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:20:37.023875+0000","last_undegraded":"2023-10-06T21:20:37.023875+0000","last_fullsized":"2023-10-06T21:20:37.023875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894894+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:20:25.894894+0000","last_peered":"2023-10-06T21:20:25.894894+0000","last_clean":"2023-10-06T21:20:25.894894+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:20:25.894894+0000","last_undegraded":"2023-10-06T21:20:25.894894+0000","last_fullsized":"2023-10-06T21:20:25.894894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889049+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:20:25.889049+0000","last_peered":"2023-10-06T21:20:25.889049+0000","last_clean":"2023-10-06T21:20:25.889049+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:20:25.889049+0000","last_undegraded":"2023-10-06T21:20:25.889049+0000","last_fullsized":"2023-10-06T21:20:25.889049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":323,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891086+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:20:25.891086+0000","last_peered":"2023-10-06T21:20:25.891086+0000","last_clean":"2023-10-06T21:20:25.891086+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:20:25.891086+0000","last_undegraded":"2023-10-06T21:20:25.891086+0000","last_fullsized":"2023-10-06T21:20:25.891086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893217+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:20:25.893217+0000","last_peered":"2023-10-06T21:20:25.893217+0000","last_clean":"2023-10-06T21:20:25.893217+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:20:25.893217+0000","last_undegraded":"2023-10-06T21:20:25.893217+0000","last_fullsized":"2023-10-06T21:20:25.893217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908323+0000","last_change":"2023-10-06T21:19:26.864852+0000","last_active":"2023-10-06T21:20:25.908323+0000","last_peered":"2023-10-06T21:20:25.908323+0000","last_clean":"2023-10-06T21:20:25.908323+0000","last_became_active":"2023-10-06T21:19:26.846795+0000","last_became_peered":"2023-10-06T21:19:26.846795+0000","last_unstale":"2023-10-06T21:20:25.908323+0000","last_undegraded":"2023-10-06T21:20:25.908323+0000","last_fullsized":"2023-10-06T21:20:25.908323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:15:22.977641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898390+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:20:25.898390+0000","last_peered":"2023-10-06T21:20:25.898390+0000","last_clean":"2023-10-06T21:20:25.898390+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:20:25.898390+0000","last_undegraded":"2023-10-06T21:20:25.898390+0000","last_fullsized":"2023-10-06T21:20:25.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893558+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:20:25.893558+0000","last_peered":"2023-10-06T21:20:25.893558+0000","last_clean":"2023-10-06T21:20:25.893558+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:20:25.893558+0000","last_undegraded":"2023-10-06T21:20:25.893558+0000","last_fullsized":"2023-10-06T21:20:25.893558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.893269+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:20:26.893269+0000","last_peered":"2023-10-06T21:20:26.893269+0000","last_clean":"2023-10-06T21:20:26.893269+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:20:26.893269+0000","last_undegraded":"2023-10-06T21:20:26.893269+0000","last_fullsized":"2023-10-06T21:20:26.893269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895118+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:20:25.895118+0000","last_peered":"2023-10-06T21:20:25.895118+0000","last_clean":"2023-10-06T21:20:25.895118+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:20:25.895118+0000","last_undegraded":"2023-10-06T21:20:25.895118+0000","last_fullsized":"2023-10-06T21:20:25.895118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889178+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:20:25.889178+0000","last_peered":"2023-10-06T21:20:25.889178+0000","last_clean":"2023-10-06T21:20:25.889178+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:20:25.889178+0000","last_undegraded":"2023-10-06T21:20:25.889178+0000","last_fullsized":"2023-10-06T21:20:25.889178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502163+0000","last_change":"2023-10-06T21:19:26.841631+0000","last_active":"2023-10-06T21:20:26.502163+0000","last_peered":"2023-10-06T21:20:26.502163+0000","last_clean":"2023-10-06T21:20:26.502163+0000","last_became_active":"2023-10-06T21:19:26.841352+0000","last_became_peered":"2023-10-06T21:19:26.841352+0000","last_unstale":"2023-10-06T21:20:26.502163+0000","last_undegraded":"2023-10-06T21:20:26.502163+0000","last_fullsized":"2023-10-06T21:20:26.502163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:26:07.482368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893584+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:20:25.893584+0000","last_peered":"2023-10-06T21:20:25.893584+0000","last_clean":"2023-10-06T21:20:25.893584+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:20:25.893584+0000","last_undegraded":"2023-10-06T21:20:25.893584+0000","last_fullsized":"2023-10-06T21:20:25.893584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897765+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:20:25.897765+0000","last_peered":"2023-10-06T21:20:25.897765+0000","last_clean":"2023-10-06T21:20:25.897765+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:20:25.897765+0000","last_undegraded":"2023-10-06T21:20:25.897765+0000","last_fullsized":"2023-10-06T21:20:25.897765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894378+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:20:25.894378+0000","last_peered":"2023-10-06T21:20:25.894378+0000","last_clean":"2023-10-06T21:20:25.894378+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:20:25.894378+0000","last_undegraded":"2023-10-06T21:20:25.894378+0000","last_fullsized":"2023-10-06T21:20:25.894378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908495+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:20:25.908495+0000","last_peered":"2023-10-06T21:20:25.908495+0000","last_clean":"2023-10-06T21:20:25.908495+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:20:25.908495+0000","last_undegraded":"2023-10-06T21:20:25.908495+0000","last_fullsized":"2023-10-06T21:20:25.908495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889012+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:20:25.889012+0000","last_peered":"2023-10-06T21:20:25.889012+0000","last_clean":"2023-10-06T21:20:25.889012+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:20:25.889012+0000","last_undegraded":"2023-10-06T21:20:25.889012+0000","last_fullsized":"2023-10-06T21:20:25.889012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894653+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:20:25.894653+0000","last_peered":"2023-10-06T21:20:25.894653+0000","last_clean":"2023-10-06T21:20:25.894653+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:20:25.894653+0000","last_undegraded":"2023-10-06T21:20:25.894653+0000","last_fullsized":"2023-10-06T21:20:25.894653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898404+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:20:25.898404+0000","last_peered":"2023-10-06T21:20:25.898404+0000","last_clean":"2023-10-06T21:20:25.898404+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:20:25.898404+0000","last_undegraded":"2023-10-06T21:20:25.898404+0000","last_fullsized":"2023-10-06T21:20:25.898404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891356+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:20:25.891356+0000","last_peered":"2023-10-06T21:20:25.891356+0000","last_clean":"2023-10-06T21:20:25.891356+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:20:25.891356+0000","last_undegraded":"2023-10-06T21:20:25.891356+0000","last_fullsized":"2023-10-06T21:20:25.891356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.020044+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:20:37.020044+0000","last_peered":"2023-10-06T21:20:37.020044+0000","last_clean":"2023-10-06T21:20:37.020044+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:20:37.020044+0000","last_undegraded":"2023-10-06T21:20:37.020044+0000","last_fullsized":"2023-10-06T21:20:37.020044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1669,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891410+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:20:25.891410+0000","last_peered":"2023-10-06T21:20:25.891410+0000","last_clean":"2023-10-06T21:20:25.891410+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:20:25.891410+0000","last_undegraded":"2023-10-06T21:20:25.891410+0000","last_fullsized":"2023-10-06T21:20:25.891410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897736+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:20:25.897736+0000","last_peered":"2023-10-06T21:20:25.897736+0000","last_clean":"2023-10-06T21:20:25.897736+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:20:25.897736+0000","last_undegraded":"2023-10-06T21:20:25.897736+0000","last_fullsized":"2023-10-06T21:20:25.897736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891052+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:20:25.891052+0000","last_peered":"2023-10-06T21:20:25.891052+0000","last_clean":"2023-10-06T21:20:25.891052+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:20:25.891052+0000","last_undegraded":"2023-10-06T21:20:25.891052+0000","last_fullsized":"2023-10-06T21:20:25.891052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893514+0000","last_change":"2023-10-06T21:19:26.851061+0000","last_active":"2023-10-06T21:20:25.893514+0000","last_peered":"2023-10-06T21:20:25.893514+0000","last_clean":"2023-10-06T21:20:25.893514+0000","last_became_active":"2023-10-06T21:19:26.845797+0000","last_became_peered":"2023-10-06T21:19:26.845797+0000","last_unstale":"2023-10-06T21:20:25.893514+0000","last_undegraded":"2023-10-06T21:20:25.893514+0000","last_fullsized":"2023-10-06T21:20:25.893514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:48.212793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891426+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:20:25.891426+0000","last_peered":"2023-10-06T21:20:25.891426+0000","last_clean":"2023-10-06T21:20:25.891426+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:20:25.891426+0000","last_undegraded":"2023-10-06T21:20:25.891426+0000","last_fullsized":"2023-10-06T21:20:25.891426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908593+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:20:25.908593+0000","last_peered":"2023-10-06T21:20:25.908593+0000","last_clean":"2023-10-06T21:20:25.908593+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:20:25.908593+0000","last_undegraded":"2023-10-06T21:20:25.908593+0000","last_fullsized":"2023-10-06T21:20:25.908593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888944+0000","last_change":"2023-10-06T21:19:26.844504+0000","last_active":"2023-10-06T21:20:25.888944+0000","last_peered":"2023-10-06T21:20:25.888944+0000","last_clean":"2023-10-06T21:20:25.888944+0000","last_became_active":"2023-10-06T21:19:26.844105+0000","last_became_peered":"2023-10-06T21:19:26.844105+0000","last_unstale":"2023-10-06T21:20:25.888944+0000","last_undegraded":"2023-10-06T21:20:25.888944+0000","last_fullsized":"2023-10-06T21:20:25.888944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:08.655868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888876+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:20:25.888876+0000","last_peered":"2023-10-06T21:20:25.888876+0000","last_clean":"2023-10-06T21:20:25.888876+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:20:25.888876+0000","last_undegraded":"2023-10-06T21:20:25.888876+0000","last_fullsized":"2023-10-06T21:20:25.888876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502036+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:20:26.502036+0000","last_peered":"2023-10-06T21:20:26.502036+0000","last_clean":"2023-10-06T21:20:26.502036+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:20:26.502036+0000","last_undegraded":"2023-10-06T21:20:26.502036+0000","last_fullsized":"2023-10-06T21:20:26.502036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893458+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:20:25.893458+0000","last_peered":"2023-10-06T21:20:25.893458+0000","last_clean":"2023-10-06T21:20:25.893458+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:20:25.893458+0000","last_undegraded":"2023-10-06T21:20:25.893458+0000","last_fullsized":"2023-10-06T21:20:25.893458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908546+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:20:25.908546+0000","last_peered":"2023-10-06T21:20:25.908546+0000","last_clean":"2023-10-06T21:20:25.908546+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:20:25.908546+0000","last_undegraded":"2023-10-06T21:20:25.908546+0000","last_fullsized":"2023-10-06T21:20:25.908546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889295+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:20:25.889295+0000","last_peered":"2023-10-06T21:20:25.889295+0000","last_clean":"2023-10-06T21:20:25.889295+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:20:25.889295+0000","last_undegraded":"2023-10-06T21:20:25.889295+0000","last_fullsized":"2023-10-06T21:20:25.889295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895021+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:20:25.895021+0000","last_peered":"2023-10-06T21:20:25.895021+0000","last_clean":"2023-10-06T21:20:25.895021+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:20:25.895021+0000","last_undegraded":"2023-10-06T21:20:25.895021+0000","last_fullsized":"2023-10-06T21:20:25.895021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890975+0000","last_change":"2023-10-06T21:19:26.847714+0000","last_active":"2023-10-06T21:20:25.890975+0000","last_peered":"2023-10-06T21:20:25.890975+0000","last_clean":"2023-10-06T21:20:25.890975+0000","last_became_active":"2023-10-06T21:19:26.847302+0000","last_became_peered":"2023-10-06T21:19:26.847302+0000","last_unstale":"2023-10-06T21:20:25.890975+0000","last_undegraded":"2023-10-06T21:20:25.890975+0000","last_fullsized":"2023-10-06T21:20:25.890975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:10:24.079203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894276+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:20:25.894276+0000","last_peered":"2023-10-06T21:20:25.894276+0000","last_clean":"2023-10-06T21:20:25.894276+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:20:25.894276+0000","last_undegraded":"2023-10-06T21:20:25.894276+0000","last_fullsized":"2023-10-06T21:20:25.894276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899339+0000","last_change":"2023-10-06T21:20:26.897996+0000","last_active":"2023-10-06T21:20:26.899339+0000","last_peered":"2023-10-06T21:20:26.899339+0000","last_clean":"2023-10-06T21:20:26.899339+0000","last_became_active":"2023-10-06T21:20:26.897555+0000","last_became_peered":"2023-10-06T21:20:26.897555+0000","last_unstale":"2023-10-06T21:20:26.899339+0000","last_undegraded":"2023-10-06T21:20:26.899339+0000","last_fullsized":"2023-10-06T21:20:26.899339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:59:28.968861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893964+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:20:25.893964+0000","last_peered":"2023-10-06T21:20:25.893964+0000","last_clean":"2023-10-06T21:20:25.893964+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:20:25.893964+0000","last_undegraded":"2023-10-06T21:20:25.893964+0000","last_fullsized":"2023-10-06T21:20:25.893964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908609+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:20:25.908609+0000","last_peered":"2023-10-06T21:20:25.908609+0000","last_clean":"2023-10-06T21:20:25.908609+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:20:25.908609+0000","last_undegraded":"2023-10-06T21:20:25.908609+0000","last_fullsized":"2023-10-06T21:20:25.908609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68678,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":598,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206034,"data_compressed":5511,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478543,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3819999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.59699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.038,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.087},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.59799999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93999999999999995,"5min":1.536,"15min":1.536},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.016,"5min":1.23,"15min":1.23},"last":1.052},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.083,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.022}]},{"osd":5,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73199999999999998,"15min":0.747},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86199999999999999,"5min":2.036,"15min":2.036},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85799999999999998},"min":{"1min":0.68600000000000005,"5min":0.626,"15min":0.626},"max":{"1min":1.0600000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":1.631,"15min":1.631},"last":0.88500000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.034},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.143}]},{"osd":9,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.71099999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2949999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.65300000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.302,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.1679999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93600000000000005,"15min":0.93999999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.165,"5min":1.671,"15min":1.671},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.96799999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2749999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0760000000000001}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1129395,"data_compressed":49169,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.177,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.256,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.57199999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.024,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.111,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.64300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.288,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.87}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.71099999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3129999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79300000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.1279999999999999,"15min":1.0940000000000001},"min":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3440000000000001,"5min":10.827999999999999,"15min":10.827999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.63500000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.0089999999999999,"5min":10.808999999999999,"15min":10.808999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0620000000000001,"15min":1.0820000000000001},"min":{"1min":0.70899999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.155,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89500000000000002,"15min":0.88800000000000001},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.143,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71699999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":477862,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":0.749,"15min":0.749},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.131,"15min":1.131},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74299999999999999,"15min":0.76500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.5},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.67000000000000004,"15min":0.71799999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.21,"15min":1.21},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.879,"15min":0.92900000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.242,"15min":1.242},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.85299999999999998,"15min":0.89500000000000002},"min":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.222,"15min":1.222},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.88100000000000001,"15min":0.91900000000000004},"min":{"1min":0.63,"5min":0.62,"15min":0.62},"max":{"1min":1.262,"5min":1.288,"15min":1.288},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96899999999999997,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.91900000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.88100000000000001,"15min":0.93899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0609999999999999,"5min":1.282,"15min":1.282},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98199999999999998,"15min":1.042},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1180000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":1.022},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0980000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.751}]},{"osd":8,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.242,"5min":1.379,"15min":1.379},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":1.044},"min":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1859999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.93999999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1779999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.0369999999999999,"5min":1.575,"15min":1.575},"last":0.877}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477797,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.435}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.749,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.196,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.60199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.167,"5min":0.92100000000000004,"15min":0.89000000000000001},"min":{"1min":0.58399999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.63100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2450000000000001,"5min":1.468,"15min":1.468},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.89800000000000002,"15min":0.88900000000000001},"min":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.266,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.97499999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.85499999999999998},"min":{"1min":0.65400000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.171,"5min":0.96399999999999997,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88400000000000001,"15min":0.85199999999999998},"min":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94999999999999996,"15min":0.91100000000000003},"min":{"1min":0.65400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.94699999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.54600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1072070,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.58399999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.57299999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":1.044,"15min":1.044},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66300000000000003},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.62},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.59799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0309999999999999,"5min":1.073,"15min":1.073},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.82999999999999996,"15min":0.77700000000000002},"min":{"1min":0.65100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95299999999999996,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91800000000000004,"5min":1.444,"15min":1.444},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98199999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":1.0469999999999999},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.119,"15min":1.119},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.878,"15min":0.88400000000000001},"min":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.198,"15min":1.198},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.85799999999999998,"15min":0.85399999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0049999999999999,"5min":1.034,"15min":1.034},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.91000000000000003,"15min":0.92600000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1819999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 21:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0069999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478084,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.871},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.80900000000000005,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.66100000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.84099999999999997,"5min":1.887,"15min":1.887},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":0.84999999999999998,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.752},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.097,"5min":0.86199999999999999,"15min":0.81299999999999994},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.441},"max":{"1min":5.6870000000000003,"5min":5.6870000000000003,"15min":5.6870000000000003},"last":0.99299999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.53300000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.568,"5min":10.568,"15min":10.568},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.827,"15min":1.827},"last":0.75}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.54900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":0.96199999999999997,"15min":0.89000000000000001},"min":{"1min":0.55600000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.551,"5min":10.551,"15min":10.551},"last":0.95699999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.399,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.997,"5min":1.905,"15min":1.905},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.337,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.024},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.93700000000000006,"5min":1.232,"15min":1.232},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.0640000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":538720,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.67300000000000004,"15min":0.629},"min":{"1min":0.5,"5min":0.373,"15min":0.373},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.78900000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":15.016999999999999,"5min":15.016999999999999,"15min":15.016999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72799999999999998,"15min":0.65600000000000003},"min":{"1min":0.57299999999999995,"5min":0.39300000000000002,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69699999999999995,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98699999999999999,"5min":1.619,"15min":1.619},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":6.3419999999999996,"5min":6.3419999999999996,"15min":6.3419999999999996},"last":0.87},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64800000000000002,"15min":0.57199999999999995},"min":{"1min":0.54200000000000004,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.63400000000000001,"15min":0.55400000000000005},"min":{"1min":0.53700000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.66100000000000003},"min":{"1min":0.59599999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.079,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.79100000000000004,"15min":0.68000000000000005},"min":{"1min":0.69399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":3.149,"15min":3.149},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82299999999999995,"15min":0.76100000000000001},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.0800000000000001,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.64400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.9410000000000001,"5min":3.218,"15min":3.218},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94099999999999995,"5min":8.7620000000000005,"15min":8.7620000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0660000000000001,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":0.629}]},{"osd":8,"last update":"Fri Oct 6 21:19:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.83499999999999996,"15min":0.78900000000000003},"min":{"1min":0.73999999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.91800000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.92300000000000004,"15min":0.84499999999999997},"min":{"1min":0.82599999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2989999999999999,"5min":7.1760000000000002,"15min":7.1760000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.96999999999999997,"15min":0.89500000000000002},"min":{"1min":0.80400000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":0.93799999999999994,"15min":0.89400000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":6.0419999999999998,"5min":6.0419999999999998,"15min":6.0419999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.016,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.95799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":479417,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":2.6030000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71499999999999997,"15min":0.72799999999999998},"min":{"1min":0.51400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93100000000000005,"5min":1.204,"15min":2.5419999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.749},"min":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":1.1759999999999999,"5min":1.228,"15min":2.5150000000000001},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.74299999999999999,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74099999999999999,"15min":0.69799999999999995},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96699999999999997,"5min":1.139,"15min":1.139},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.71599999999999997,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.85799999999999998,"15min":0.83499999999999996},"min":{"1min":0.65500000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.159,"5min":1.228,"15min":1.228},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.89900000000000002,"15min":0.91000000000000003},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.163,"5min":1.286,"15min":1.286},"last":0.872},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.98999999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0329999999999999,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.94899999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.71799999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.127,"5min":1.302,"15min":1.302},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.89500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":539166,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.56799999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0620000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.57499999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84399999999999997,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.627},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.137,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68899999999999995,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.048,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.56899999999999995,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1140000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":2.5899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.123,"5min":1.298,"15min":1.298},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1220000000000001,"5min":1.266,"15min":1.266},"last":0.68300000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90000000000000002,"15min":0.88200000000000001},"min":{"1min":0.73899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.85799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.194},"last":0.505}]},{"osd":10,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.93500000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2490000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.88900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477416,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45000000000000001,"15min":0.40999999999999998},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71299999999999997,"15min":0.65400000000000003},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":1.0009999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.75,"15min":0.67200000000000004},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.1799999999999997,"5min":5.1799999999999997,"15min":5.1799999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72099999999999997,"15min":0.67900000000000005},"min":{"1min":0.51000000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.181,"5min":1.206,"15min":1.206},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98799999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.125,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92700000000000005,"5min":1.26,"15min":1.26},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.254,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96499999999999997,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.60699999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.88100000000000001,"15min":0.90700000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.47099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.93400000000000005,"15min":0.94099999999999995},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.113,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.75},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72499999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478470,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.40100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":0.874,"15min":0.874},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.5,"15min":0.43099999999999999},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97699999999999998,"5min":1.601,"15min":1.601},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.874},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.74399999999999999,"15min":0.66400000000000003},"min":{"1min":0.53900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.53300000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.173,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81100000000000005,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0720000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.73799999999999999},"min":{"1min":0.59199999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.55200000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.758,"15min":1.758},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":1.018},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0640000000000001,"5min":1.127,"15min":1.127},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83599999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0629999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86799999999999999,"15min":0.84999999999999998},"min":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.86699999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.044,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.92300000000000004},"min":{"1min":0.72699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.081,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.89800000000000002},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.66800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.042,"5min":1.093,"15min":1.093},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85799999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1070550,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.66800000000000004},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1799999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.45900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.093}]},{"osd":3,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.151},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.61299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.999,"5min":1.036,"15min":1.036},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77700000000000002,"15min":0.71599999999999997},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1040000000000001,"5min":1.196,"15min":1.196},"last":1.258}]},{"osd":6,"last update":"Fri Oct 6 21:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.78000000000000003,"15min":0.753},"min":{"1min":0.64800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.347},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.72899999999999998,"15min":0.70399999999999996},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.196}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.86199999999999999,"15min":0.84199999999999997},"min":{"1min":0.77600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.103,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.401},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80500000000000005,"15min":0.77300000000000002},"min":{"1min":0.69099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.232,"15min":1.232},"last":1.329}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":1.383},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.4830000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85899999999999999,"15min":0.84099999999999997},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.873,"15min":0.877},"min":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.143,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.306}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.91000000000000003,"15min":0.87},"min":{"1min":0.88700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.82399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.3660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61980,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62699,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62495,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:20:43.481 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-06T21:20:43.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 -- 172.21.15.31:0/2276858331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb4074e90 msgr2=0x7fceb40732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/2276858331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb4074e90 0x7fceb40732e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009990 tx=0x7fceb0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 -- 172.21.15.31:0/2276858331 shutdown_connections 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/2276858331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4108dc0 0x7fceb410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/2276858331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb4074e90 0x7fceb40732e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/2276858331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb40738b0 0x7fceb4073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.714+0000 7fcebc1ff700 1 -- 172.21.15.31:0/2276858331 >> 172.21.15.31:0/2276858331 conn(0x7fceb40fc420 msgr2=0x7fceb40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.715+0000 7fcebc1ff700 1 -- 172.21.15.31:0/2276858331 shutdown_connections 2023-10-06T21:20:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.715+0000 7fcebc1ff700 1 -- 172.21.15.31:0/2276858331 wait complete. 2023-10-06T21:20:43.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.715+0000 7fcebc1ff700 1 Processor -- start 2023-10-06T21:20:43.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 -- start start 2023-10-06T21:20:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4074e90 0x7fceb4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 0x7fceb41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb41a08c0 con 0x7fceb40738b0 2023-10-06T21:20:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb41a0a00 con 0x7fceb4108dc0 2023-10-06T21:20:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fcebc1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb41a0b90 con 0x7fceb4074e90 2023-10-06T21:20:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fceb9f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fceb9f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56754/0 (socket says 172.21.15.31:56754) 2023-10-06T21:20:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fceb9f9b700 1 -- 172.21.15.31:0/449309636 learned_addr learned my addr 172.21.15.31:0/449309636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:43.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.716+0000 7fceb979a700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4074e90 0x7fceb4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceba79c700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 0x7fceb41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 -- 172.21.15.31:0/449309636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4074e90 msgr2=0x7fceb4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4074e90 0x7fceb4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 -- 172.21.15.31:0/449309636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 msgr2=0x7fceb41a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 0x7fceb41a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 -- 172.21.15.31:0/449309636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00095f0 con 0x7fceb40738b0 2023-10-06T21:20:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceba79c700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 0x7fceb41a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceb9f9b700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fceb0000c00 tx=0x7fceb0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb001d070 con 0x7fceb40738b0 2023-10-06T21:20:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceb000bd70 con 0x7fceb40738b0 2023-10-06T21:20:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb41a0e10 con 0x7fceb40738b0 2023-10-06T21:20:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0022cf0 con 0x7fceb40738b0 2023-10-06T21:20:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.717+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb41a1250 con 0x7fceb40738b0 2023-10-06T21:20:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.718+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb41074c0 con 0x7fceb40738b0 2023-10-06T21:20:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.720+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fceb0022470 con 0x7fceb40738b0 2023-10-06T21:20:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.723+0000 7fceaaffd700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 0x7fcea00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.723+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fceb00cea10 con 0x7fceb40738b0 2023-10-06T21:20:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.723+0000 7fceb979a700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 0x7fcea00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.724+0000 7fceb979a700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 0x7fcea00a3a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcea4005950 tx=0x7fcea40058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.725+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fceb0098040 con 0x7fceb40738b0 2023-10-06T21:20:43.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.935+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fceb402d230 con 0x7fcea00a1550 2023-10-06T21:20:43.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.945+0000 7fceaaffd700 1 -- 172.21.15.31:0/449309636 <== mgr.14148 v2:172.21.15.31:6800/964869793 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343452 (secure 0 0 0) 0x7fceb402d230 con 0x7fcea00a1550 2023-10-06T21:20:43.953 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 msgr2=0x7fcea00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 0x7fcea00a3a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcea4005950 tx=0x7fcea40058e0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 msgr2=0x7fceb4198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fceb0000c00 tx=0x7fceb0003b00 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 shutdown_connections 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fcea00a1550 0x7fcea00a3a10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4108dc0 0x7fceb41a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fceb40738b0 0x7fceb4198d40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 --2- 172.21.15.31:0/449309636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb4074e90 0x7fceb4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 >> 172.21.15.31:0/449309636 conn(0x7fceb40fc420 msgr2=0x7fceb40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 shutdown_connections 2023-10-06T21:20:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:43.950+0000 7fcebc1ff700 1 -- 172.21.15.31:0/449309636 wait complete. 2023-10-06T21:20:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T21:20:43.973 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2023-10-06T21:20:43.836905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659292,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1191,"num_read_kb":722,"num_write":436,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336292,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605532,"statfs":{"total":1151940427776,"available":1151596064768,"internally_reserved":0,"allocated":12357632,"data_stored":7697490,"data_compressed":251907,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19127,"internal_metadata":331855177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2495,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":14,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002643"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908715+0000","last_change":"2023-10-06T21:18:54.855989+0000","last_active":"2023-10-06T21:20:25.908715+0000","last_peered":"2023-10-06T21:20:25.908715+0000","last_clean":"2023-10-06T21:20:25.908715+0000","last_became_active":"2023-10-06T21:18:54.855447+0000","last_became_peered":"2023-10-06T21:18:54.855447+0000","last_unstale":"2023-10-06T21:20:25.908715+0000","last_undegraded":"2023-10-06T21:20:25.908715+0000","last_fullsized":"2023-10-06T21:20:25.908715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:47.101576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502125+0000","last_change":"2023-10-06T21:18:54.858605+0000","last_active":"2023-10-06T21:20:26.502125+0000","last_peered":"2023-10-06T21:20:26.502125+0000","last_clean":"2023-10-06T21:20:26.502125+0000","last_became_active":"2023-10-06T21:18:54.858260+0000","last_became_peered":"2023-10-06T21:18:54.858260+0000","last_unstale":"2023-10-06T21:20:26.502125+0000","last_undegraded":"2023-10-06T21:20:26.502125+0000","last_fullsized":"2023-10-06T21:20:26.502125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:03:26.990741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502218+0000","last_change":"2023-10-06T21:18:54.854803+0000","last_active":"2023-10-06T21:20:26.502218+0000","last_peered":"2023-10-06T21:20:26.502218+0000","last_clean":"2023-10-06T21:20:26.502218+0000","last_became_active":"2023-10-06T21:18:54.854431+0000","last_became_peered":"2023-10-06T21:18:54.854431+0000","last_unstale":"2023-10-06T21:20:26.502218+0000","last_undegraded":"2023-10-06T21:20:26.502218+0000","last_fullsized":"2023-10-06T21:20:26.502218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:57:56.330648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889201+0000","last_change":"2023-10-06T21:18:54.846687+0000","last_active":"2023-10-06T21:20:25.889201+0000","last_peered":"2023-10-06T21:20:25.889201+0000","last_clean":"2023-10-06T21:20:25.889201+0000","last_became_active":"2023-10-06T21:18:54.846219+0000","last_became_peered":"2023-10-06T21:18:54.846219+0000","last_unstale":"2023-10-06T21:20:25.889201+0000","last_undegraded":"2023-10-06T21:20:25.889201+0000","last_fullsized":"2023-10-06T21:20:25.889201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:35.669787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898528+0000","last_change":"2023-10-06T21:18:54.852499+0000","last_active":"2023-10-06T21:20:25.898528+0000","last_peered":"2023-10-06T21:20:25.898528+0000","last_clean":"2023-10-06T21:20:25.898528+0000","last_became_active":"2023-10-06T21:18:54.852156+0000","last_became_peered":"2023-10-06T21:18:54.852156+0000","last_unstale":"2023-10-06T21:20:25.898528+0000","last_undegraded":"2023-10-06T21:20:25.898528+0000","last_fullsized":"2023-10-06T21:20:25.898528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:29:50.116500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908557+0000","last_change":"2023-10-06T21:18:54.850584+0000","last_active":"2023-10-06T21:20:25.908557+0000","last_peered":"2023-10-06T21:20:25.908557+0000","last_clean":"2023-10-06T21:20:25.908557+0000","last_became_active":"2023-10-06T21:18:54.850264+0000","last_became_peered":"2023-10-06T21:18:54.850264+0000","last_unstale":"2023-10-06T21:20:25.908557+0000","last_undegraded":"2023-10-06T21:20:25.908557+0000","last_fullsized":"2023-10-06T21:20:25.908557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:36:06.193162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890967+0000","last_change":"2023-10-06T21:18:54.850255+0000","last_active":"2023-10-06T21:20:25.890967+0000","last_peered":"2023-10-06T21:20:25.890967+0000","last_clean":"2023-10-06T21:20:25.890967+0000","last_became_active":"2023-10-06T21:18:54.849949+0000","last_became_peered":"2023-10-06T21:18:54.849949+0000","last_unstale":"2023-10-06T21:20:25.890967+0000","last_undegraded":"2023-10-06T21:20:25.890967+0000","last_fullsized":"2023-10-06T21:20:25.890967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:10.002647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894114+0000","last_change":"2023-10-06T21:18:54.855546+0000","last_active":"2023-10-06T21:20:25.894114+0000","last_peered":"2023-10-06T21:20:25.894114+0000","last_clean":"2023-10-06T21:20:25.894114+0000","last_became_active":"2023-10-06T21:18:54.855198+0000","last_became_peered":"2023-10-06T21:18:54.855198+0000","last_unstale":"2023-10-06T21:20:25.894114+0000","last_undegraded":"2023-10-06T21:20:25.894114+0000","last_fullsized":"2023-10-06T21:20:25.894114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:44.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894063+0000","last_change":"2023-10-06T21:18:54.851208+0000","last_active":"2023-10-06T21:20:25.894063+0000","last_peered":"2023-10-06T21:20:25.894063+0000","last_clean":"2023-10-06T21:20:25.894063+0000","last_became_active":"2023-10-06T21:18:54.850928+0000","last_became_peered":"2023-10-06T21:18:54.850928+0000","last_unstale":"2023-10-06T21:20:25.894063+0000","last_undegraded":"2023-10-06T21:20:25.894063+0000","last_fullsized":"2023-10-06T21:20:25.894063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:05:56.567918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908894+0000","last_change":"2023-10-06T21:18:54.859383+0000","last_active":"2023-10-06T21:20:25.908894+0000","last_peered":"2023-10-06T21:20:25.908894+0000","last_clean":"2023-10-06T21:20:25.908894+0000","last_became_active":"2023-10-06T21:18:54.859047+0000","last_became_peered":"2023-10-06T21:18:54.859047+0000","last_unstale":"2023-10-06T21:20:25.908894+0000","last_undegraded":"2023-10-06T21:20:25.908894+0000","last_fullsized":"2023-10-06T21:20:25.908894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.845844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891256+0000","last_change":"2023-10-06T21:18:54.854688+0000","last_active":"2023-10-06T21:20:25.891256+0000","last_peered":"2023-10-06T21:20:25.891256+0000","last_clean":"2023-10-06T21:20:25.891256+0000","last_became_active":"2023-10-06T21:18:54.854264+0000","last_became_peered":"2023-10-06T21:18:54.854264+0000","last_unstale":"2023-10-06T21:20:25.891256+0000","last_undegraded":"2023-10-06T21:20:25.891256+0000","last_fullsized":"2023-10-06T21:20:25.891256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:15.181054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889121+0000","last_change":"2023-10-06T21:18:54.847784+0000","last_active":"2023-10-06T21:20:25.889121+0000","last_peered":"2023-10-06T21:20:25.889121+0000","last_clean":"2023-10-06T21:20:25.889121+0000","last_became_active":"2023-10-06T21:18:54.846720+0000","last_became_peered":"2023-10-06T21:18:54.846720+0000","last_unstale":"2023-10-06T21:20:25.889121+0000","last_undegraded":"2023-10-06T21:20:25.889121+0000","last_fullsized":"2023-10-06T21:20:25.889121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:09:26.865807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893926+0000","last_change":"2023-10-06T21:18:54.846303+0000","last_active":"2023-10-06T21:20:25.893926+0000","last_peered":"2023-10-06T21:20:25.893926+0000","last_clean":"2023-10-06T21:20:25.893926+0000","last_became_active":"2023-10-06T21:18:54.845895+0000","last_became_peered":"2023-10-06T21:18:54.845895+0000","last_unstale":"2023-10-06T21:20:25.893926+0000","last_undegraded":"2023-10-06T21:20:25.893926+0000","last_fullsized":"2023-10-06T21:20:25.893926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:21:31.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501878+0000","last_change":"2023-10-06T21:18:54.854003+0000","last_active":"2023-10-06T21:20:26.501878+0000","last_peered":"2023-10-06T21:20:26.501878+0000","last_clean":"2023-10-06T21:20:26.501878+0000","last_became_active":"2023-10-06T21:18:54.853674+0000","last_became_peered":"2023-10-06T21:18:54.853674+0000","last_unstale":"2023-10-06T21:20:26.501878+0000","last_undegraded":"2023-10-06T21:20:26.501878+0000","last_fullsized":"2023-10-06T21:20:26.501878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:50:43.041179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.501915+0000","last_change":"2023-10-06T21:18:54.856311+0000","last_active":"2023-10-06T21:20:26.501915+0000","last_peered":"2023-10-06T21:20:26.501915+0000","last_clean":"2023-10-06T21:20:26.501915+0000","last_became_active":"2023-10-06T21:18:54.855959+0000","last_became_peered":"2023-10-06T21:18:54.855959+0000","last_unstale":"2023-10-06T21:20:26.501915+0000","last_undegraded":"2023-10-06T21:20:26.501915+0000","last_fullsized":"2023-10-06T21:20:26.501915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:14.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893547+0000","last_change":"2023-10-06T21:18:54.857522+0000","last_active":"2023-10-06T21:20:25.893547+0000","last_peered":"2023-10-06T21:20:25.893547+0000","last_clean":"2023-10-06T21:20:25.893547+0000","last_became_active":"2023-10-06T21:18:54.857229+0000","last_became_peered":"2023-10-06T21:18:54.857229+0000","last_unstale":"2023-10-06T21:20:25.893547+0000","last_undegraded":"2023-10-06T21:20:25.893547+0000","last_fullsized":"2023-10-06T21:20:25.893547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:33.367679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908765+0000","last_change":"2023-10-06T21:18:54.856389+0000","last_active":"2023-10-06T21:20:25.908765+0000","last_peered":"2023-10-06T21:20:25.908765+0000","last_clean":"2023-10-06T21:20:25.908765+0000","last_became_active":"2023-10-06T21:18:54.855953+0000","last_became_peered":"2023-10-06T21:18:54.855953+0000","last_unstale":"2023-10-06T21:20:25.908765+0000","last_undegraded":"2023-10-06T21:20:25.908765+0000","last_fullsized":"2023-10-06T21:20:25.908765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:04.428206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889141+0000","last_change":"2023-10-06T21:18:54.856643+0000","last_active":"2023-10-06T21:20:25.889141+0000","last_peered":"2023-10-06T21:20:25.889141+0000","last_clean":"2023-10-06T21:20:25.889141+0000","last_became_active":"2023-10-06T21:18:54.856273+0000","last_became_peered":"2023-10-06T21:18:54.856273+0000","last_unstale":"2023-10-06T21:20:25.889141+0000","last_undegraded":"2023-10-06T21:20:25.889141+0000","last_fullsized":"2023-10-06T21:20:25.889141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:52:47.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889176+0000","last_change":"2023-10-06T21:18:54.851141+0000","last_active":"2023-10-06T21:20:25.889176+0000","last_peered":"2023-10-06T21:20:25.889176+0000","last_clean":"2023-10-06T21:20:25.889176+0000","last_became_active":"2023-10-06T21:18:54.850845+0000","last_became_peered":"2023-10-06T21:18:54.850845+0000","last_unstale":"2023-10-06T21:20:25.889176+0000","last_undegraded":"2023-10-06T21:20:25.889176+0000","last_fullsized":"2023-10-06T21:20:25.889176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:41.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":133,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:41.012182+0000","last_change":"2023-10-06T21:18:53.855174+0000","last_active":"2023-10-06T21:20:41.012182+0000","last_peered":"2023-10-06T21:20:41.012182+0000","last_clean":"2023-10-06T21:20:41.012182+0000","last_became_active":"2023-10-06T21:18:53.854787+0000","last_became_peered":"2023-10-06T21:18:53.854787+0000","last_unstale":"2023-10-06T21:20:41.012182+0000","last_undegraded":"2023-10-06T21:20:41.012182+0000","last_fullsized":"2023-10-06T21:20:41.012182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:11.897451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894964+0000","last_change":"2023-10-06T21:18:54.850411+0000","last_active":"2023-10-06T21:20:25.894964+0000","last_peered":"2023-10-06T21:20:25.894964+0000","last_clean":"2023-10-06T21:20:25.894964+0000","last_became_active":"2023-10-06T21:18:54.850050+0000","last_became_peered":"2023-10-06T21:18:54.850050+0000","last_unstale":"2023-10-06T21:20:25.894964+0000","last_undegraded":"2023-10-06T21:20:25.894964+0000","last_fullsized":"2023-10-06T21:20:25.894964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:10:06.072337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895174+0000","last_change":"2023-10-06T21:18:53.854721+0000","last_active":"2023-10-06T21:20:25.895174+0000","last_peered":"2023-10-06T21:20:25.895174+0000","last_clean":"2023-10-06T21:20:25.895174+0000","last_became_active":"2023-10-06T21:18:53.854328+0000","last_became_peered":"2023-10-06T21:18:53.854328+0000","last_unstale":"2023-10-06T21:20:25.895174+0000","last_undegraded":"2023-10-06T21:20:25.895174+0000","last_fullsized":"2023-10-06T21:20:25.895174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:57:35.982443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502253+0000","last_change":"2023-10-06T21:18:54.854584+0000","last_active":"2023-10-06T21:20:26.502253+0000","last_peered":"2023-10-06T21:20:26.502253+0000","last_clean":"2023-10-06T21:20:26.502253+0000","last_became_active":"2023-10-06T21:18:54.854232+0000","last_became_peered":"2023-10-06T21:18:54.854232+0000","last_unstale":"2023-10-06T21:20:26.502253+0000","last_undegraded":"2023-10-06T21:20:26.502253+0000","last_fullsized":"2023-10-06T21:20:26.502253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:54:21.626308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898484+0000","last_change":"2023-10-06T21:19:26.863512+0000","last_active":"2023-10-06T21:20:25.898484+0000","last_peered":"2023-10-06T21:20:25.898484+0000","last_clean":"2023-10-06T21:20:25.898484+0000","last_became_active":"2023-10-06T21:19:26.845274+0000","last_became_peered":"2023-10-06T21:19:26.845274+0000","last_unstale":"2023-10-06T21:20:25.898484+0000","last_undegraded":"2023-10-06T21:20:25.898484+0000","last_fullsized":"2023-10-06T21:20:25.898484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:16.751914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893672+0000","last_change":"2023-10-06T21:18:54.852041+0000","last_active":"2023-10-06T21:20:25.893672+0000","last_peered":"2023-10-06T21:20:25.893672+0000","last_clean":"2023-10-06T21:20:25.893672+0000","last_became_active":"2023-10-06T21:18:54.851588+0000","last_became_peered":"2023-10-06T21:18:54.851588+0000","last_unstale":"2023-10-06T21:20:25.893672+0000","last_undegraded":"2023-10-06T21:20:25.893672+0000","last_fullsized":"2023-10-06T21:20:25.893672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:43.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898508+0000","last_change":"2023-10-06T21:18:53.849646+0000","last_active":"2023-10-06T21:20:25.898508+0000","last_peered":"2023-10-06T21:20:25.898508+0000","last_clean":"2023-10-06T21:20:25.898508+0000","last_became_active":"2023-10-06T21:18:53.849165+0000","last_became_peered":"2023-10-06T21:18:53.849165+0000","last_unstale":"2023-10-06T21:20:25.898508+0000","last_undegraded":"2023-10-06T21:20:25.898508+0000","last_fullsized":"2023-10-06T21:20:25.898508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:08:31.208522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897746+0000","last_change":"2023-10-06T21:19:26.991142+0000","last_active":"2023-10-06T21:20:25.897746+0000","last_peered":"2023-10-06T21:20:25.897746+0000","last_clean":"2023-10-06T21:20:25.897746+0000","last_became_active":"2023-10-06T21:19:26.990765+0000","last_became_peered":"2023-10-06T21:19:26.990765+0000","last_unstale":"2023-10-06T21:20:25.897746+0000","last_undegraded":"2023-10-06T21:20:25.897746+0000","last_fullsized":"2023-10-06T21:20:25.897746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:35:44.692174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:39.987358+0000","last_change":"2023-10-06T21:18:53.850371+0000","last_active":"2023-10-06T21:20:39.987358+0000","last_peered":"2023-10-06T21:20:39.987358+0000","last_clean":"2023-10-06T21:20:39.987358+0000","last_became_active":"2023-10-06T21:18:53.849897+0000","last_became_peered":"2023-10-06T21:18:53.849897+0000","last_unstale":"2023-10-06T21:20:39.987358+0000","last_undegraded":"2023-10-06T21:20:39.987358+0000","last_fullsized":"2023-10-06T21:20:39.987358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:53.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1121,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897701+0000","last_change":"2023-10-06T21:18:54.850953+0000","last_active":"2023-10-06T21:20:25.897701+0000","last_peered":"2023-10-06T21:20:25.897701+0000","last_clean":"2023-10-06T21:20:25.897701+0000","last_became_active":"2023-10-06T21:18:54.850634+0000","last_became_peered":"2023-10-06T21:18:54.850634+0000","last_unstale":"2023-10-06T21:20:25.897701+0000","last_undegraded":"2023-10-06T21:20:25.897701+0000","last_fullsized":"2023-10-06T21:20:25.897701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:09.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908501+0000","last_change":"2023-10-06T21:19:26.998407+0000","last_active":"2023-10-06T21:20:25.908501+0000","last_peered":"2023-10-06T21:20:25.908501+0000","last_clean":"2023-10-06T21:20:25.908501+0000","last_became_active":"2023-10-06T21:19:26.991918+0000","last_became_peered":"2023-10-06T21:19:26.991918+0000","last_unstale":"2023-10-06T21:20:25.908501+0000","last_undegraded":"2023-10-06T21:20:25.908501+0000","last_fullsized":"2023-10-06T21:20:25.908501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:05.903796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899375+0000","last_change":"2023-10-06T21:20:26.898661+0000","last_active":"2023-10-06T21:20:26.899375+0000","last_peered":"2023-10-06T21:20:26.899375+0000","last_clean":"2023-10-06T21:20:26.899375+0000","last_became_active":"2023-10-06T21:20:26.898216+0000","last_became_peered":"2023-10-06T21:20:26.898216+0000","last_unstale":"2023-10-06T21:20:26.899375+0000","last_undegraded":"2023-10-06T21:20:26.899375+0000","last_fullsized":"2023-10-06T21:20:26.899375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:55:14.473221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893480+0000","last_change":"2023-10-06T21:18:53.846107+0000","last_active":"2023-10-06T21:20:25.893480+0000","last_peered":"2023-10-06T21:20:25.893480+0000","last_clean":"2023-10-06T21:20:25.893480+0000","last_became_active":"2023-10-06T21:18:53.845841+0000","last_became_peered":"2023-10-06T21:18:53.845841+0000","last_unstale":"2023-10-06T21:20:25.893480+0000","last_undegraded":"2023-10-06T21:20:25.893480+0000","last_fullsized":"2023-10-06T21:20:25.893480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.969231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894320+0000","last_change":"2023-10-06T21:18:54.858828+0000","last_active":"2023-10-06T21:20:25.894320+0000","last_peered":"2023-10-06T21:20:25.894320+0000","last_clean":"2023-10-06T21:20:25.894320+0000","last_became_active":"2023-10-06T21:18:54.858500+0000","last_became_peered":"2023-10-06T21:18:54.858500+0000","last_unstale":"2023-10-06T21:20:25.894320+0000","last_undegraded":"2023-10-06T21:20:25.894320+0000","last_fullsized":"2023-10-06T21:20:25.894320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:46.744930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897315+0000","last_change":"2023-10-06T21:18:53.850663+0000","last_active":"2023-10-06T21:20:25.897315+0000","last_peered":"2023-10-06T21:20:25.897315+0000","last_clean":"2023-10-06T21:20:25.897315+0000","last_became_active":"2023-10-06T21:18:53.850316+0000","last_became_peered":"2023-10-06T21:18:53.850316+0000","last_unstale":"2023-10-06T21:20:25.897315+0000","last_undegraded":"2023-10-06T21:20:25.897315+0000","last_fullsized":"2023-10-06T21:20:25.897315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:32:12.653059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888820+0000","last_change":"2023-10-06T21:18:54.851377+0000","last_active":"2023-10-06T21:20:25.888820+0000","last_peered":"2023-10-06T21:20:25.888820+0000","last_clean":"2023-10-06T21:20:25.888820+0000","last_became_active":"2023-10-06T21:18:54.851072+0000","last_became_peered":"2023-10-06T21:18:54.851072+0000","last_unstale":"2023-10-06T21:20:25.888820+0000","last_undegraded":"2023-10-06T21:20:25.888820+0000","last_fullsized":"2023-10-06T21:20:25.888820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:14:57.246710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":226,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988250+0000","last_change":"2023-10-06T21:18:53.851374+0000","last_active":"2023-10-06T21:20:40.988250+0000","last_peered":"2023-10-06T21:20:40.988250+0000","last_clean":"2023-10-06T21:20:40.988250+0000","last_became_active":"2023-10-06T21:18:53.850873+0000","last_became_peered":"2023-10-06T21:18:53.850873+0000","last_unstale":"2023-10-06T21:20:40.988250+0000","last_undegraded":"2023-10-06T21:20:40.988250+0000","last_fullsized":"2023-10-06T21:20:40.988250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:08.653335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":203,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894451+0000","last_change":"2023-10-06T21:18:54.856977+0000","last_active":"2023-10-06T21:20:25.894451+0000","last_peered":"2023-10-06T21:20:25.894451+0000","last_clean":"2023-10-06T21:20:25.894451+0000","last_became_active":"2023-10-06T21:18:54.856486+0000","last_became_peered":"2023-10-06T21:18:54.856486+0000","last_unstale":"2023-10-06T21:20:25.894451+0000","last_undegraded":"2023-10-06T21:20:25.894451+0000","last_fullsized":"2023-10-06T21:20:25.894451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:11:43.121219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891255+0000","last_change":"2023-10-06T21:18:53.854235+0000","last_active":"2023-10-06T21:20:25.891255+0000","last_peered":"2023-10-06T21:20:25.891255+0000","last_clean":"2023-10-06T21:20:25.891255+0000","last_became_active":"2023-10-06T21:18:53.853856+0000","last_became_peered":"2023-10-06T21:18:53.853856+0000","last_unstale":"2023-10-06T21:20:25.891255+0000","last_undegraded":"2023-10-06T21:20:25.891255+0000","last_fullsized":"2023-10-06T21:20:25.891255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:51:02.681795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502189+0000","last_change":"2023-10-06T21:18:54.859538+0000","last_active":"2023-10-06T21:20:26.502189+0000","last_peered":"2023-10-06T21:20:26.502189+0000","last_clean":"2023-10-06T21:20:26.502189+0000","last_became_active":"2023-10-06T21:18:54.857383+0000","last_became_peered":"2023-10-06T21:18:54.857383+0000","last_unstale":"2023-10-06T21:20:26.502189+0000","last_undegraded":"2023-10-06T21:20:26.502189+0000","last_fullsized":"2023-10-06T21:20:26.502189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:56:44.964731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897849+0000","last_change":"2023-10-06T21:18:53.846041+0000","last_active":"2023-10-06T21:20:25.897849+0000","last_peered":"2023-10-06T21:20:25.897849+0000","last_clean":"2023-10-06T21:20:25.897849+0000","last_became_active":"2023-10-06T21:18:53.845063+0000","last_became_peered":"2023-10-06T21:18:53.845063+0000","last_unstale":"2023-10-06T21:20:25.897849+0000","last_undegraded":"2023-10-06T21:20:25.897849+0000","last_fullsized":"2023-10-06T21:20:25.897849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:32:05.722562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893778+0000","last_change":"2023-10-06T21:18:54.848235+0000","last_active":"2023-10-06T21:20:25.893778+0000","last_peered":"2023-10-06T21:20:25.893778+0000","last_clean":"2023-10-06T21:20:25.893778+0000","last_became_active":"2023-10-06T21:18:54.847938+0000","last_became_peered":"2023-10-06T21:18:54.847938+0000","last_unstale":"2023-10-06T21:20:25.893778+0000","last_undegraded":"2023-10-06T21:20:25.893778+0000","last_fullsized":"2023-10-06T21:20:25.893778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:38.339397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.988126+0000","last_change":"2023-10-06T21:18:53.853466+0000","last_active":"2023-10-06T21:20:40.988126+0000","last_peered":"2023-10-06T21:20:40.988126+0000","last_clean":"2023-10-06T21:20:40.988126+0000","last_became_active":"2023-10-06T21:18:53.853058+0000","last_became_peered":"2023-10-06T21:18:53.853058+0000","last_unstale":"2023-10-06T21:20:40.988126+0000","last_undegraded":"2023-10-06T21:20:40.988126+0000","last_fullsized":"2023-10-06T21:20:40.988126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:29.289867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59729,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":37,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898054+0000","last_change":"2023-10-06T21:18:54.855883+0000","last_active":"2023-10-06T21:20:25.898054+0000","last_peered":"2023-10-06T21:20:25.898054+0000","last_clean":"2023-10-06T21:20:25.898054+0000","last_became_active":"2023-10-06T21:18:54.855452+0000","last_became_peered":"2023-10-06T21:18:54.855452+0000","last_unstale":"2023-10-06T21:20:25.898054+0000","last_undegraded":"2023-10-06T21:20:25.898054+0000","last_fullsized":"2023-10-06T21:20:25.898054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:26.513459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889077+0000","last_change":"2023-10-06T21:18:53.845863+0000","last_active":"2023-10-06T21:20:25.889077+0000","last_peered":"2023-10-06T21:20:25.889077+0000","last_clean":"2023-10-06T21:20:25.889077+0000","last_became_active":"2023-10-06T21:18:53.845465+0000","last_became_peered":"2023-10-06T21:18:53.845465+0000","last_unstale":"2023-10-06T21:20:25.889077+0000","last_undegraded":"2023-10-06T21:20:25.889077+0000","last_fullsized":"2023-10-06T21:20:25.889077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:07.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893449+0000","last_change":"2023-10-06T21:18:54.846963+0000","last_active":"2023-10-06T21:20:25.893449+0000","last_peered":"2023-10-06T21:20:25.893449+0000","last_clean":"2023-10-06T21:20:25.893449+0000","last_became_active":"2023-10-06T21:18:54.846675+0000","last_became_peered":"2023-10-06T21:18:54.846675+0000","last_unstale":"2023-10-06T21:20:25.893449+0000","last_undegraded":"2023-10-06T21:20:25.893449+0000","last_fullsized":"2023-10-06T21:20:25.893449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:32:54.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897611+0000","last_change":"2023-10-06T21:18:53.851548+0000","last_active":"2023-10-06T21:20:25.897611+0000","last_peered":"2023-10-06T21:20:25.897611+0000","last_clean":"2023-10-06T21:20:25.897611+0000","last_became_active":"2023-10-06T21:18:53.850688+0000","last_became_peered":"2023-10-06T21:18:53.850688+0000","last_unstale":"2023-10-06T21:20:25.897611+0000","last_undegraded":"2023-10-06T21:20:25.897611+0000","last_fullsized":"2023-10-06T21:20:25.897611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:54.103855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.896400+0000","last_change":"2023-10-06T21:20:26.896400+0000","last_active":"2023-10-06T21:20:26.896400+0000","last_peered":"2023-10-06T21:20:26.896400+0000","last_clean":"2023-10-06T21:20:26.896400+0000","last_became_active":"2023-10-06T21:20:26.896036+0000","last_became_peered":"2023-10-06T21:20:26.896036+0000","last_unstale":"2023-10-06T21:20:26.896400+0000","last_undegraded":"2023-10-06T21:20:26.896400+0000","last_fullsized":"2023-10-06T21:20:26.896400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:30:04.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:40.030597+0000","last_change":"2023-10-06T21:18:53.847978+0000","last_active":"2023-10-06T21:20:40.030597+0000","last_peered":"2023-10-06T21:20:40.030597+0000","last_clean":"2023-10-06T21:20:40.030597+0000","last_became_active":"2023-10-06T21:18:53.847495+0000","last_became_peered":"2023-10-06T21:18:53.847495+0000","last_unstale":"2023-10-06T21:20:40.030597+0000","last_undegraded":"2023-10-06T21:20:40.030597+0000","last_fullsized":"2023-10-06T21:20:40.030597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:34.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891007+0000","last_change":"2023-10-06T21:18:54.850447+0000","last_active":"2023-10-06T21:20:25.891007+0000","last_peered":"2023-10-06T21:20:25.891007+0000","last_clean":"2023-10-06T21:20:25.891007+0000","last_became_active":"2023-10-06T21:18:54.850148+0000","last_became_peered":"2023-10-06T21:18:54.850148+0000","last_unstale":"2023-10-06T21:20:25.891007+0000","last_undegraded":"2023-10-06T21:20:25.891007+0000","last_fullsized":"2023-10-06T21:20:25.891007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:01:10.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908465+0000","last_change":"2023-10-06T21:19:26.862156+0000","last_active":"2023-10-06T21:20:25.908465+0000","last_peered":"2023-10-06T21:20:25.908465+0000","last_clean":"2023-10-06T21:20:25.908465+0000","last_became_active":"2023-10-06T21:19:26.844987+0000","last_became_peered":"2023-10-06T21:19:26.844987+0000","last_unstale":"2023-10-06T21:20:25.908465+0000","last_undegraded":"2023-10-06T21:20:25.908465+0000","last_fullsized":"2023-10-06T21:20:25.908465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:12:56.546689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897320+0000","last_change":"2023-10-06T21:18:54.848437+0000","last_active":"2023-10-06T21:20:25.897320+0000","last_peered":"2023-10-06T21:20:25.897320+0000","last_clean":"2023-10-06T21:20:25.897320+0000","last_became_active":"2023-10-06T21:18:54.848102+0000","last_became_peered":"2023-10-06T21:18:54.848102+0000","last_unstale":"2023-10-06T21:20:25.897320+0000","last_undegraded":"2023-10-06T21:20:25.897320+0000","last_fullsized":"2023-10-06T21:20:25.897320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:34:12.352850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897952+0000","last_change":"2023-10-06T21:18:53.848815+0000","last_active":"2023-10-06T21:20:25.897952+0000","last_peered":"2023-10-06T21:20:25.897952+0000","last_clean":"2023-10-06T21:20:25.897952+0000","last_became_active":"2023-10-06T21:18:53.848441+0000","last_became_peered":"2023-10-06T21:18:53.848441+0000","last_unstale":"2023-10-06T21:20:25.897952+0000","last_undegraded":"2023-10-06T21:20:25.897952+0000","last_fullsized":"2023-10-06T21:20:25.897952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:38:41.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908587+0000","last_change":"2023-10-06T21:18:54.860855+0000","last_active":"2023-10-06T21:20:25.908587+0000","last_peered":"2023-10-06T21:20:25.908587+0000","last_clean":"2023-10-06T21:20:25.908587+0000","last_became_active":"2023-10-06T21:18:54.860277+0000","last_became_peered":"2023-10-06T21:18:54.860277+0000","last_unstale":"2023-10-06T21:20:25.908587+0000","last_undegraded":"2023-10-06T21:20:25.908587+0000","last_fullsized":"2023-10-06T21:20:25.908587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:03:02.795990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889118+0000","last_change":"2023-10-06T21:18:53.856882+0000","last_active":"2023-10-06T21:20:25.889118+0000","last_peered":"2023-10-06T21:20:25.889118+0000","last_clean":"2023-10-06T21:20:25.889118+0000","last_became_active":"2023-10-06T21:18:53.856562+0000","last_became_peered":"2023-10-06T21:18:53.856562+0000","last_unstale":"2023-10-06T21:20:25.889118+0000","last_undegraded":"2023-10-06T21:20:25.889118+0000","last_fullsized":"2023-10-06T21:20:25.889118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:56.945455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908797+0000","last_change":"2023-10-06T21:18:54.852419+0000","last_active":"2023-10-06T21:20:25.908797+0000","last_peered":"2023-10-06T21:20:25.908797+0000","last_clean":"2023-10-06T21:20:25.908797+0000","last_became_active":"2023-10-06T21:18:54.852041+0000","last_became_peered":"2023-10-06T21:18:54.852041+0000","last_unstale":"2023-10-06T21:20:25.908797+0000","last_undegraded":"2023-10-06T21:20:25.908797+0000","last_fullsized":"2023-10-06T21:20:25.908797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:19:15.651469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.006783+0000","last_change":"2023-10-06T21:18:53.851909+0000","last_active":"2023-10-06T21:20:37.006783+0000","last_peered":"2023-10-06T21:20:37.006783+0000","last_clean":"2023-10-06T21:20:37.006783+0000","last_became_active":"2023-10-06T21:18:53.851461+0000","last_became_peered":"2023-10-06T21:18:53.851461+0000","last_unstale":"2023-10-06T21:20:37.006783+0000","last_undegraded":"2023-10-06T21:20:37.006783+0000","last_fullsized":"2023-10-06T21:20:37.006783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:20.852813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908595+0000","last_change":"2023-10-06T21:18:54.851325+0000","last_active":"2023-10-06T21:20:25.908595+0000","last_peered":"2023-10-06T21:20:25.908595+0000","last_clean":"2023-10-06T21:20:25.908595+0000","last_became_active":"2023-10-06T21:18:54.850980+0000","last_became_peered":"2023-10-06T21:18:54.850980+0000","last_unstale":"2023-10-06T21:20:25.908595+0000","last_undegraded":"2023-10-06T21:20:25.908595+0000","last_fullsized":"2023-10-06T21:20:25.908595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:20:22.030691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897427+0000","last_change":"2023-10-06T21:18:54.844981+0000","last_active":"2023-10-06T21:20:25.897427+0000","last_peered":"2023-10-06T21:20:25.897427+0000","last_clean":"2023-10-06T21:20:25.897427+0000","last_became_active":"2023-10-06T21:18:54.841980+0000","last_became_peered":"2023-10-06T21:18:54.841980+0000","last_unstale":"2023-10-06T21:20:25.897427+0000","last_undegraded":"2023-10-06T21:20:25.897427+0000","last_fullsized":"2023-10-06T21:20:25.897427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:51.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:37.023875+0000","last_change":"2023-10-06T21:18:53.838809+0000","last_active":"2023-10-06T21:20:37.023875+0000","last_peered":"2023-10-06T21:20:37.023875+0000","last_clean":"2023-10-06T21:20:37.023875+0000","last_became_active":"2023-10-06T21:18:53.838509+0000","last_became_peered":"2023-10-06T21:18:53.838509+0000","last_unstale":"2023-10-06T21:20:37.023875+0000","last_undegraded":"2023-10-06T21:20:37.023875+0000","last_fullsized":"2023-10-06T21:20:37.023875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:08:41.958978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894894+0000","last_change":"2023-10-06T21:18:54.860475+0000","last_active":"2023-10-06T21:20:25.894894+0000","last_peered":"2023-10-06T21:20:25.894894+0000","last_clean":"2023-10-06T21:20:25.894894+0000","last_became_active":"2023-10-06T21:18:54.860125+0000","last_became_peered":"2023-10-06T21:18:54.860125+0000","last_unstale":"2023-10-06T21:20:25.894894+0000","last_undegraded":"2023-10-06T21:20:25.894894+0000","last_fullsized":"2023-10-06T21:20:25.894894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:56:22.978323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889049+0000","last_change":"2023-10-06T21:18:53.853299+0000","last_active":"2023-10-06T21:20:25.889049+0000","last_peered":"2023-10-06T21:20:25.889049+0000","last_clean":"2023-10-06T21:20:25.889049+0000","last_became_active":"2023-10-06T21:18:53.852812+0000","last_became_peered":"2023-10-06T21:18:53.852812+0000","last_unstale":"2023-10-06T21:20:25.889049+0000","last_undegraded":"2023-10-06T21:20:25.889049+0000","last_fullsized":"2023-10-06T21:20:25.889049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:25:37.310029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":323,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891086+0000","last_change":"2023-10-06T21:17:35.809753+0000","last_active":"2023-10-06T21:20:25.891086+0000","last_peered":"2023-10-06T21:20:25.891086+0000","last_clean":"2023-10-06T21:20:25.891086+0000","last_became_active":"2023-10-06T21:17:35.809314+0000","last_became_peered":"2023-10-06T21:17:35.809314+0000","last_unstale":"2023-10-06T21:20:25.891086+0000","last_undegraded":"2023-10-06T21:20:25.891086+0000","last_fullsized":"2023-10-06T21:20:25.891086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:14:04.689774+0000","last_clean_scrub_stamp":"2023-10-06T21:14:04.689774+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:46:54.305237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893217+0000","last_change":"2023-10-06T21:18:54.841443+0000","last_active":"2023-10-06T21:20:25.893217+0000","last_peered":"2023-10-06T21:20:25.893217+0000","last_clean":"2023-10-06T21:20:25.893217+0000","last_became_active":"2023-10-06T21:18:54.841089+0000","last_became_peered":"2023-10-06T21:18:54.841089+0000","last_unstale":"2023-10-06T21:20:25.893217+0000","last_undegraded":"2023-10-06T21:20:25.893217+0000","last_fullsized":"2023-10-06T21:20:25.893217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.165501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908323+0000","last_change":"2023-10-06T21:19:26.864852+0000","last_active":"2023-10-06T21:20:25.908323+0000","last_peered":"2023-10-06T21:20:25.908323+0000","last_clean":"2023-10-06T21:20:25.908323+0000","last_became_active":"2023-10-06T21:19:26.846795+0000","last_became_peered":"2023-10-06T21:19:26.846795+0000","last_unstale":"2023-10-06T21:20:25.908323+0000","last_undegraded":"2023-10-06T21:20:25.908323+0000","last_fullsized":"2023-10-06T21:20:25.908323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:15:22.977641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898390+0000","last_change":"2023-10-06T21:18:53.851792+0000","last_active":"2023-10-06T21:20:25.898390+0000","last_peered":"2023-10-06T21:20:25.898390+0000","last_clean":"2023-10-06T21:20:25.898390+0000","last_became_active":"2023-10-06T21:18:53.851015+0000","last_became_peered":"2023-10-06T21:18:53.851015+0000","last_unstale":"2023-10-06T21:20:25.898390+0000","last_undegraded":"2023-10-06T21:20:25.898390+0000","last_fullsized":"2023-10-06T21:20:25.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:55.144348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893558+0000","last_change":"2023-10-06T21:18:54.850573+0000","last_active":"2023-10-06T21:20:25.893558+0000","last_peered":"2023-10-06T21:20:25.893558+0000","last_clean":"2023-10-06T21:20:25.893558+0000","last_became_active":"2023-10-06T21:18:54.850261+0000","last_became_peered":"2023-10-06T21:18:54.850261+0000","last_unstale":"2023-10-06T21:20:25.893558+0000","last_undegraded":"2023-10-06T21:20:25.893558+0000","last_fullsized":"2023-10-06T21:20:25.893558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:07.141106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:28.140981+0000","last_change":"2023-10-06T21:18:53.859231+0000","last_active":"2023-10-06T21:20:28.140981+0000","last_peered":"2023-10-06T21:20:28.140981+0000","last_clean":"2023-10-06T21:20:28.140981+0000","last_became_active":"2023-10-06T21:18:53.858870+0000","last_became_peered":"2023-10-06T21:18:53.858870+0000","last_unstale":"2023-10-06T21:20:28.140981+0000","last_undegraded":"2023-10-06T21:20:28.140981+0000","last_fullsized":"2023-10-06T21:20:28.140981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:53:20.545076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895118+0000","last_change":"2023-10-06T21:18:54.857420+0000","last_active":"2023-10-06T21:20:25.895118+0000","last_peered":"2023-10-06T21:20:25.895118+0000","last_clean":"2023-10-06T21:20:25.895118+0000","last_became_active":"2023-10-06T21:18:54.857026+0000","last_became_peered":"2023-10-06T21:18:54.857026+0000","last_unstale":"2023-10-06T21:20:25.895118+0000","last_undegraded":"2023-10-06T21:20:25.895118+0000","last_fullsized":"2023-10-06T21:20:25.895118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:38:27.672493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889178+0000","last_change":"2023-10-06T21:18:53.848415+0000","last_active":"2023-10-06T21:20:25.889178+0000","last_peered":"2023-10-06T21:20:25.889178+0000","last_clean":"2023-10-06T21:20:25.889178+0000","last_became_active":"2023-10-06T21:18:53.848012+0000","last_became_peered":"2023-10-06T21:18:53.848012+0000","last_unstale":"2023-10-06T21:20:25.889178+0000","last_undegraded":"2023-10-06T21:20:25.889178+0000","last_fullsized":"2023-10-06T21:20:25.889178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:26:37.554162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502163+0000","last_change":"2023-10-06T21:19:26.841631+0000","last_active":"2023-10-06T21:20:26.502163+0000","last_peered":"2023-10-06T21:20:26.502163+0000","last_clean":"2023-10-06T21:20:26.502163+0000","last_became_active":"2023-10-06T21:19:26.841352+0000","last_became_peered":"2023-10-06T21:19:26.841352+0000","last_unstale":"2023-10-06T21:20:26.502163+0000","last_undegraded":"2023-10-06T21:20:26.502163+0000","last_fullsized":"2023-10-06T21:20:26.502163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:26:07.482368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893584+0000","last_change":"2023-10-06T21:18:53.844804+0000","last_active":"2023-10-06T21:20:25.893584+0000","last_peered":"2023-10-06T21:20:25.893584+0000","last_clean":"2023-10-06T21:20:25.893584+0000","last_became_active":"2023-10-06T21:18:53.844323+0000","last_became_peered":"2023-10-06T21:18:53.844323+0000","last_unstale":"2023-10-06T21:20:25.893584+0000","last_undegraded":"2023-10-06T21:20:25.893584+0000","last_fullsized":"2023-10-06T21:20:25.893584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:38:19.234298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897765+0000","last_change":"2023-10-06T21:18:54.845548+0000","last_active":"2023-10-06T21:20:25.897765+0000","last_peered":"2023-10-06T21:20:25.897765+0000","last_clean":"2023-10-06T21:20:25.897765+0000","last_became_active":"2023-10-06T21:18:54.845181+0000","last_became_peered":"2023-10-06T21:18:54.845181+0000","last_unstale":"2023-10-06T21:20:25.897765+0000","last_undegraded":"2023-10-06T21:20:25.897765+0000","last_fullsized":"2023-10-06T21:20:25.897765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:08.335565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894378+0000","last_change":"2023-10-06T21:18:53.852868+0000","last_active":"2023-10-06T21:20:25.894378+0000","last_peered":"2023-10-06T21:20:25.894378+0000","last_clean":"2023-10-06T21:20:25.894378+0000","last_became_active":"2023-10-06T21:18:53.852459+0000","last_became_peered":"2023-10-06T21:18:53.852459+0000","last_unstale":"2023-10-06T21:20:25.894378+0000","last_undegraded":"2023-10-06T21:20:25.894378+0000","last_fullsized":"2023-10-06T21:20:25.894378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:34:54.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908495+0000","last_change":"2023-10-06T21:18:54.851790+0000","last_active":"2023-10-06T21:20:25.908495+0000","last_peered":"2023-10-06T21:20:25.908495+0000","last_clean":"2023-10-06T21:20:25.908495+0000","last_became_active":"2023-10-06T21:18:54.851321+0000","last_became_peered":"2023-10-06T21:18:54.851321+0000","last_unstale":"2023-10-06T21:20:25.908495+0000","last_undegraded":"2023-10-06T21:20:25.908495+0000","last_fullsized":"2023-10-06T21:20:25.908495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:06:53.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:27.001025+0000","last_change":"2023-10-06T21:18:53.843489+0000","last_active":"2023-10-06T21:20:27.001025+0000","last_peered":"2023-10-06T21:20:27.001025+0000","last_clean":"2023-10-06T21:20:27.001025+0000","last_became_active":"2023-10-06T21:18:53.842871+0000","last_became_peered":"2023-10-06T21:18:53.842871+0000","last_unstale":"2023-10-06T21:20:27.001025+0000","last_undegraded":"2023-10-06T21:20:27.001025+0000","last_fullsized":"2023-10-06T21:20:27.001025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:33:21.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894653+0000","last_change":"2023-10-06T21:18:54.852752+0000","last_active":"2023-10-06T21:20:25.894653+0000","last_peered":"2023-10-06T21:20:25.894653+0000","last_clean":"2023-10-06T21:20:25.894653+0000","last_became_active":"2023-10-06T21:18:54.852406+0000","last_became_peered":"2023-10-06T21:18:54.852406+0000","last_unstale":"2023-10-06T21:20:25.894653+0000","last_undegraded":"2023-10-06T21:20:25.894653+0000","last_fullsized":"2023-10-06T21:20:25.894653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:34.432924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.898404+0000","last_change":"2023-10-06T21:18:53.849320+0000","last_active":"2023-10-06T21:20:25.898404+0000","last_peered":"2023-10-06T21:20:25.898404+0000","last_clean":"2023-10-06T21:20:25.898404+0000","last_became_active":"2023-10-06T21:18:53.848943+0000","last_became_peered":"2023-10-06T21:18:53.848943+0000","last_unstale":"2023-10-06T21:20:25.898404+0000","last_undegraded":"2023-10-06T21:20:25.898404+0000","last_fullsized":"2023-10-06T21:20:25.898404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:14:54.806565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891356+0000","last_change":"2023-10-06T21:18:54.854559+0000","last_active":"2023-10-06T21:20:25.891356+0000","last_peered":"2023-10-06T21:20:25.891356+0000","last_clean":"2023-10-06T21:20:25.891356+0000","last_became_active":"2023-10-06T21:18:54.854237+0000","last_became_peered":"2023-10-06T21:18:54.854237+0000","last_unstale":"2023-10-06T21:20:25.891356+0000","last_undegraded":"2023-10-06T21:20:25.891356+0000","last_fullsized":"2023-10-06T21:20:25.891356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:13.657759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:42.028741+0000","last_change":"2023-10-06T21:18:53.847685+0000","last_active":"2023-10-06T21:20:42.028741+0000","last_peered":"2023-10-06T21:20:42.028741+0000","last_clean":"2023-10-06T21:20:42.028741+0000","last_became_active":"2023-10-06T21:18:53.847234+0000","last_became_peered":"2023-10-06T21:18:53.847234+0000","last_unstale":"2023-10-06T21:20:42.028741+0000","last_undegraded":"2023-10-06T21:20:42.028741+0000","last_fullsized":"2023-10-06T21:20:42.028741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:24:37.091753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1669,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891410+0000","last_change":"2023-10-06T21:18:54.857924+0000","last_active":"2023-10-06T21:20:25.891410+0000","last_peered":"2023-10-06T21:20:25.891410+0000","last_clean":"2023-10-06T21:20:25.891410+0000","last_became_active":"2023-10-06T21:18:54.857644+0000","last_became_peered":"2023-10-06T21:18:54.857644+0000","last_unstale":"2023-10-06T21:20:25.891410+0000","last_undegraded":"2023-10-06T21:20:25.891410+0000","last_fullsized":"2023-10-06T21:20:25.891410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:22:14.935988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.897736+0000","last_change":"2023-10-06T21:18:53.846298+0000","last_active":"2023-10-06T21:20:25.897736+0000","last_peered":"2023-10-06T21:20:25.897736+0000","last_clean":"2023-10-06T21:20:25.897736+0000","last_became_active":"2023-10-06T21:18:53.845649+0000","last_became_peered":"2023-10-06T21:18:53.845649+0000","last_unstale":"2023-10-06T21:20:25.897736+0000","last_undegraded":"2023-10-06T21:20:25.897736+0000","last_fullsized":"2023-10-06T21:20:25.897736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:56:01.014198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891052+0000","last_change":"2023-10-06T21:18:54.852501+0000","last_active":"2023-10-06T21:20:25.891052+0000","last_peered":"2023-10-06T21:20:25.891052+0000","last_clean":"2023-10-06T21:20:25.891052+0000","last_became_active":"2023-10-06T21:18:54.852113+0000","last_became_peered":"2023-10-06T21:18:54.852113+0000","last_unstale":"2023-10-06T21:20:25.891052+0000","last_undegraded":"2023-10-06T21:20:25.891052+0000","last_fullsized":"2023-10-06T21:20:25.891052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:28:34.066499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893514+0000","last_change":"2023-10-06T21:19:26.851061+0000","last_active":"2023-10-06T21:20:25.893514+0000","last_peered":"2023-10-06T21:20:25.893514+0000","last_clean":"2023-10-06T21:20:25.893514+0000","last_became_active":"2023-10-06T21:19:26.845797+0000","last_became_peered":"2023-10-06T21:19:26.845797+0000","last_unstale":"2023-10-06T21:20:25.893514+0000","last_undegraded":"2023-10-06T21:20:25.893514+0000","last_fullsized":"2023-10-06T21:20:25.893514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:52.812555+0000","last_clean_scrub_stamp":"2023-10-06T21:18:52.812555+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:48.212793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.891426+0000","last_change":"2023-10-06T21:18:54.857034+0000","last_active":"2023-10-06T21:20:25.891426+0000","last_peered":"2023-10-06T21:20:25.891426+0000","last_clean":"2023-10-06T21:20:25.891426+0000","last_became_active":"2023-10-06T21:18:54.856736+0000","last_became_peered":"2023-10-06T21:18:54.856736+0000","last_unstale":"2023-10-06T21:20:25.891426+0000","last_undegraded":"2023-10-06T21:20:25.891426+0000","last_fullsized":"2023-10-06T21:20:25.891426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:46:27.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908593+0000","last_change":"2023-10-06T21:18:54.857395+0000","last_active":"2023-10-06T21:20:25.908593+0000","last_peered":"2023-10-06T21:20:25.908593+0000","last_clean":"2023-10-06T21:20:25.908593+0000","last_became_active":"2023-10-06T21:18:54.856881+0000","last_became_peered":"2023-10-06T21:18:54.856881+0000","last_unstale":"2023-10-06T21:20:25.908593+0000","last_undegraded":"2023-10-06T21:20:25.908593+0000","last_fullsized":"2023-10-06T21:20:25.908593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:52:02.398738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888944+0000","last_change":"2023-10-06T21:19:26.844504+0000","last_active":"2023-10-06T21:20:25.888944+0000","last_peered":"2023-10-06T21:20:25.888944+0000","last_clean":"2023-10-06T21:20:25.888944+0000","last_became_active":"2023-10-06T21:19:26.844105+0000","last_became_peered":"2023-10-06T21:19:26.844105+0000","last_unstale":"2023-10-06T21:20:25.888944+0000","last_undegraded":"2023-10-06T21:20:25.888944+0000","last_fullsized":"2023-10-06T21:20:25.888944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:08.655868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.888876+0000","last_change":"2023-10-06T21:18:54.852968+0000","last_active":"2023-10-06T21:20:25.888876+0000","last_peered":"2023-10-06T21:20:25.888876+0000","last_clean":"2023-10-06T21:20:25.888876+0000","last_became_active":"2023-10-06T21:18:54.852663+0000","last_became_peered":"2023-10-06T21:18:54.852663+0000","last_unstale":"2023-10-06T21:20:25.888876+0000","last_undegraded":"2023-10-06T21:20:25.888876+0000","last_fullsized":"2023-10-06T21:20:25.888876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:16:42.493962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.502036+0000","last_change":"2023-10-06T21:18:54.861106+0000","last_active":"2023-10-06T21:20:26.502036+0000","last_peered":"2023-10-06T21:20:26.502036+0000","last_clean":"2023-10-06T21:20:26.502036+0000","last_became_active":"2023-10-06T21:18:54.860666+0000","last_became_peered":"2023-10-06T21:18:54.860666+0000","last_unstale":"2023-10-06T21:20:26.502036+0000","last_undegraded":"2023-10-06T21:20:26.502036+0000","last_fullsized":"2023-10-06T21:20:26.502036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:50:21.857449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893458+0000","last_change":"2023-10-06T21:18:54.852671+0000","last_active":"2023-10-06T21:20:25.893458+0000","last_peered":"2023-10-06T21:20:25.893458+0000","last_clean":"2023-10-06T21:20:25.893458+0000","last_became_active":"2023-10-06T21:18:54.852255+0000","last_became_peered":"2023-10-06T21:18:54.852255+0000","last_unstale":"2023-10-06T21:20:25.893458+0000","last_undegraded":"2023-10-06T21:20:25.893458+0000","last_fullsized":"2023-10-06T21:20:25.893458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:30:57.481080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908546+0000","last_change":"2023-10-06T21:18:54.854679+0000","last_active":"2023-10-06T21:20:25.908546+0000","last_peered":"2023-10-06T21:20:25.908546+0000","last_clean":"2023-10-06T21:20:25.908546+0000","last_became_active":"2023-10-06T21:18:54.854323+0000","last_became_peered":"2023-10-06T21:18:54.854323+0000","last_unstale":"2023-10-06T21:20:25.908546+0000","last_undegraded":"2023-10-06T21:20:25.908546+0000","last_fullsized":"2023-10-06T21:20:25.908546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:04.591995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.889295+0000","last_change":"2023-10-06T21:18:54.855629+0000","last_active":"2023-10-06T21:20:25.889295+0000","last_peered":"2023-10-06T21:20:25.889295+0000","last_clean":"2023-10-06T21:20:25.889295+0000","last_became_active":"2023-10-06T21:18:54.855268+0000","last_became_peered":"2023-10-06T21:18:54.855268+0000","last_unstale":"2023-10-06T21:20:25.889295+0000","last_undegraded":"2023-10-06T21:20:25.889295+0000","last_fullsized":"2023-10-06T21:20:25.889295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:41:05.425565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.895021+0000","last_change":"2023-10-06T21:18:54.864368+0000","last_active":"2023-10-06T21:20:25.895021+0000","last_peered":"2023-10-06T21:20:25.895021+0000","last_clean":"2023-10-06T21:20:25.895021+0000","last_became_active":"2023-10-06T21:18:54.864012+0000","last_became_peered":"2023-10-06T21:18:54.864012+0000","last_unstale":"2023-10-06T21:20:25.895021+0000","last_undegraded":"2023-10-06T21:20:25.895021+0000","last_fullsized":"2023-10-06T21:20:25.895021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:39:21.176026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.890975+0000","last_change":"2023-10-06T21:19:26.847714+0000","last_active":"2023-10-06T21:20:25.890975+0000","last_peered":"2023-10-06T21:20:25.890975+0000","last_clean":"2023-10-06T21:20:25.890975+0000","last_became_active":"2023-10-06T21:19:26.847302+0000","last_became_peered":"2023-10-06T21:19:26.847302+0000","last_unstale":"2023-10-06T21:20:25.890975+0000","last_undegraded":"2023-10-06T21:20:25.890975+0000","last_fullsized":"2023-10-06T21:20:25.890975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:10:24.079203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.894276+0000","last_change":"2023-10-06T21:18:54.859074+0000","last_active":"2023-10-06T21:20:25.894276+0000","last_peered":"2023-10-06T21:20:25.894276+0000","last_clean":"2023-10-06T21:20:25.894276+0000","last_became_active":"2023-10-06T21:18:54.858718+0000","last_became_peered":"2023-10-06T21:18:54.858718+0000","last_unstale":"2023-10-06T21:20:25.894276+0000","last_undegraded":"2023-10-06T21:20:25.894276+0000","last_fullsized":"2023-10-06T21:20:25.894276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:50.117849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T21:20:26.899339+0000","last_change":"2023-10-06T21:20:26.897996+0000","last_active":"2023-10-06T21:20:26.899339+0000","last_peered":"2023-10-06T21:20:26.899339+0000","last_clean":"2023-10-06T21:20:26.899339+0000","last_became_active":"2023-10-06T21:20:26.897555+0000","last_became_peered":"2023-10-06T21:20:26.897555+0000","last_unstale":"2023-10-06T21:20:26.899339+0000","last_undegraded":"2023-10-06T21:20:26.899339+0000","last_fullsized":"2023-10-06T21:20:26.899339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:59:28.968861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.893964+0000","last_change":"2023-10-06T21:18:54.852641+0000","last_active":"2023-10-06T21:20:25.893964+0000","last_peered":"2023-10-06T21:20:25.893964+0000","last_clean":"2023-10-06T21:20:25.893964+0000","last_became_active":"2023-10-06T21:18:54.852178+0000","last_became_peered":"2023-10-06T21:18:54.852178+0000","last_unstale":"2023-10-06T21:20:25.893964+0000","last_undegraded":"2023-10-06T21:20:25.893964+0000","last_fullsized":"2023-10-06T21:20:25.893964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:08:02.655046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T21:20:25.908609+0000","last_change":"2023-10-06T21:18:54.861200+0000","last_active":"2023-10-06T21:20:25.908609+0000","last_peered":"2023-10-06T21:20:25.908609+0000","last_clean":"2023-10-06T21:20:25.908609+0000","last_became_active":"2023-10-06T21:18:54.860840+0000","last_became_peered":"2023-10-06T21:18:54.860840+0000","last_unstale":"2023-10-06T21:20:25.908609+0000","last_undegraded":"2023-10-06T21:20:25.908609+0000","last_fullsized":"2023-10-06T21:20:25.908609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:18:53.818445+0000","last_clean_scrub_stamp":"2023-10-06T21:18:53.818445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:15.431706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246838,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68678,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1055,"num_read_kb":603,"num_write":253,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":210354,"data_compressed":5511,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478543,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3819999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.59699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.038,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.59799999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.75}]},{"osd":2,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93999999999999995,"5min":1.536,"15min":1.536},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.016,"5min":1.23,"15min":1.23},"last":0.87},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.083,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73199999999999998,"15min":0.747},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86199999999999999,"5min":2.036,"15min":2.036},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85799999999999998},"min":{"1min":0.68600000000000005,"5min":0.626,"15min":0.626},"max":{"1min":1.0600000000000001,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":1.631,"15min":1.631},"last":0.66300000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.71099999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2949999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.627},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.65300000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.302,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93600000000000005,"15min":0.93999999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.165,"5min":1.671,"15min":1.671},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.96799999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2749999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.72199999999999998}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1129395,"data_compressed":49169,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.177,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.256,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.57199999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.024,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.111,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.64300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.288,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.87}]},{"osd":7,"last update":"Fri Oct 6 21:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.496},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.71099999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3129999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79300000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.1279999999999999,"15min":1.0940000000000001},"min":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3440000000000001,"5min":10.827999999999999,"15min":10.827999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.63500000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.0089999999999999,"5min":10.808999999999999,"15min":10.808999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0620000000000001,"15min":1.0820000000000001},"min":{"1min":0.70899999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.155,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89500000000000002,"15min":0.88800000000000001},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.143,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71699999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477862,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":0.749,"15min":0.749},"last":0.376},{"interface":"front","average":{"1min":0.502,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.131,"15min":1.131},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.74299999999999999,"15min":0.76500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.67000000000000004,"15min":0.71799999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":1.21,"15min":1.21},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.879,"15min":0.92900000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.242,"15min":1.242},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.85299999999999998,"15min":0.89500000000000002},"min":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.222,"15min":1.222},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.88100000000000001,"15min":0.91900000000000004},"min":{"1min":0.63,"5min":0.62,"15min":0.62},"max":{"1min":1.262,"5min":1.288,"15min":1.288},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96899999999999997,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.91900000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.755},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.88100000000000001,"15min":0.93899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0609999999999999,"5min":1.282,"15min":1.282},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98199999999999998,"15min":1.042},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1180000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":1.022},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0980000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.242,"5min":1.379,"15min":1.379},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":1.044},"min":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1859999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.99099999999999999,"15min":1.0389999999999999},"min":{"1min":0.68200000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2849999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.022,"15min":1.042},"min":{"1min":0.69599999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":4.3970000000000002,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":0.93100000000000005}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477797,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.1599999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.435}]},{"osd":3,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.749,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.196,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.51000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.60199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.167,"5min":0.92100000000000004,"15min":0.89000000000000001},"min":{"1min":0.58399999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.63100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2450000000000001,"5min":1.468,"15min":1.468},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.89800000000000002,"15min":0.88900000000000001},"min":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.266,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.97499999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.85499999999999998},"min":{"1min":0.65400000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.171,"5min":0.96399999999999997,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88400000000000001,"15min":0.85199999999999998},"min":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94999999999999996,"15min":0.91100000000000003},"min":{"1min":0.65400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.94699999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.54600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1072070,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86399999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.58399999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.57299999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":1.044,"15min":1.044},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66300000000000003},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.62},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.59799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0309999999999999,"5min":1.073,"15min":1.073},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.82999999999999996,"15min":0.77700000000000002},"min":{"1min":0.65100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95299999999999996,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Oct 6 21:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91800000000000004,"5min":1.444,"15min":1.444},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98199999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":1.0469999999999999},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.119,"15min":1.119},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.878,"15min":0.88400000000000001},"min":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.198,"15min":1.198},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.85799999999999998,"15min":0.85399999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0049999999999999,"5min":1.034,"15min":1.034},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.91000000000000003,"15min":0.92600000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1819999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 21:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0069999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":478084,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.871},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.80900000000000005,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.66100000000000003},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.84099999999999997,"5min":1.887,"15min":1.887},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":0.84999999999999998,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":10.499000000000001,"5min":10.499000000000001,"15min":10.499000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.752},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.097,"5min":0.86199999999999999,"15min":0.81299999999999994},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.441},"max":{"1min":5.6870000000000003,"5min":5.6870000000000003,"15min":5.6870000000000003},"last":0.99299999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.53300000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.568,"5min":10.568,"15min":10.568},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.827,"15min":1.827},"last":0.75}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.54900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":0.96199999999999997,"15min":0.89000000000000001},"min":{"1min":0.55600000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.551,"5min":10.551,"15min":10.551},"last":0.95699999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.399,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.997,"5min":1.905,"15min":1.905},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.337,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.024},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.93700000000000006,"5min":1.232,"15min":1.232},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.0640000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":538720,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.67300000000000004,"15min":0.629},"min":{"1min":0.5,"5min":0.373,"15min":0.373},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.78900000000000003,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":15.016999999999999,"5min":15.016999999999999,"15min":15.016999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.72799999999999998,"15min":0.65600000000000003},"min":{"1min":0.57299999999999995,"5min":0.39300000000000002,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69699999999999995,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98699999999999999,"5min":1.619,"15min":1.619},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":6.3419999999999996,"5min":6.3419999999999996,"15min":6.3419999999999996},"last":0.87},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64800000000000002,"15min":0.57199999999999995},"min":{"1min":0.54200000000000004,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.63400000000000001,"15min":0.55400000000000005},"min":{"1min":0.53700000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.66100000000000003},"min":{"1min":0.59599999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.079,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.79100000000000004,"15min":0.68000000000000005},"min":{"1min":0.69399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":3.149,"15min":3.149},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82299999999999995,"15min":0.76100000000000001},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.0800000000000001,"5min":5.0910000000000002,"15min":5.0910000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.64400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.9410000000000001,"5min":3.218,"15min":3.218},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94099999999999995,"5min":8.7620000000000005,"15min":8.7620000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0660000000000001,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":0.629}]},{"osd":8,"last update":"Fri Oct 6 21:19:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.83499999999999996,"15min":0.78900000000000003},"min":{"1min":0.73999999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.91800000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.92300000000000004,"15min":0.84499999999999997},"min":{"1min":0.82599999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2989999999999999,"5min":7.1760000000000002,"15min":7.1760000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.96999999999999997,"15min":0.89500000000000002},"min":{"1min":0.80400000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":0.93799999999999994,"15min":0.89400000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":6.0419999999999998,"5min":6.0419999999999998,"15min":6.0419999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.016,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.95799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":479417,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":2.6030000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71499999999999997,"15min":0.72799999999999998},"min":{"1min":0.51400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93100000000000005,"5min":1.204,"15min":2.5419999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.749},"min":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":1.1759999999999999,"5min":1.228,"15min":2.5150000000000001},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.74299999999999999,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74099999999999999,"15min":0.69799999999999995},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96699999999999997,"5min":1.139,"15min":1.139},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.71599999999999997,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.772,"15min":1.772},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.85799999999999998,"15min":0.83499999999999996},"min":{"1min":0.65500000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.159,"5min":1.228,"15min":1.228},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.89900000000000002,"15min":0.91000000000000003},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.163,"5min":1.286,"15min":1.286},"last":0.872},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.98999999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0329999999999999,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.94899999999999995}]},{"osd":11,"last update":"Fri Oct 6 21:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.71799999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.127,"5min":1.302,"15min":1.302},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.89500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":539166,"data_compressed":13437,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.56799999999999995},"min":{"1min":0.499,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0620000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.57499999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84399999999999997,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.627},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.137,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68899999999999995,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.048,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Oct 6 21:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.70699999999999996,"15min":0.64300000000000002},"min":{"1min":0.51600000000000001,"5min":0.39800000000000002,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.59799999999999998,"15min":0.51000000000000001},"min":{"1min":0.48999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61899999999999999,"15min":0.51200000000000001},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1140000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.83499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.873}]},{"osd":8,"last update":"Fri Oct 6 21:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75700000000000001,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.123,"5min":1.298,"15min":1.298},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1220000000000001,"5min":1.266,"15min":1.266},"last":0.871}]},{"osd":9,"last update":"Fri Oct 6 21:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90000000000000002,"15min":0.88200000000000001},"min":{"1min":0.73899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.85799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.194},"last":0.95899999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.874,"15min":0.85599999999999998},"min":{"1min":0.60899999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.121,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.95399999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.262,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.024}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0549999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477416,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45000000000000001,"15min":0.40999999999999998},"min":{"1min":0.34699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71299999999999997,"15min":0.65400000000000003},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":1.0009999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.75,"15min":0.67200000000000004},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.1799999999999997,"5min":5.1799999999999997,"15min":5.1799999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:20:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72099999999999997,"15min":0.67900000000000005},"min":{"1min":0.51000000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.181,"5min":1.206,"15min":1.206},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98799999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.125,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92700000000000005,"5min":1.26,"15min":1.26},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.254,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96499999999999997,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.60699999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.88100000000000001,"15min":0.90700000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.47099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.93400000000000005,"15min":0.94099999999999995},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.113,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.75},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72499999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478470,"data_compressed":11600,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1160000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.40100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":0.874,"15min":0.874},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.5,"15min":0.43099999999999999},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97699999999999998,"5min":1.601,"15min":1.601},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.874},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.74399999999999999,"15min":0.66400000000000003},"min":{"1min":0.53900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.53300000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.173,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81100000000000005,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0720000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.73799999999999999},"min":{"1min":0.59199999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.55200000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.758,"15min":1.758},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":1.018},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.68799999999999994},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0640000000000001,"5min":1.127,"15min":1.127},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83599999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0629999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86799999999999999,"15min":0.84999999999999998},"min":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.86699999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.044,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.92300000000000004},"min":{"1min":0.72699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.081,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.89800000000000002},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.66800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.042,"5min":1.093,"15min":1.093},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85799999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1070550,"data_compressed":47332,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.66800000000000004},"min":{"1min":0.58199999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1799999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.45900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.093}]},{"osd":3,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.151},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.69499999999999995},"min":{"1min":0.61299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.999,"5min":1.036,"15min":1.036},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77700000000000002,"15min":0.71599999999999997},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1040000000000001,"5min":1.196,"15min":1.196},"last":1.258}]},{"osd":6,"last update":"Fri Oct 6 21:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.78000000000000003,"15min":0.753},"min":{"1min":0.64800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.347},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.72899999999999998,"15min":0.70399999999999996},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.196}]},{"osd":7,"last update":"Fri Oct 6 21:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.86199999999999999,"15min":0.84199999999999997},"min":{"1min":0.77600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.103,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.401},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80500000000000005,"15min":0.77300000000000002},"min":{"1min":0.69099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.232,"15min":1.232},"last":1.329}]},{"osd":8,"last update":"Fri Oct 6 21:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":1.383},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.4830000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85899999999999999,"15min":0.84099999999999997},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.873,"15min":0.877},"min":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.143,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.306}]},{"osd":10,"last update":"Fri Oct 6 21:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.91000000000000003,"15min":0.87},"min":{"1min":0.88700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.82399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.3660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35732,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61980,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67019,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62495,"data_compressed":1837,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:20:43.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2989832623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68102a40 msgr2=0x7fbc68109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2989832623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68102a40 0x7fbc68109360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b0a0 tx=0x7fbc6000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2989832623 shutdown_connections 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2989832623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68102a40 0x7fbc68109360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2989832623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68100ea0 0x7fbc681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2989832623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 0x7fbc68102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2989832623 >> 172.21.15.31:0/2989832623 conn(0x7fbc680fc420 msgr2=0x7fbc680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2989832623 shutdown_connections 2023-10-06T21:20:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.215+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2989832623 wait complete. 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 Processor -- start 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 -- start start 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 0x7fbc68199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68102a40 0x7fbc681a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681a0900 con 0x7fbc68100ea0 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681a0a70 con 0x7fbc68102a40 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.216+0000 7fbc6f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681a0c30 con 0x7fbc68102080 2023-10-06T21:20:44.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56778/0 (socket says 172.21.15.31:56778) 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 -- 172.21.15.31:0/2361110536 learned_addr learned my addr 172.21.15.31:0/2361110536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6cf9a700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 0x7fbc68199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6df9c700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68102a40 0x7fbc681a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 -- 172.21.15.31:0/2361110536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 msgr2=0x7fbc68199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 0x7fbc68199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 -- 172.21.15.31:0/2361110536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68102a40 msgr2=0x7fbc681a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68102a40 0x7fbc681a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.217+0000 7fbc6d79b700 1 -- 172.21.15.31:0/2361110536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc60009d00 con 0x7fbc68100ea0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc6d79b700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400d980 tx=0x7fbc6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004020 con 0x7fbc68100ea0 2023-10-06T21:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc6400f690 con 0x7fbc68100ea0 2023-10-06T21:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc681a0f10 con 0x7fbc68100ea0 2023-10-06T21:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640039e0 con 0x7fbc68100ea0 2023-10-06T21:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.218+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc681a1330 con 0x7fbc68100ea0 2023-10-06T21:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.219+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6810c610 con 0x7fbc68100ea0 2023-10-06T21:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.221+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbc6400f800 con 0x7fbc68100ea0 2023-10-06T21:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.222+0000 7fbc5e7fc700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 0x7fbc540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.222+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fbc640cc920 con 0x7fbc68100ea0 2023-10-06T21:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.223+0000 7fbc6cf9a700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 0x7fbc540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.223+0000 7fbc6cf9a700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 0x7fbc540a3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc68100bc0 tx=0x7fbc5800a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.227+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc64095ec0 con 0x7fbc68100ea0 2023-10-06T21:20:44.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:43 smithi031 ceph-mon[98539]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:43 smithi146 ceph-mon[102809]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:43 smithi142 ceph-mon[102951]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:44.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.439+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6810c820 con 0x7fbc68100ea0 2023-10-06T21:20:44.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.440+0000 7fbc5e7fc700 1 -- 172.21.15.31:0/2361110536 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22812 (secure 0 0 0) 0x7fbc640994e0 con 0x7fbc68100ea0 2023-10-06T21:20:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:44.444 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:26.886529+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 msgr2=0x7fbc540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 0x7fbc540a3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc68100bc0 tx=0x7fbc5800a3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 msgr2=0x7fbc68198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400d980 tx=0x7fbc6400dc90 comp rx=0 tx=0).stop 2023-10-06T21:20:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 shutdown_connections 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fbc540a14d0 0x7fbc540a3990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc68102a40 0x7fbc681a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc68100ea0 0x7fbc68198dc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 --2- 172.21.15.31:0/2361110536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc68102080 0x7fbc68199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 >> 172.21.15.31:0/2361110536 conn(0x7fbc680fc420 msgr2=0x7fbc681079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 shutdown_connections 2023-10-06T21:20:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.443+0000 7fbc6f9ff700 1 -- 172.21.15.31:0/2361110536 wait complete. 2023-10-06T21:20:44.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:20:44.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:20:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 -- 172.21.15.31:0/2795055010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102070 msgr2=0x7f8a6c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2795055010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102070 0x7f8a6c1024f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009990 tx=0x7f8a5c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 -- 172.21.15.31:0/2795055010 shutdown_connections 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2795055010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102a30 0x7f8a6c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2795055010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c100e90 0x7f8a6c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2795055010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102070 0x7f8a6c1024f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 -- 172.21.15.31:0/2795055010 >> 172.21.15.31:0/2795055010 conn(0x7f8a6c0fc430 msgr2=0x7f8a6c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 -- 172.21.15.31:0/2795055010 shutdown_connections 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.669+0000 7f8a737ff700 1 -- 172.21.15.31:0/2795055010 wait complete. 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.670+0000 7f8a737ff700 1 Processor -- start 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.670+0000 7f8a737ff700 1 -- start start 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102070 0x7f8a6c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c102a30 0x7f8a6c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1a08f0 con 0x7f8a6c100e90 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1a0a30 con 0x7f8a6c102070 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a737ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1a0bc0 con 0x7f8a6c102a30 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56800/0 (socket says 172.21.15.31:56800) 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 -- 172.21.15.31:0/2350752780 learned_addr learned my addr 172.21.15.31:0/2350752780 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a71d9c700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c102a30 0x7f8a6c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a70d9a700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102070 0x7f8a6c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 -- 172.21.15.31:0/2350752780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c102a30 msgr2=0x7f8a6c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c102a30 0x7f8a6c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 -- 172.21.15.31:0/2350752780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102070 msgr2=0x7f8a6c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102070 0x7f8a6c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 -- 172.21.15.31:0/2350752780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0095f0 con 0x7f8a6c100e90 2023-10-06T21:20:44.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.671+0000 7f8a7159b700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800d980 tx=0x7f8a6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.672+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800de60 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.672+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a680044b0 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.672+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c1a0ea0 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.672+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68003b40 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.672+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c1a12d0 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.673+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c10c600 con 0x7f8a6c100e90 2023-10-06T21:20:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.675+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8a68004870 con 0x7f8a6c100e90 2023-10-06T21:20:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.677+0000 7f8a627fc700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 0x7f8a580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.677+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8a680ccd90 con 0x7f8a6c100e90 2023-10-06T21:20:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.677+0000 7f8a70d9a700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 0x7f8a580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.677+0000 7f8a70d9a700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 0x7f8a580a39c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00b5c0 tx=0x7f8a5c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.678+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8a680963b0 con 0x7f8a6c100e90 2023-10-06T21:20:44.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.890+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a6c1a1520 con 0x7f8a6c100e90 2023-10-06T21:20:44.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.891+0000 7f8a627fc700 1 -- 172.21.15.31:0/2350752780 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a68017020 con 0x7f8a6c100e90 2023-10-06T21:20:44.894 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:20:44.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 msgr2=0x7f8a580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 0x7f8a580a39c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00b5c0 tx=0x7f8a5c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 msgr2=0x7f8a6c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800d980 tx=0x7f8a6800dd40 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 shutdown_connections 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8a580a1500 0x7f8a580a39c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a6c102a30 0x7f8a6c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c100e90 0x7f8a6c198d70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 --2- 172.21.15.31:0/2350752780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a6c102070 0x7f8a6c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 >> 172.21.15.31:0/2350752780 conn(0x7f8a6c0fc430 msgr2=0x7f8a6c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 shutdown_connections 2023-10-06T21:20:44.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:44.893+0000 7f8a737ff700 1 -- 172.21.15.31:0/2350752780 wait complete. 2023-10-06T21:20:44.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:20:45.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 -- 172.21.15.31:0/42335331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 msgr2=0x7f90101024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 --2- 172.21.15.31:0/42335331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f90101024f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009990 tx=0x7f8ff8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 -- 172.21.15.31:0/42335331 shutdown_connections 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 --2- 172.21.15.31:0/42335331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010102a30 0x7f9010109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 --2- 172.21.15.31:0/42335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010100e90 0x7f90101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 --2- 172.21.15.31:0/42335331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f90101024f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 -- 172.21.15.31:0/42335331 >> 172.21.15.31:0/42335331 conn(0x7f90100fc430 msgr2=0x7f90100fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 -- 172.21.15.31:0/42335331 shutdown_connections 2023-10-06T21:20:45.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.118+0000 7f90169ff700 1 -- 172.21.15.31:0/42335331 wait complete. 2023-10-06T21:20:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.119+0000 7f90169ff700 1 Processor -- start 2023-10-06T21:20:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.119+0000 7f90169ff700 1 -- start start 2023-10-06T21:20:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 0x7f9010196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 0x7f901019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901019e6d0 con 0x7f9010102070 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901019e810 con 0x7f9010102a30 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f90169ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901019e9a0 con 0x7f9010100e90 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56812/0 (socket says 172.21.15.31:56812) 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 -- 172.21.15.31:0/1856360369 learned_addr learned my addr 172.21.15.31:0/1856360369 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9014f9c700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 0x7f901019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f900ffff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 0x7f9010196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 -- 172.21.15.31:0/1856360369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 msgr2=0x7f9010196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 0x7f9010196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 -- 172.21.15.31:0/1856360369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 msgr2=0x7f901019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 0x7f901019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9007fff700 1 -- 172.21.15.31:0/1856360369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80095f0 con 0x7f9010102070 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f900ffff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 0x7f9010196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.120+0000 7f9014f9c700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 0x7f901019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f9007fff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80098e0 tx=0x7f8ff8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff801d070 con 0x7f9010102070 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901019ec20 con 0x7f9010102070 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ff800f460 con 0x7f9010102070 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8021670 con 0x7f9010102070 2023-10-06T21:20:45.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.121+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901019f050 con 0x7f9010102070 2023-10-06T21:20:45.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.122+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9010107f20 con 0x7f9010102070 2023-10-06T21:20:45.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.124+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ff800f5d0 con 0x7f9010102070 2023-10-06T21:20:45.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.125+0000 7f900dffb700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 0x7f8ffc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.126+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8ff80ce220 con 0x7f9010102070 2023-10-06T21:20:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.126+0000 7f900ffff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 0x7f8ffc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.126+0000 7f900ffff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 0x7f8ffc0a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9010101be0 tx=0x7f9000009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.127+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ff8097850 con 0x7f9010102070 2023-10-06T21:20:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:44 smithi031 ceph-mon[98539]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:44 smithi031 ceph-mon[98539]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-06T21:20:45.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:44 smithi031 ceph-mon[98539]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:44 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2361110536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:45.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:44 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2350752780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:44 smithi146 ceph-mon[102809]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:44 smithi146 ceph-mon[102809]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-06T21:20:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:44 smithi146 ceph-mon[102809]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:44 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2361110536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:45.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:44 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2350752780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.336+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f901010c790 con 0x7f9010102070 2023-10-06T21:20:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.337+0000 7f900dffb700 1 -- 172.21.15.31:0/1856360369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ff809ae70 con 0x7f9010102070 2023-10-06T21:20:45.340 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:20:45.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 msgr2=0x7f8ffc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 0x7f8ffc0a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9010101be0 tx=0x7f9000009260 comp rx=0 tx=0).stop 2023-10-06T21:20:45.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 msgr2=0x7f9010197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80098e0 tx=0x7f8ff8005ef0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 shutdown_connections 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ffc0a1480 0x7f8ffc0a3940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010102a30 0x7f901019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9010100e90 0x7f9010196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 --2- 172.21.15.31:0/1856360369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9010102070 0x7f9010197090 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.339+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 >> 172.21.15.31:0/1856360369 conn(0x7f90100fc430 msgr2=0x7f90100fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.340+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 shutdown_connections 2023-10-06T21:20:45.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.340+0000 7f90169ff700 1 -- 172.21.15.31:0/1856360369 wait complete. 2023-10-06T21:20:45.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:20:45.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:44 smithi142 ceph-mon[102951]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:44 smithi142 ceph-mon[102951]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-06T21:20:45.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:44 smithi142 ceph-mon[102951]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:20:45.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:44 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2361110536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:45.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:44 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2350752780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.564+0000 7f05b47ff700 1 -- 172.21.15.31:0/1189599862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 msgr2=0x7f05ac073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.564+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1189599862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac073d90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f059c009990 tx=0x7f059c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 -- 172.21.15.31:0/1189599862 shutdown_connections 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1189599862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac108db0 0x7f05ac10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1189599862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac074ef0 0x7f05ac073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1189599862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac073d90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 -- 172.21.15.31:0/1189599862 >> 172.21.15.31:0/1189599862 conn(0x7f05ac0fc460 msgr2=0x7f05ac0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 -- 172.21.15.31:0/1189599862 shutdown_connections 2023-10-06T21:20:45.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.565+0000 7f05b47ff700 1 -- 172.21.15.31:0/1189599862 wait complete. 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 Processor -- start 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 -- start start 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac074ef0 0x7f05ac1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 0x7f05ac1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a08e0 con 0x7f05ac073910 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a0a20 con 0x7f05ac108db0 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a0bb0 con 0x7f05ac074ef0 2023-10-06T21:20:45.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.566+0000 7f05b259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56828/0 (socket says 172.21.15.31:56828) 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 -- 172.21.15.31:0/1751190907 learned_addr learned my addr 172.21.15.31:0/1751190907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b1d9a700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac074ef0 0x7f05ac1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b2d9c700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 0x7f05ac1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 -- 172.21.15.31:0/1751190907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac074ef0 msgr2=0x7f05ac1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac074ef0 0x7f05ac1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 -- 172.21.15.31:0/1751190907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 msgr2=0x7f05ac1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 0x7f05ac1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 -- 172.21.15.31:0/1751190907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0095f0 con 0x7f05ac073910 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b2d9c700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 0x7f05ac1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:45.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05b259b700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e9a0 tx=0x7f05a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.567+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8009790 con 0x7f05ac073910 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.568+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05a800cbd0 con 0x7f05ac073910 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.568+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac1a0e90 con 0x7f05ac073910 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.568+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8018680 con 0x7f05ac073910 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.568+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac1a12b0 con 0x7f05ac073910 2023-10-06T21:20:45.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.569+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac107460 con 0x7f05ac073910 2023-10-06T21:20:45.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.571+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f05a800cd40 con 0x7f05ac073910 2023-10-06T21:20:45.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.573+0000 7f05a37fe700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 0x7f05980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.573+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f05a8014070 con 0x7f05ac073910 2023-10-06T21:20:45.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.573+0000 7f05b1d9a700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 0x7f05980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.574+0000 7f05b1d9a700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 0x7f05980a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f059c006010 tx=0x7f059c005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.575+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f05a8096980 con 0x7f05ac073910 2023-10-06T21:20:45.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.786+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05ac10ea90 con 0x7f05ac073910 2023-10-06T21:20:45.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.786+0000 7f05a37fe700 1 -- 172.21.15.31:0/1751190907 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05a8099fa0 con 0x7f05ac073910 2023-10-06T21:20:45.789 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 msgr2=0x7f05980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 0x7f05980a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f059c006010 tx=0x7f059c005a90 comp rx=0 tx=0).stop 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 msgr2=0x7f05ac198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e9a0 tx=0x7f05a800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 shutdown_connections 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f05980a1480 0x7f05980a3940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05ac108db0 0x7f05ac1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac073910 0x7f05ac198d60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 --2- 172.21.15.31:0/1751190907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05ac074ef0 0x7f05ac1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 >> 172.21.15.31:0/1751190907 conn(0x7f05ac0fc460 msgr2=0x7f05ac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 shutdown_connections 2023-10-06T21:20:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:45.789+0000 7f05b47ff700 1 -- 172.21.15.31:0/1751190907 wait complete. 2023-10-06T21:20:45.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.015+0000 7f164bdff700 1 -- 172.21.15.31:0/3052789897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 msgr2=0x7f1644102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.015+0000 7f164bdff700 1 --2- 172.21.15.31:0/3052789897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f1644102500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1634009990 tx=0x7f1634009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 -- 172.21.15.31:0/3052789897 shutdown_connections 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 --2- 172.21.15.31:0/3052789897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644102a40 0x7f1644109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 --2- 172.21.15.31:0/3052789897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644100ea0 0x7f16441012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 --2- 172.21.15.31:0/3052789897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f1644102500 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 -- 172.21.15.31:0/3052789897 >> 172.21.15.31:0/3052789897 conn(0x7f16440fc420 msgr2=0x7f16440fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 -- 172.21.15.31:0/3052789897 shutdown_connections 2023-10-06T21:20:46.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.016+0000 7f164bdff700 1 -- 172.21.15.31:0/3052789897 wait complete. 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 Processor -- start 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 -- start start 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644100ea0 0x7f1644198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644102a40 0x7f16441a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a08e0 con 0x7f1644102080 2023-10-06T21:20:46.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a0a50 con 0x7f1644100ea0 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.017+0000 7f164bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a0c10 con 0x7f1644102a40 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56838/0 (socket says 172.21.15.31:56838) 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 -- 172.21.15.31:0/2968568583 learned_addr learned my addr 172.21.15.31:0/2968568583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164a39c700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644102a40 0x7f16441a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f1649b9b700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644100ea0 0x7f1644198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 -- 172.21.15.31:0/2968568583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644102a40 msgr2=0x7f16441a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644102a40 0x7f16441a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 -- 172.21.15.31:0/2968568583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644100ea0 msgr2=0x7f1644198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644100ea0 0x7f1644198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 -- 172.21.15.31:0/2968568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340095f0 con 0x7f1644102080 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.018+0000 7f164939a700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16340098e0 tx=0x7f1634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.019+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163401d070 con 0x7f1644102080 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.019+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f163400be30 con 0x7f1644102080 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.019+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16441a0e90 con 0x7f1644102080 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.019+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634022dd0 con 0x7f1644102080 2023-10-06T21:20:46.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.019+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16441a1200 con 0x7f1644102080 2023-10-06T21:20:46.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.020+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164410c610 con 0x7f1644102080 2023-10-06T21:20:46.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.022+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1634022570 con 0x7f1644102080 2023-10-06T21:20:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.023+0000 7f163affd700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 0x7f16300a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.023+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f16340ce360 con 0x7f1644102080 2023-10-06T21:20:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.023+0000 7f1649b9b700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 0x7f16300a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.024+0000 7f1649b9b700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 0x7f16300a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f164419a270 tx=0x7f1640009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.027+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1634097990 con 0x7f1644102080 2023-10-06T21:20:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1856360369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:45 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1751190907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.286+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16441a13f0 con 0x7f1644102080 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.287+0000 7f163affd700 1 -- 172.21.15.31:0/2968568583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f163409afb0 con 0x7f1644102080 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 msgr2=0x7f16300a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 0x7f16300a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f164419a270 tx=0x7f1640009260 comp rx=0 tx=0).stop 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 msgr2=0x7f16441992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16340098e0 tx=0x7f1634004730 comp rx=0 tx=0).stop 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 shutdown_connections 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f16300a14d0 0x7f16300a3990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1644102a40 0x7f16441a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1644100ea0 0x7f1644198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 --2- 172.21.15.31:0/2968568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1644102080 0x7f16441992e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 >> 172.21.15.31:0/2968568583 conn(0x7f16440fc420 msgr2=0x7f16440fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 shutdown_connections 2023-10-06T21:20:46.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.290+0000 7f164bdff700 1 -- 172.21.15.31:0/2968568583 wait complete. 2023-10-06T21:20:46.294 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:20:46.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1856360369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:45 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1751190907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:20:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1856360369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:45 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1751190907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.512+0000 7f15e01ff700 1 -- 172.21.15.31:0/3007563578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8100ea0 msgr2=0x7f15d81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.512+0000 7f15e01ff700 1 --2- 172.21.15.31:0/3007563578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8100ea0 0x7f15d81012c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009990 tx=0x7f15d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 -- 172.21.15.31:0/3007563578 shutdown_connections 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 --2- 172.21.15.31:0/3007563578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8102a40 0x7f15d8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 --2- 172.21.15.31:0/3007563578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8100ea0 0x7f15d81012c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 --2- 172.21.15.31:0/3007563578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 -- 172.21.15.31:0/3007563578 >> 172.21.15.31:0/3007563578 conn(0x7f15d80fc420 msgr2=0x7f15d80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 -- 172.21.15.31:0/3007563578 shutdown_connections 2023-10-06T21:20:46.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.513+0000 7f15e01ff700 1 -- 172.21.15.31:0/3007563578 wait complete. 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 Processor -- start 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 -- start start 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8100ea0 0x7f15d8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a0870 con 0x7f15d8102a40 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a09e0 con 0x7f15d8102080 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15e01ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81a0ba0 con 0x7f15d8100ea0 2023-10-06T21:20:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15de79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15de79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56852/0 (socket says 172.21.15.31:56852) 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.514+0000 7f15de79c700 1 -- 172.21.15.31:0/4006187231 learned_addr learned my addr 172.21.15.31:0/4006187231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15ddf9b700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8100ea0 0x7f15d8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15dd79a700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 -- 172.21.15.31:0/4006187231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8100ea0 msgr2=0x7f15d8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8100ea0 0x7f15d8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 -- 172.21.15.31:0/4006187231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 msgr2=0x7f15d8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 -- 172.21.15.31:0/4006187231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40095f0 con 0x7f15d8102a40 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15dd79a700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15de79c700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f15d000ef30 tx=0x7f15d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0018070 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15d000f040 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d81a0e20 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.515+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000cc70 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.516+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d81a11f0 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.517+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d81933b0 con 0x7f15d8102a40 2023-10-06T21:20:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.519+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f15d0003a00 con 0x7f15d8102a40 2023-10-06T21:20:46.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.520+0000 7f15ceffd700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 0x7f15c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.520+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f15d00ccf30 con 0x7f15d8102a40 2023-10-06T21:20:46.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.520+0000 7f15ddf9b700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 0x7f15c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.521+0000 7f15ddf9b700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 0x7f15c40a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.522+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f15d00fa100 con 0x7f15d8102a40 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.737+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d810c790 con 0x7f15d8102a40 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.738+0000 7f15ceffd700 1 -- 172.21.15.31:0/4006187231 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22812 (secure 0 0 0) 0x7f15d0099ff0 con 0x7f15d8102a40 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:20:26.886529+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 msgr2=0x7f15c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 0x7f15c40a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d4005960 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 msgr2=0x7f15d81a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f15d000ef30 tx=0x7f15d000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 shutdown_connections 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f15c40a14d0 0x7f15c40a3990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d8102a40 0x7f15d81a02a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15d8100ea0 0x7f15d8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 --2- 172.21.15.31:0/4006187231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d8102080 0x7f15d8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 >> 172.21.15.31:0/4006187231 conn(0x7f15d80fc420 msgr2=0x7f15d8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 shutdown_connections 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.741+0000 7f15e01ff700 1 -- 172.21.15.31:0/4006187231 wait complete. 2023-10-06T21:20:46.763 DEBUG:tasks.fwd_scrub:Yielding 2023-10-06T21:20:46.763 INFO:teuthology.run_tasks:Running task exec... 2023-10-06T21:20:46.778 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T21:20:46.779 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:20:46.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-10-06T21:20:46.783 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-06T21:20:46.783 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:20:46.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:20:46.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 -- 172.21.15.31:0/468985302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc768106be0 msgr2=0x7fc768109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/468985302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc768106be0 0x7fc768109050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b0a0 tx=0x7fc76000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 -- 172.21.15.31:0/468985302 shutdown_connections 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/468985302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc768106be0 0x7fc768109050 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/468985302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768101980 0x7fc768103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/468985302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7681042b0 0x7fc7681066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 -- 172.21.15.31:0/468985302 >> 172.21.15.31:0/468985302 conn(0x7fc7680fb390 msgr2=0x7fc7680fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 -- 172.21.15.31:0/468985302 shutdown_connections 2023-10-06T21:20:46.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.990+0000 7fc76e9ff700 1 -- 172.21.15.31:0/468985302 wait complete. 2023-10-06T21:20:46.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.991+0000 7fc76e9ff700 1 Processor -- start 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.991+0000 7fc76e9ff700 1 -- start start 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc768101980 0x7fc7680ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768106be0 0x7fc7681003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0bc0 con 0x7fc7681042b0 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0d00 con 0x7fc768101980 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76e9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0e40 con 0x7fc768106be0 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56878/0 (socket says 172.21.15.31:56878) 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 -- 172.21.15.31:0/3578549011 learned_addr learned my addr 172.21.15.31:0/3578549011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc767fff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc768101980 0x7fc7680ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc76cf9c700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768106be0 0x7fc7681003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 -- 172.21.15.31:0/3578549011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768106be0 msgr2=0x7fc7681003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768106be0 0x7fc7681003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 -- 172.21.15.31:0/3578549011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc768101980 msgr2=0x7fc7680ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc768101980 0x7fc7680ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.992+0000 7fc7677fe700 1 -- 172.21.15.31:0/3578549011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009d00 con 0x7fc7681042b0 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc7677fe700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c009790 con 0x7fc7681042b0 2023-10-06T21:20:46.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7681a10d0 con 0x7fc7681042b0 2023-10-06T21:20:46.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc75c010430 con 0x7fc7681042b0 2023-10-06T21:20:46.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00cea0 con 0x7fc7681042b0 2023-10-06T21:20:46.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.993+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681a1630 con 0x7fc7681042b0 2023-10-06T21:20:46.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.994+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680fcf10 con 0x7fc7681042b0 2023-10-06T21:20:46.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.996+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc75c0105a0 con 0x7fc7681042b0 2023-10-06T21:20:47.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.998+0000 7fc7657fa700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 0x7fc7500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.998+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fc75c014070 con 0x7fc7681042b0 2023-10-06T21:20:47.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.998+0000 7fc767fff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 0x7fc7500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:46.999+0000 7fc767fff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 0x7fc7500a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc768101240 tx=0x7fc75800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.002+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc75c096480 con 0x7fc7681042b0 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 -- 172.21.15.31:0/1864185963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00100ea0 msgr2=0x7f8f001012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1864185963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00100ea0 0x7f8f001012c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009990 tx=0x7f8ef0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 -- 172.21.15.31:0/1864185963 shutdown_connections 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1864185963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00102a40 0x7f8f00109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1864185963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00100ea0 0x7f8f001012c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1864185963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 0x7f8f00102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 -- 172.21.15.31:0/1864185963 >> 172.21.15.31:0/1864185963 conn(0x7f8f000fc460 msgr2=0x7f8f000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 -- 172.21.15.31:0/1864185963 shutdown_connections 2023-10-06T21:20:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.003+0000 7f8f061ff700 1 -- 172.21.15.31:0/1864185963 wait complete. 2023-10-06T21:20:47.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.004+0000 7f8f061ff700 1 Processor -- start 2023-10-06T21:20:47.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.004+0000 7f8f061ff700 1 -- start start 2023-10-06T21:20:47.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.004+0000 7f8f061ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 0x7f8f00074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8f061ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 0x7f8f000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8f061ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8f061ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f00073f80 con 0x7f8f00102a40 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8f061ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000740f0 con 0x7f8f00100ea0 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8f061ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f001a0c00 con 0x7f8f00102080 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56892/0 (socket says 172.21.15.31:56892) 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 -- 172.21.15.31:0/1343463159 learned_addr learned my addr 172.21.15.31:0/1343463159 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 0x7f8f00074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efeffd700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 0x7f8f000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 -- 172.21.15.31:0/1343463159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 msgr2=0x7f8f000732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 0x7f8f000732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 -- 172.21.15.31:0/1343463159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 msgr2=0x7f8f00074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 0x7f8f00074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8efffff700 1 -- 172.21.15.31:0/1343463159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4009520 con 0x7f8f00102a40 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.005+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 0x7f8f00074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8efffff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c370 tx=0x7f8ef800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8efcff9700 1 -- 172.21.15.31:0/1343463159 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8017070 con 0x7f8f00102a40 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8efcff9700 1 -- 172.21.15.31:0/1343463159 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ef800f040 con 0x7f8f00102a40 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef8009c40 con 0x7f8f00102a40 2023-10-06T21:20:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8efcff9700 1 -- 172.21.15.31:0/1343463159 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800e050 con 0x7f8f00102a40 2023-10-06T21:20:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.006+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef00095f0 con 0x7f8f00102a40 2023-10-06T21:20:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.011+0000 7f8f19aaeb80 1 Processor -- start 2023-10-06T21:20:47.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.011+0000 7f8efcff9700 1 -- 172.21.15.31:0/1343463159 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ef800cb60 con 0x7f8f00102a40 2023-10-06T21:20:47.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.011+0000 7f8f19aaeb80 1 -- start start 2023-10-06T21:20:47.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c02700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c02c40 0x5596a1c03500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1b6a4e0 con 0x5596a1c03ae0 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1b5bca0 con 0x5596a1c022e0 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1b595c0 con 0x5596a1c02c40 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56900/0 (socket says 172.21.15.31:56900) 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8efeffd700 1 -- 172.21.15.31:0/183962215 learned_addr learned my addr 172.21.15.31:0/183962215 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c02c40 0x5596a1c03500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.012+0000 7f8efeffd700 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c02700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 -- 172.21.15.31:0/183962215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c02c40 msgr2=0x5596a1c03500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c02c40 0x5596a1c03500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 -- 172.21.15.31:0/183962215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 msgr2=0x5596a1c02700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c02700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 -- 172.21.15.31:0/183962215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a197fe60 con 0x5596a1c03ae0 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efcff9700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 0x7f8ee80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efeffd700 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4005950 tx=0x7f8ef4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e01d94e5ebcbf18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8efcff9700 1 -- 172.21.15.31:0/1343463159 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8ef80cc710 con 0x7f8f00102a40 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 0x7f8ee80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8ee5ffb700 1 -- 172.21.15.31:0/183962215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011110 con 0x5596a1c03ae0 2023-10-06T21:20:47.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8ee5ffb700 1 -- 172.21.15.31:0/183962215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ef40044c0 con 0x5596a1c03ae0 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.013+0000 7f8ee5ffb700 1 -- 172.21.15.31:0/183962215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4003d30 con 0x5596a1c03ae0 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/183962215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 msgr2=0x5596a1c03f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 0x7f8ee80a3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005910 tx=0x7f8ef000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4005950 tx=0x7f8ef4009420 comp rx=0 tx=0).stop 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/183962215 shutdown_connections 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c03ae0 0x5596a1c03f60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c02700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/183962215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c02c40 0x5596a1c03500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/183962215 >> 172.21.15.31:0/183962215 conn(0x5596a1b762e0 msgr2=0x5596a1bfd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/183962215 shutdown_connections 2023-10-06T21:20:47.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.014+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/183962215 wait complete. 2023-10-06T21:20:47.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.015+0000 7f8f19aaeb80 1 -- start start 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.015+0000 7f8f19aaeb80 1 Processor -- start 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1b54eb0 con 0x5596a1c02c40 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1af1790 con 0x5596a1c022e0 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1af1430 con 0x5596a1c03ae0 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8f19aaeb80 10 client.0 _ll_register_callbacks cb 0x5596a194e880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:47.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56914/0 (socket says 172.21.15.31:56914) 2023-10-06T21:20:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.016+0000 7f8efeffd700 1 -- 172.21.15.31:0/2107071567 learned_addr learned my addr 172.21.15.31:0/2107071567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efffff700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 -- 172.21.15.31:0/2107071567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 msgr2=0x5596a1c13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c13690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 -- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 msgr2=0x5596a1c0c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 -- 172.21.15.31:0/2107071567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4019040 con 0x5596a1c02c40 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8efeffd700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009b80 tx=0x7f8ef4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a742ec72185967b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.017+0000 7f8edeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4026070 con 0x5596a1c02c40 2023-10-06T21:20:47.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8ef4000e40 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596a1bf4f90 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ef4005280 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8f19aaeb80 10 client.15534 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596a1968ce0 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.018+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5596a1bf4f90 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ef401d070 con 0x5596a1c02c40 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 10 client.15534 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 20 client.15534 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c17d20 0x5596a1c1a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 4 client.15534 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596a1b4de90 con 0x5596a1c17d20 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.019+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c17d20 0x5596a1c1a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.020+0000 7f8edeffd700 10 client.15534 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.020+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c17d20 0x5596a1c1a140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46925e394cbb838 server_cookie=836121c02e5b53ce in_seq=0 out_seq=0 2023-10-06T21:20:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.021+0000 7f8edeffd700 1 -- 172.21.15.31:0/2107071567 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596a1b4de90 con 0x5596a1c17d20 2023-10-06T21:20:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.021+0000 7f8edeffd700 10 client.15534 handle_command_reply: tid=0 2023-10-06T21:20:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.021+0000 7f8f19aaeb80 1 client.15534 shutdown 2023-10-06T21:20:47.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.021+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 msgr2=0x5596a1c0c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.021+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009b80 tx=0x7f8ef4009e90 comp rx=0 tx=0).stop 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 shutdown_connections 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c17d20 0x5596a1c1a140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/2107071567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c0c650 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 >> 172.21.15.31:0/2107071567 conn(0x5596a1b762e0 msgr2=0x5596a1bfcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8edeffd700 0 client.15534 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 shutdown_connections 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/2107071567 wait complete. 2023-10-06T21:20:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.022+0000 7f8f19aaeb80 20 client.15534 trim_cache size 0 max 16384 2023-10-06T21:20:47.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.040+0000 7f8f19aaeb80 1 Processor -- start 2023-10-06T21:20:47.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.040+0000 7f8f19aaeb80 1 -- start start 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.040+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c09ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a19d59a0 con 0x5596a1c02c40 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1c03020 con 0x5596a1c022e0 2023-10-06T21:20:47.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1c0bc80 con 0x5596a1c03ae0 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56922/0 (socket says 172.21.15.31:56922) 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 -- 172.21.15.31:0/1487248461 learned_addr learned my addr 172.21.15.31:0/1487248461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efffff700 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c09ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 -- 172.21.15.31:0/1487248461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 msgr2=0x5596a1c09ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c09ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 -- 172.21.15.31:0/1487248461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 msgr2=0x5596a1c0ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.041+0000 7f8efeffd700 1 -- 172.21.15.31:0/1487248461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a1bfbf10 con 0x5596a1c02c40 2023-10-06T21:20:47.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.042+0000 7f8efeffd700 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e20 tx=0x7f8ef4003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d104b68bd8823a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.042+0000 7f8ee67fc700 1 -- 172.21.15.31:0/1487248461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4026020 con 0x5596a1c02c40 2023-10-06T21:20:47.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.042+0000 7f8ee67fc700 1 -- 172.21.15.31:0/1487248461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ef40098e0 con 0x5596a1c02c40 2023-10-06T21:20:47.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.042+0000 7f8ee67fc700 1 -- 172.21.15.31:0/1487248461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40054e0 con 0x5596a1c02c40 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/1487248461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 msgr2=0x5596a1c13650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e20 tx=0x7f8ef4003b00 comp rx=0 tx=0).stop 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/1487248461 shutdown_connections 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c03ae0 0x5596a1c09ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c022e0 0x5596a1c0ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/1487248461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c13650 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/1487248461 >> 172.21.15.31:0/1487248461 conn(0x5596a1b762e0 msgr2=0x5596a1c13f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/1487248461 shutdown_connections 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/1487248461 wait complete. 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 -- start start 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.043+0000 7f8f19aaeb80 1 Processor -- start 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 0x5596a1c11d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c03ae0 0x5596a1c12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1c13a00 con 0x5596a1c02c40 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1c6bb80 con 0x5596a1c03ae0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a1c6bd40 con 0x5596a1c022e0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8f19aaeb80 10 client.0 _ll_register_callbacks cb 0x7f8ef4011500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8efeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56932/0 (socket says 172.21.15.31:56932) 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8efeffd700 1 -- 172.21.15.31:0/3822404290 learned_addr learned my addr 172.21.15.31:0/3822404290 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 0x5596a1c11d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.044+0000 7f8efffff700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c03ae0 0x5596a1c12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 -- 172.21.15.31:0/3822404290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 msgr2=0x5596a1c11d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 0x5596a1c11d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 -- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c03ae0 msgr2=0x5596a1c12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c03ae0 0x5596a1c12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 -- 172.21.15.31:0/3822404290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596a1968ce0 con 0x5596a1c02c40 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 0x5596a1c11d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8efeffd700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009340 tx=0x7f8ef40043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19741fe9c7e65367 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8edeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4026030 con 0x5596a1c02c40 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ef40044c0 con 0x5596a1c02c40 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f8ef4003c50 con 0x5596a1c02c40 2023-10-06T21:20:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.045+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ef4005280 con 0x5596a1c02c40 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 10 client.15540 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596a19a62d0 con 0x5596a1c02c40 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ef40044c0 con 0x5596a1c02c40 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ef4011cf0 con 0x5596a1c02c40 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 20 client.15540 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c6d320 0x5596a1c6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 4 client.15540 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:20:47.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5596a1b4dbb0 con 0x5596a1c6d320 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.046+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c6d320 0x5596a1c6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.047+0000 7f8eff7fe700 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c6d320 0x5596a1c6f740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e21f8f27855239 server_cookie=f60ad84aabd4b532 in_seq=0 out_seq=0 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.047+0000 7f8edeffd700 10 client.15540 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.048+0000 7f8edeffd700 1 -- 172.21.15.31:0/3822404290 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596a1b4dbb0 con 0x5596a1c6d320 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.048+0000 7f8edeffd700 10 client.15540 handle_command_reply: tid=0 2023-10-06T21:20:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.048+0000 7f8f19aaeb80 1 client.15540 shutdown 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.048+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 msgr2=0x5596a1c122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.048+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009340 tx=0x7f8ef40043d0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 shutdown_connections 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596a1c03ae0 0x5596a1c12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596a1c022e0 0x5596a1c11d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5596a1c6d320 0x5596a1c6f740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 --2- 172.21.15.31:0/3822404290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596a1c02c40 0x5596a1c122c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8edeffd700 0 client.15540 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 >> 172.21.15.31:0/3822404290 conn(0x5596a1b762e0 msgr2=0x5596a1c0ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 shutdown_connections 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 1 -- 172.21.15.31:0/3822404290 wait complete. 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.049+0000 7f8f19aaeb80 20 client.15540 trim_cache size 0 max 16384 2023-10-06T21:20:47.058 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-10-06T21:20:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "7e857854-40d4-4b6c-990c-3be4cf80a89c", 2023-10-06T21:20:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-10-06T21:20:47.059 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 msgr2=0x7f8ee80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 0x7f8ee80a3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005910 tx=0x7f8ef000b260 comp rx=0 tx=0).stop 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 msgr2=0x7f8f000737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c370 tx=0x7f8ef800c680 comp rx=0 tx=0).stop 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 shutdown_connections 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f8ee80a1480 0x7f8ee80a3940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f00102a40 0x7f8f000737e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00100ea0 0x7f8f00074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 --2- 172.21.15.31:0/1343463159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f00102080 0x7f8f000732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 >> 172.21.15.31:0/1343463159 conn(0x7f8f000fc460 msgr2=0x7f8f000fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 shutdown_connections 2023-10-06T21:20:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.050+0000 7f8f061ff700 1 -- 172.21.15.31:0/1343463159 wait complete. 2023-10-06T21:20:47.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:46 smithi031 ceph-mon[98539]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T21:20:47.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:46 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2968568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.076 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:46 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4006187231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:20:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 -- 172.21.15.31:0/2890691466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f50074ef0 msgr2=0x7f5f50073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2890691466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f50074ef0 0x7f5f50073340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009990 tx=0x7f5f4c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 -- 172.21.15.31:0/2890691466 shutdown_connections 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2890691466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5010d4b0 0x7f5f5010f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2890691466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f50074ef0 0x7f5f50073340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2890691466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50073910 0x7f5f50073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 -- 172.21.15.31:0/2890691466 >> 172.21.15.31:0/2890691466 conn(0x7f5f500fc430 msgr2=0x7f5f500fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 -- 172.21.15.31:0/2890691466 shutdown_connections 2023-10-06T21:20:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.280+0000 7f5f581ff700 1 -- 172.21.15.31:0/2890691466 wait complete. 2023-10-06T21:20:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.281+0000 7f5f581ff700 1 Processor -- start 2023-10-06T21:20:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.281+0000 7f5f581ff700 1 -- start start 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50074ef0 0x7f5f5019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 0x7f5f501a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f501a4c50 con 0x7f5f50073910 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f501a4d90 con 0x7f5f5010d4b0 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f581ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f501a4f20 con 0x7f5f50074ef0 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56950/0 (socket says 172.21.15.31:56950) 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 -- 172.21.15.31:0/2050607287 learned_addr learned my addr 172.21.15.31:0/2050607287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f5579a700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50074ef0 0x7f5f5019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 0x7f5f501a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 -- 172.21.15.31:0/2050607287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50074ef0 msgr2=0x7f5f5019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50074ef0 0x7f5f5019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 -- 172.21.15.31:0/2050607287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 msgr2=0x7f5f501a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 0x7f5f501a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f55f9b700 1 -- 172.21.15.31:0/2050607287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0095f0 con 0x7f5f50073910 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.282+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 0x7f5f501a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c003f60 tx=0x7f5f4c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f46ffd700 1 -- 172.21.15.31:0/2050607287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c01d070 con 0x7f5f50073910 2023-10-06T21:20:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f501a51a0 con 0x7f5f50073910 2023-10-06T21:20:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f46ffd700 1 -- 172.21.15.31:0/2050607287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f4c00f460 con 0x7f5f50073910 2023-10-06T21:20:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f46ffd700 1 -- 172.21.15.31:0/2050607287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c021670 con 0x7f5f50073910 2023-10-06T21:20:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.283+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f501a56a0 con 0x7f5f50073910 2023-10-06T21:20:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 Processor -- start 2023-10-06T21:20:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 -- start start 2023-10-06T21:20:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37bae8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f46ffd700 1 -- 172.21.15.31:0/2050607287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5f4c00f5d0 con 0x7f5f50073910 2023-10-06T21:20:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae9a60 0x55a37bae9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37ba50460 con 0x55a37bae8bc0 2023-10-06T21:20:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37ba41c20 con 0x55a37bae9a60 2023-10-06T21:20:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37ba3f540 con 0x55a37bae8260 2023-10-06T21:20:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.288+0000 7f5f55f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f5679c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37bae8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56960/0 (socket says 172.21.15.31:56960) 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 -- 172.21.15.31:0/1710553835 learned_addr learned my addr 172.21.15.31:0/1710553835 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f5579a700 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae9a60 0x55a37bae9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 -- 172.21.15.31:0/1710553835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 msgr2=0x55a37bae8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37bae8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 -- 172.21.15.31:0/1710553835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae9a60 msgr2=0x55a37bae9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae9a60 0x55a37bae9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f55f9b700 1 -- 172.21.15.31:0/1710553835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37b865de0 con 0x55a37bae8bc0 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.289+0000 7f5f5679c700 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37bae8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0d1fb0 tx=0x7f5f4c0d63c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b51a6310da7779d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f3bfff700 1 -- 172.21.15.31:0/1710553835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0d6ac0 con 0x55a37bae8bc0 2023-10-06T21:20:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f3bfff700 1 -- 172.21.15.31:0/1710553835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f4c0d6c20 con 0x55a37bae8bc0 2023-10-06T21:20:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f3bfff700 1 -- 172.21.15.31:0/1710553835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0dc660 con 0x55a37bae8bc0 2023-10-06T21:20:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f46ffd700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 0x7f5f3c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.290+0000 7f5f46ffd700 1 -- 172.21.15.31:0/2050607287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f5f4c0ce080 con 0x7f5f50073910 2023-10-06T21:20:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 0x7f5f3c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1710553835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 msgr2=0x55a37bae9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0d1fb0 tx=0x7f5f4c0d63c0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1710553835 shutdown_connections 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae9a60 0x55a37bae9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37bae8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1710553835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37bae9480 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1710553835 >> 172.21.15.31:0/1710553835 conn(0x55a37ba5c260 msgr2=0x55a37bae3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 0x7f5f3c0a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5f48007650 tx=0x7f5f4800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1710553835 shutdown_connections 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.291+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1710553835 wait complete. 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.292+0000 7f5f6bbcdb80 1 -- start start 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.292+0000 7f5f6bbcdb80 1 Processor -- start 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.292+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37baf1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8bc0 0x55a37baf2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37ba3ae30 con 0x55a37bae9a60 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37b9d7710 con 0x55a37bae8bc0 2023-10-06T21:20:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37b9d73b0 con 0x55a37bae8260 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f6bbcdb80 10 client.0 _ll_register_callbacks cb 0x55a37b834800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56974/0 (socket says 172.21.15.31:56974) 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 -- 172.21.15.31:0/2396407203 learned_addr learned my addr 172.21.15.31:0/2396407203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37baf1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5579a700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8bc0 0x55a37baf2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 -- 172.21.15.31:0/2396407203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 msgr2=0x55a37baf1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37baf1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 -- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8bc0 msgr2=0x55a37baf2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8bc0 0x55a37baf2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.293+0000 7f5f5679c700 1 -- 172.21.15.31:0/2396407203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f480090d0 con 0x55a37bae9a60 2023-10-06T21:20:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f5679c700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009080 tx=0x7f5f48004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68a23f4b4583060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f397fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48016cc0 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a37badaf10 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f5f48016e20 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f48018e80 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f6bbcdb80 10 client.15552 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.294+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5f501a56a0 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a37badaf10 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f4802c590 con 0x55a37bae9a60 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 20 client.15552 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bafdb90 0x55a37bafffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 4 client.15552 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a37ba33e10 con 0x55a37bafdb90 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.295+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bafdb90 0x55a37bafffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.296+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bafdb90 0x55a37bafffb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e10ee04d2b17f63f server_cookie=63fc24386d6ccefd in_seq=0 out_seq=0 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.296+0000 7f5f397fa700 10 client.15552 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f397fa700 1 -- 172.21.15.31:0/2396407203 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a37ba33e10 con 0x55a37bafdb90 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f397fa700 10 client.15552 handle_command_reply: tid=0 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 client.15552 shutdown 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 msgr2=0x55a37baf9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009080 tx=0x7f5f48004b40 comp rx=0 tx=0).stop 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 shutdown_connections 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bafdb90 0x55a37bafffb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae9a60 0x55a37baf9270 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae8260 0x55a37baf1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/2396407203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8bc0 0x55a37baf2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f397fa700 0 client.15552 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 >> 172.21.15.31:0/2396407203 conn(0x55a37ba5c260 msgr2=0x55a37bae2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 shutdown_connections 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/2396407203 wait complete. 2023-10-06T21:20:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.297+0000 7f5f6bbcdb80 20 client.15552 trim_cache size 0 max 16384 2023-10-06T21:20:47.307 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:46 smithi146 ceph-mon[102809]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T21:20:47.307 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:46 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2968568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.307 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:46 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4006187231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.306+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc76810c590 con 0x7fc7681042b0 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 Processor -- start 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 -- start start 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37b8bb920 con 0x55a37bae8bc0 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37bae7ed0 con 0x55a37bae8260 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37badfa50 con 0x55a37bae9a60 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f5579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f5579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56984/0 (socket says 172.21.15.31:56984) 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.315+0000 7f5f5579a700 1 -- 172.21.15.31:0/953646253 learned_addr learned my addr 172.21.15.31:0/953646253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 -- 172.21.15.31:0/953646253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 msgr2=0x55a37baf1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 -- 172.21.15.31:0/953646253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 msgr2=0x55a37baf09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 -- 172.21.15.31:0/953646253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37b84ec60 con 0x55a37bae8bc0 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f5579a700 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e90 tx=0x7f5f40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5734313eea76718 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f44ff9700 1 -- 172.21.15.31:0/953646253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40017070 con 0x55a37bae8bc0 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f44ff9700 1 -- 172.21.15.31:0/953646253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f40018040 con 0x55a37bae8bc0 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.316+0000 7f5f44ff9700 1 -- 172.21.15.31:0/953646253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40003ae0 con 0x55a37bae8bc0 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/953646253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 msgr2=0x55a37baf9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e90 tx=0x7f5f40009260 comp rx=0 tx=0).stop 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/953646253 shutdown_connections 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/953646253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf9230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/953646253 >> 172.21.15.31:0/953646253 conn(0x55a37ba5c260 msgr2=0x55a37bb04910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/953646253 shutdown_connections 2023-10-06T21:20:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/953646253 wait complete. 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.317+0000 7f5f6bbcdb80 1 -- start start 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 Processor -- start 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37baf8c00 con 0x55a37bae8bc0 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37baf8d70 con 0x55a37bae8260 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37baf8f30 con 0x55a37bae9a60 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f6bbcdb80 10 client.0 _ll_register_callbacks cb 0x7f5f48018830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.318+0000 7f5f5579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56986/0 (socket says 172.21.15.31:56986) 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 -- 172.21.15.31:0/1604042572 learned_addr learned my addr 172.21.15.31:0/1604042572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5679c700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 -- 172.21.15.31:0/1604042572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 msgr2=0x55a37baf86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 -- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 msgr2=0x55a37baf5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 -- 172.21.15.31:0/1604042572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40009b40 con 0x55a37bae8bc0 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f5579a700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e90 tx=0x7f5f40009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8379c8f4530b44ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f397fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40017040 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.319+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f4c0d6c20 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f5f40003820 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f40004560 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 10 client.15558 fetch_fsmap learned FSMap version 27 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a37b88c250 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f4c0d6c20 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f40030060 con 0x55a37bae8bc0 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:20:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 20 client.15558 populate_metadata read hostname 'smithi031' 2023-10-06T21:20:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bb52a40 0x55a37bb54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 4 client.15558 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:20:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.320+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a37ba33b30 con 0x55a37bb52a40 2023-10-06T21:20:47.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bb52a40 0x55a37bb54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f55f9b700 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bb52a40 0x55a37bb54e60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3aad420f9f2946 server_cookie=326780510895416a in_seq=0 out_seq=0 2023-10-06T21:20:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f397fa700 10 client.15558 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f397fa700 1 -- 172.21.15.31:0/1604042572 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a37ba33b30 con 0x55a37bb52a40 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f397fa700 10 client.15558 handle_command_reply: tid=0 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.321+0000 7f5f6bbcdb80 1 client.15558 shutdown 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 msgr2=0x55a37baf56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e90 tx=0x7f5f40009760 comp rx=0 tx=0).stop 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 shutdown_connections 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a37bae9a60 0x55a37baf86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a37bae8260 0x55a37baf5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a37bb52a40 0x55a37bb54e60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 --2- 172.21.15.31:0/1604042572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a37bae8bc0 0x55a37baf56a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 >> 172.21.15.31:0/1604042572 conn(0x55a37ba5c260 msgr2=0x55a37baf09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f397fa700 0 client.15558 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 shutdown_connections 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 1 -- 172.21.15.31:0/1604042572 wait complete. 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f6bbcdb80 20 client.15558 trim_cache size 0 max 16384 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-10-06T21:20:47.331 INFO:teuthology.orchestra.run.smithi031.stdout: "7e857854-40d4-4b6c-990c-3be4cf80a89c": { 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "7e857854-40d4-4b6c-990c-3be4cf80a89c", 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:20:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 msgr2=0x7f5f3c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.322+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 0x7f5f3c0a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5f48007650 tx=0x7f5f4800b040 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 msgr2=0x7f5f5019d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c003f60 tx=0x7f5f4c0057c0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 shutdown_connections 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f5f3c0a1480 0x7f5f3c0a3940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f5010d4b0 0x7f5f501a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f50073910 0x7f5f5019d0d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 --2- 172.21.15.31:0/2050607287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f50074ef0 0x7f5f5019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 >> 172.21.15.31:0/2050607287 conn(0x7f5f500fc430 msgr2=0x7f5f500fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 shutdown_connections 2023-10-06T21:20:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.323+0000 7f5f581ff700 1 -- 172.21.15.31:0/2050607287 wait complete. 2023-10-06T21:20:47.343 INFO:tasks.cephfs.filesystem:scrub status for tag:7e857854-40d4-4b6c-990c-3be4cf80a89c - {'path': '/', 'tag': '7e857854-40d4-4b6c-990c-3be4cf80a89c', 'options': 'recursive,force'} 2023-10-06T21:20:47.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:46 smithi142 ceph-mon[102951]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T21:20:47.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:46 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2968568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:46 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4006187231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:20:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.972+0000 7fc7657fa700 1 -- 172.21.15.31:0/3578549011 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc75c099aa0 con 0x7fc7681042b0 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 msgr2=0x7fc7500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 0x7fc7500a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc768101240 tx=0x7fc75800a380 comp rx=0 tx=0).stop 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 msgr2=0x7fc7680ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 shutdown_connections 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fc7500a1500 0x7fc7500a39c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc768106be0 0x7fc7681003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc768101980 0x7fc7680ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 --2- 172.21.15.31:0/3578549011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7681042b0 0x7fc7680ffe70 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.977+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 >> 172.21.15.31:0/3578549011 conn(0x7fc7680fb390 msgr2=0x7fc768102690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.978+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 shutdown_connections 2023-10-06T21:20:47.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:47.978+0000 7fc76e9ff700 1 -- 172.21.15.31:0/3578549011 wait complete. 2023-10-06T21:20:48.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-10-06T21:20:48.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 -- 172.21.15.31:0/3383014379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102a20 msgr2=0x7fd2fc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3383014379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102a20 0x7fd2fc109340 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b0a0 tx=0x7fd2f400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 -- 172.21.15.31:0/3383014379 shutdown_connections 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3383014379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102a20 0x7fd2fc109340 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3383014379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc100f10 0x7fd2fc101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3383014379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc102060 0x7fd2fc1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.232+0000 7fd3015ff700 1 -- 172.21.15.31:0/3383014379 >> 172.21.15.31:0/3383014379 conn(0x7fd2fc0fc460 msgr2=0x7fd2fc0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.233+0000 7fd3015ff700 1 -- 172.21.15.31:0/3383014379 shutdown_connections 2023-10-06T21:20:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.233+0000 7fd3015ff700 1 -- 172.21.15.31:0/3383014379 wait complete. 2023-10-06T21:20:48.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.234+0000 7fd3015ff700 1 Processor -- start 2023-10-06T21:20:48.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 -- start start 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 0x7fd2fc196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc102a20 0x7fd2fc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1986f0 con 0x7fd2fc102060 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc198860 con 0x7fd2fc100f10 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd3015ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc198a20 con 0x7fd2fc102a20 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd2fa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd2faffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 0x7fd2fc196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd2fa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57008/0 (socket says 172.21.15.31:57008) 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd2fa7fc700 1 -- 172.21.15.31:0/3222611504 learned_addr learned my addr 172.21.15.31:0/3222611504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.235+0000 7fd2faffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 0x7fd2fc196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34210/0 (socket says 172.21.15.31:34210) 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fb7fe700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc102a20 0x7fd2fc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc102a20 msgr2=0x7fd2fc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc102a20 0x7fd2fc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 msgr2=0x7fd2fc196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 0x7fd2fc196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 -- 172.21.15.31:0/3222611504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009d00 con 0x7fd2fc102060 2023-10-06T21:20:48.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2fa7fc700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e9a0 tx=0x7fd2f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.236+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0009790 con 0x7fd2fc102060 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.237+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc198d00 con 0x7fd2fc102060 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.237+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2f0010430 con 0x7fd2fc102060 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.237+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000cea0 con 0x7fd2fc102060 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.237+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc19f480 con 0x7fd2fc102060 2023-10-06T21:20:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.238+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc1084f0 con 0x7fd2fc102060 2023-10-06T21:20:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.240+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd2f00105a0 con 0x7fd2fc102060 2023-10-06T21:20:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.242+0000 7fd2e3fff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 0x7fd2e40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.242+0000 7fd2faffd700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 msgr2=0x7fd2e40a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.242+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7fd2f0014070 con 0x7fd2fc102060 2023-10-06T21:20:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.242+0000 7fd2faffd700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 0x7fd2e40a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:20:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.245+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd2f0052100 con 0x7fd2fc102060 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: ignoring --setuser ceph since I am not root 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: ignoring --setgroup ceph since I am not root 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mgr[104435]: -- 172.21.15.142:0/3684027998 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560eb997d1e0 con 0x560eb9952800 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:48.256+0000 7f6087d00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:47 smithi031 ceph-mon[98539]: scrub queued for path: / 2023-10-06T21:20:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:47 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:47 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:47 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:47 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:47 smithi142 ceph-mon[102951]: scrub queued for path: / 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:47 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:47 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:47 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:47 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:20:48.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setuser ceph since I am not root 2023-10-06T21:20:48.262 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: ignoring --setgroup ceph since I am not root 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: ignoring --setuser ceph since I am not root 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: ignoring --setgroup ceph since I am not root 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:47 smithi146 ceph-mon[102809]: scrub queued for path: / 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:47 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:47 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:47 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.264 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:47 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:20:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.443+0000 7fd2faffd700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 msgr2=0x7fd2e40a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.443+0000 7fd2faffd700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 0x7fd2e40a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:20:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.468+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd2fc04ec90 con 0x7fd2fc102060 2023-10-06T21:20:48.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.472+0000 7fd2e3fff700 1 -- 172.21.15.31:0/3222611504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd2f009a010 con 0x7fd2fc102060 2023-10-06T21:20:48.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 msgr2=0x7fd2e40a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 0x7fd2e40a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 msgr2=0x7fd2fc1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e9a0 tx=0x7fd2f000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 shutdown_connections 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7fd2e40a14d0 0x7fd2e40a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc102a20 0x7fd2fc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2fc100f10 0x7fd2fc196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 --2- 172.21.15.31:0/3222611504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc102060 0x7fd2fc1972b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.475+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 >> 172.21.15.31:0/3222611504 conn(0x7fd2fc0fc460 msgr2=0x7fd2fc105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.476+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 shutdown_connections 2023-10-06T21:20:48.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.476+0000 7fd3015ff700 1 -- 172.21.15.31:0/3222611504 wait complete. 2023-10-06T21:20:48.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-10-06T21:20:48.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:48.261+0000 7f0bd992b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:20:48.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:48.367+0000 7f0bd992b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:48.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:48.482+0000 7f0bd992b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:20:48.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:48.280+0000 7fc3fbae7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:20:48.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:48.388+0000 7fc3fbae7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:48.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:48.509+0000 7fc3fbae7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:20:48.596 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:48.362+0000 7f6087d00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:48.597 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:48.475+0000 7f6087d00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 -- 172.21.15.31:0/401493470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0074ef0 msgr2=0x7f4eb0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/401493470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0073340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009990 tx=0x7f4eac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 -- 172.21.15.31:0/401493470 shutdown_connections 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/401493470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0108dc0 0x7f4eb010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/401493470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0073340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/401493470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0073910 0x7f4eb0073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.699+0000 7f4eb7fff700 1 -- 172.21.15.31:0/401493470 >> 172.21.15.31:0/401493470 conn(0x7f4eb00fc430 msgr2=0x7f4eb00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.700+0000 7f4eb7fff700 1 -- 172.21.15.31:0/401493470 shutdown_connections 2023-10-06T21:20:48.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.700+0000 7f4eb7fff700 1 -- 172.21.15.31:0/401493470 wait complete. 2023-10-06T21:20:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 Processor -- start 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 -- start start 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0073910 0x7f4eb0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb019a850 con 0x7f4eb0108dc0 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb019a9c0 con 0x7f4eb0074ef0 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb7fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb019ab80 con 0x7f4eb0073910 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57024/0 (socket says 172.21.15.31:57024) 2023-10-06T21:20:48.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb659c700 1 -- 172.21.15.31:0/4204996338 learned_addr learned my addr 172.21.15.31:0/4204996338 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb5d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0073910 0x7f4eb0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.701+0000 7f4eb559a700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0073910 msgr2=0x7f4eb0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0073910 0x7f4eb0198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0074ef0 msgr2=0x7f4eb0199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 -- 172.21.15.31:0/4204996338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0009520 con 0x7f4eb0108dc0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb659c700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800ef30 tx=0x7f4ea800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8018070 con 0x7f4eb0108dc0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ea800f040 con 0x7f4eb0108dc0 2023-10-06T21:20:48.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800cbc0 con 0x7f4eb0108dc0 2023-10-06T21:20:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.702+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac0095f0 con 0x7f4eb0108dc0 2023-10-06T21:20:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.703+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb01a13a0 con 0x7f4eb0108dc0 2023-10-06T21:20:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.704+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb0107470 con 0x7f4eb0108dc0 2023-10-06T21:20:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.706+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4ea8007500 con 0x7f4eb0108dc0 2023-10-06T21:20:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.708+0000 7f4ea6ffd700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 0x7f4e9c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.708+0000 7f4eb5d9b700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 msgr2=0x7f4e9c0a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.708+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f4ea80cc8e0 con 0x7f4eb0108dc0 2023-10-06T21:20:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.708+0000 7f4eb5d9b700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 0x7f4e9c0a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:20:48.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.712+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ea8095f10 con 0x7f4eb0108dc0 2023-10-06T21:20:48.881 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:48.638+0000 7fc3fbae7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:20:48.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:48.593+0000 7f6087d00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:20:48.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:48.816+0000 7f6087d00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:20:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.909+0000 7f4eb5d9b700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 msgr2=0x7f4e9c0a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.909+0000 7f4eb5d9b700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 0x7f4e9c0a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:20:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.925+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f4eb004ec90 con 0x7f4eb0108dc0 2023-10-06T21:20:48.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.931+0000 7f4ea6ffd700 1 -- 172.21.15.31:0/4204996338 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4ea8099530 con 0x7f4eb0108dc0 2023-10-06T21:20:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.933+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 msgr2=0x7f4e9c0a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.933+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 0x7f4e9c0a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.933+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 msgr2=0x7f4eb019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.933+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800ef30 tx=0x7f4ea800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 shutdown_connections 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f4e9c0a14d0 0x7f4e9c0a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eb0108dc0 0x7f4eb019a150 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb0073910 0x7f4eb0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 --2- 172.21.15.31:0/4204996338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb0074ef0 0x7f4eb0199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 >> 172.21.15.31:0/4204996338 conn(0x7f4eb00fc430 msgr2=0x7f4eb00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 shutdown_connections 2023-10-06T21:20:48.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:48.934+0000 7f4eb7fff700 1 -- 172.21.15.31:0/4204996338 wait complete. 2023-10-06T21:20:48.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-06T21:20:48.997 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:48.604+0000 7f0bd992b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:20:48.997 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:48.828+0000 7f0bd992b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:20:49.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 -- 172.21.15.31:0/2475495633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 msgr2=0x7f75d8073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:49.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/2475495633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d8073d90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009990 tx=0x7f75cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 -- 172.21.15.31:0/2475495633 shutdown_connections 2023-10-06T21:20:49.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/2475495633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d810d4e0 0x7f75d810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/2475495633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 0x7f75d8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/2475495633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d8073d90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 -- 172.21.15.31:0/2475495633 >> 172.21.15.31:0/2475495633 conn(0x7f75d80fc460 msgr2=0x7f75d80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 -- 172.21.15.31:0/2475495633 shutdown_connections 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.162+0000 7f75dd3ff700 1 -- 172.21.15.31:0/2475495633 wait complete. 2023-10-06T21:20:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.163+0000 7f75dd3ff700 1 Processor -- start 2023-10-06T21:20:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 -- start start 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 0x7f75d819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d810d4e0 0x7f75d819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d819ec80 con 0x7f75d8073910 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d819edf0 con 0x7f75d810d4e0 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.164+0000 7f75dd3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d819efb0 con 0x7f75d8074ef0 2023-10-06T21:20:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 0x7f75d819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d819d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57050/0 (socket says 172.21.15.31:57050) 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 learned_addr learned my addr 172.21.15.31:0/3981761286 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d77fe700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d810d4e0 0x7f75d819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d810d4e0 msgr2=0x7f75d819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d810d4e0 0x7f75d819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 msgr2=0x7f75d819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:49.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d819d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:49.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0095f0 con 0x7f75d8074ef0 2023-10-06T21:20:49.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d8073910 0x7f75d819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:49.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.165+0000 7f75d67fc700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 0x7f75d819d7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:49.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.166+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc01d070 con 0x7f75d8074ef0 2023-10-06T21:20:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.166+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75cc00bdb0 con 0x7f75d8074ef0 2023-10-06T21:20:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.166+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d819f230 con 0x7f75d8074ef0 2023-10-06T21:20:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.166+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc022d40 con 0x7f75d8074ef0 2023-10-06T21:20:49.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.166+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d81a5a50 con 0x7f75d8074ef0 2023-10-06T21:20:49.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.167+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d8103030 con 0x7f75d8074ef0 2023-10-06T21:20:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.169+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f75cc022470 con 0x7f75d8074ef0 2023-10-06T21:20:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.170+0000 7f75bffff700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.170+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.170+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8335+0+0 (secure 0 0 0) 0x7f75cc0ce2f0 con 0x7f75d8074ef0 2023-10-06T21:20:49.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.170+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:20:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.175+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f75cc097920 con 0x7f75d8074ef0 2023-10-06T21:20:49.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:48.878+0000 7fc3fbae7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:20:49.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:20:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-mon[98539]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-10-06T21:20:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:20:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3222611504' entity='client.admin' 2023-10-06T21:20:49.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:48 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4204996338' entity='client.admin' 2023-10-06T21:20:49.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:20:49.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mon[102951]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-10-06T21:20:49.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:20:49.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3222611504' entity='client.admin' 2023-10-06T21:20:49.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:48 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4204996338' entity='client.admin' 2023-10-06T21:20:49.287 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3578549011' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:20:49.287 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-mon[102809]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-10-06T21:20:49.287 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:20:49.287 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3222611504' entity='client.admin' 2023-10-06T21:20:49.287 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:48 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4204996338' entity='client.admin' 2023-10-06T21:20:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.371+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.371+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:20:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.419+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f75d80612f0 con 0x7f75c00a14d0 2023-10-06T21:20:49.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:49.284+0000 7f0bd992b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:20:49.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:49.449+0000 7f0bd992b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:20:49.556 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:49.272+0000 7f6087d00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:20:49.556 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:49.436+0000 7f6087d00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:20:49.659 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:49.366+0000 7fc3fbae7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:20:49.659 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:49.535+0000 7fc3fbae7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:20:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.772+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:49.772+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:20:49.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:49.552+0000 7f6087d00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:20:49.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:49.676+0000 7f6087d00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:20:50.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:49.656+0000 7fc3fbae7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:20:50.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:49.784+0000 7fc3fbae7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:20:50.048 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:49.567+0000 7f0bd992b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:20:50.049 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:49.691+0000 7f0bd992b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:20:50.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:50.228+0000 7f0bd992b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:20:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:50.573+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:50.573+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:20:50.651 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:50 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:50.212+0000 7f6087d00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:20:50.762 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:50.343+0000 7fc3fbae7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:20:51.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:51.060+0000 7f6087d00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:20:51.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:51.166+0000 7f6087d00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:20:51.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:51.223+0000 7fc3fbae7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:20:51.510 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:51.333+0000 7fc3fbae7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:20:51.548 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:51.075+0000 7f0bd992b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:20:51.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:51.183+0000 7f0bd992b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:20:52.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:52.175+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:52.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:52.175+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:20:52.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:52.124+0000 7f6087d00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:52.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:52.244+0000 7f6087d00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:20:52.492 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:52.144+0000 7f0bd992b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:52.492 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:52.263+0000 7f0bd992b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:20:52.691 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:52.330+0000 7fc3fbae7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:52.691 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:52.453+0000 7fc3fbae7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:20:52.798 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:52.490+0000 7f0bd992b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:52.832 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:52.468+0000 7f6087d00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:53.010 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:52.688+0000 7fc3fbae7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:20:53.111 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:52.828+0000 7f6087d00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:20:53.141 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:52.855+0000 7f0bd992b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:20:53.358 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:53.071+0000 7fc3fbae7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:20:53.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:53 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:53.108+0000 7f6087d00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:20:53.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:53 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:53.300+0000 7f6087d00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:20:53.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:53.139+0000 7f0bd992b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:20:53.549 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:53.333+0000 7f0bd992b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:20:53.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:53.354+0000 7fc3fbae7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:20:53.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:53.555+0000 7fc3fbae7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:20:53.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:53 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:53.641+0000 7f6087d00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:20:54.048 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:53.680+0000 7f0bd992b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:20:54.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:53.917+0000 7fc3fbae7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:20:54.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:54.575+0000 7f6087d00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:20:54.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:54.747+0000 7f6087d00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:20:55.043 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:54.614+0000 7f0bd992b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:20:55.044 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:54.786+0000 7f0bd992b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:20:55.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:54.893+0000 7fc3fbae7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:20:55.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:55.076+0000 7fc3fbae7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:20:55.298 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:55.041+0000 7f0bd992b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:20:55.299 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:55.190+0000 7f0bd992b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:55.376+0000 7f75d6ffd700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/964869793 2023-10-06T21:20:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:55.376+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:20:55.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:55 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:55.001+0000 7f6087d00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:20:55.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:55 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:55.145+0000 7f6087d00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:55.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:55.334+0000 7fc3fbae7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:20:55.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:55.486+0000 7fc3fbae7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:20:55.798 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:55.465+0000 7f0bd992b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:20:55.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:55 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:55.419+0000 7f6087d00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:20:56.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:55.774+0000 7fc3fbae7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:20:56.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:56.553+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f75cc097cf0 con 0x7f75d8074ef0 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z[105045]: 2023-10-06T21:20:56.543+0000 7f0bd992b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: Standby manager daemon y restarted 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: Standby manager daemon y started 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:20:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:56 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:56.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: Standby manager daemon y restarted 2023-10-06T21:20:56.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: Standby manager daemon y started 2023-10-06T21:20:56.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:20:56.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:56.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:20:56.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y[104411]: 2023-10-06T21:20:56.488+0000 7f6087d00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: Standby manager daemon y restarted 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: Standby manager daemon y started 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:20:56.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:56 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.142:0/723807989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:56.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:56.913+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 8 ==== mgrmap(e 21) v1 ==== 135688+0+0 (secure 0 0 0) 0x7f75cc0cbdb0 con 0x7f75d8074ef0 2023-10-06T21:20:56.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:56.914+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 msgr2=0x7f75c00a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:20:56.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:56.914+0000 7f75bffff700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/964869793,v1:172.21.15.31:6801/964869793] conn(0x7f75c00a14d0 0x7f75c00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:57.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:56 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:56.896+0000 7fc3fbae7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:20:57.260 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.113+0000 7fc35bbc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.113+0000 7fc35bbc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.113+0000 7fc35bbc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.113+0000 7fc35bbc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.113+0000 7fc35bbc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.115+0000 7fc35ebcc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.115+0000 7fc35ebcc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.115+0000 7fc35ebcc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.115+0000 7fc35ebcc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.261 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:20:57.115+0000 7fc35ebcc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:20:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-10-06T21:20:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:20:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: Standby manager daemon z restarted 2023-10-06T21:20:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: Standby manager daemon z started 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: Active manager daemon x restarted 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: Activating manager daemon x 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: mgrmap e21: x(active, starting, since 0.0147741s), standbys: z, y 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:20:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:20:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:20:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:20:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: Manager daemon x is now available 2023-10-06T21:20:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:20:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: Standby manager daemon z restarted 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: Standby manager daemon z started 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: Active manager daemon x restarted 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: Activating manager daemon x 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: mgrmap e21: x(active, starting, since 0.0147741s), standbys: z, y 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:20:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:20:57.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:20:57.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: Manager daemon x is now available 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:20:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: Standby manager daemon z restarted 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: Standby manager daemon z started 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:20:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.? 172.21.15.146:0/4151802131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: Active manager daemon x restarted 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: Activating manager daemon x 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: mgrmap e21: x(active, starting, since 0.0147741s), standbys: z, y 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:20:57.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:20:57.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:20:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: Manager daemon x is now available 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:20:57.803 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:20:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:57.924+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mon.1 v2:172.21.15.146:3300/0 9 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f75cc05cdd0 con 0x7f75d8074ef0 2023-10-06T21:20:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:57.925+0000 7f75bffff700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75c00282c0 0x7f75c00286a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:57.925+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f75d80612f0 con 0x7f75c00282c0 2023-10-06T21:20:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:57.928+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75c00282c0 0x7f75c00286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:57.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:57.932+0000 7f75d6ffd700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75c00282c0 0x7f75c00286a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009cf0 tx=0x7f75c8009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:58.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.002+0000 7f75bffff700 1 -- 172.21.15.31:0/3981761286 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f75d80612f0 con 0x7f75c00282c0 2023-10-06T21:20:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2023-10-06T21:20:58.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.004+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75c00282c0 msgr2=0x7f75c00286a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.004+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75c00282c0 0x7f75c00286a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009cf0 tx=0x7f75c8009260 comp rx=0 tx=0).stop 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.004+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 msgr2=0x7f75d819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.004+0000 7f75dd3ff700 1 --2- 172.21.15.31:0/3981761286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75d8074ef0 0x7f75d819d7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc005740 comp rx=0 tx=0).stop 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.004+0000 7f75d77fe700 1 -- 172.21.15.31:0/3981761286 reap_dead start 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.005+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 shutdown_connections 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.005+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 >> 172.21.15.31:0/3981761286 conn(0x7f75d80fc460 msgr2=0x7f75d80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.005+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 shutdown_connections 2023-10-06T21:20:58.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.005+0000 7f75dd3ff700 1 -- 172.21.15.31:0/3981761286 wait complete. 2023-10-06T21:20:58.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-06T21:20:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 -- 172.21.15.31:0/1489348895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 msgr2=0x7fef60073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 --2- 172.21.15.31:0/1489348895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60073340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef500098f0 tx=0x7fef50009c00 comp rx=0 tx=0).stop 2023-10-06T21:20:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 -- 172.21.15.31:0/1489348895 shutdown_connections 2023-10-06T21:20:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 --2- 172.21.15.31:0/1489348895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 0x7fef6010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 --2- 172.21.15.31:0/1489348895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60073340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 --2- 172.21.15.31:0/1489348895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 -- 172.21.15.31:0/1489348895 >> 172.21.15.31:0/1489348895 conn(0x7fef600fc430 msgr2=0x7fef600fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 -- 172.21.15.31:0/1489348895 shutdown_connections 2023-10-06T21:20:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.235+0000 7fef663ff700 1 -- 172.21.15.31:0/1489348895 wait complete. 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.236+0000 7fef663ff700 1 Processor -- start 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.236+0000 7fef663ff700 1 -- start start 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.236+0000 7fef663ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.236+0000 7fef663ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.236+0000 7fef663ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 0x7fef6019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef663ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6019a8e0 con 0x7fef60073910 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef663ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6019aa50 con 0x7fef60074ef0 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef663ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6019ac10 con 0x7fef60108dc0 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45514/0 (socket says 172.21.15.31:45514) 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 -- 172.21.15.31:0/3441656565 learned_addr learned my addr 172.21.15.31:0/3441656565 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef6499c700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 0x7fef6019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 -- 172.21.15.31:0/3441656565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 msgr2=0x7fef6019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 0x7fef6019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 -- 172.21.15.31:0/3441656565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 msgr2=0x7fef60199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 -- 172.21.15.31:0/3441656565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef540095f0 con 0x7fef60073910 2023-10-06T21:20:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.237+0000 7fef5ffff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b790 tx=0x7fef5000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.238+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5001c070 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.238+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50009520 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.238+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef5000f7b0 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.238+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50004930 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.238+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef601a14d0 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.239+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef60107470 con 0x7fef60073910 2023-10-06T21:20:58.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.241+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7fef50020070 con 0x7fef60073910 2023-10-06T21:20:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.242+0000 7fef5d7fa700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 0x7fef480a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.242+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fef500cdea0 con 0x7fef60073910 2023-10-06T21:20:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.242+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 0x7fef480a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.243+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 0x7fef480a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef60074c10 tx=0x7fef54009310 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.244+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef500977f0 con 0x7fef60073910 2023-10-06T21:20:58.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.500+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fef600612f0 con 0x7fef480a1280 2023-10-06T21:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.562+0000 7fef5d7fa700 1 -- 172.21.15.31:0/3441656565 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fef600612f0 con 0x7fef480a1280 2023-10-06T21:20:58.565 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 msgr2=0x7fef480a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 0x7fef480a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef60074c10 tx=0x7fef54009310 comp rx=0 tx=0).stop 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 msgr2=0x7fef60198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b790 tx=0x7fef5000b7c0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 shutdown_connections 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef60108dc0 0x7fef6019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef480a1280 0x7fef480a3740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60073910 0x7fef60198ec0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 --2- 172.21.15.31:0/3441656565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef60074ef0 0x7fef60199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 >> 172.21.15.31:0/3441656565 conn(0x7fef600fc430 msgr2=0x7fef600fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 shutdown_connections 2023-10-06T21:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.564+0000 7fef663ff700 1 -- 172.21.15.31:0/3441656565 wait complete. 2023-10-06T21:20:58.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T21:20:58.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 -- 172.21.15.31:0/774408491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c100eb0 msgr2=0x7f7e0c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 --2- 172.21.15.31:0/774408491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c1012d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098f0 tx=0x7f7e08009c00 comp rx=0 tx=0).stop 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 -- 172.21.15.31:0/774408491 shutdown_connections 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 --2- 172.21.15.31:0/774408491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102a50 0x7f7e0c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 --2- 172.21.15.31:0/774408491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c1012d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 --2- 172.21.15.31:0/774408491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c102090 0x7f7e0c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 -- 172.21.15.31:0/774408491 >> 172.21.15.31:0/774408491 conn(0x7f7e0c0fc430 msgr2=0x7f7e0c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 -- 172.21.15.31:0/774408491 shutdown_connections 2023-10-06T21:20:58.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.790+0000 7f7e141ff700 1 -- 172.21.15.31:0/774408491 wait complete. 2023-10-06T21:20:58.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.791+0000 7f7e141ff700 1 Processor -- start 2023-10-06T21:20:58.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.791+0000 7f7e141ff700 1 -- start start 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.791+0000 7f7e141ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.791+0000 7f7e141ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 0x7f7e0c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.791+0000 7f7e141ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 0x7f7e0c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e141ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c198740 con 0x7f7e0c102090 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e141ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1988b0 con 0x7f7e0c102a50 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e141ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c198a70 con 0x7f7e0c100eb0 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 0x7f7e0c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e11f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e11f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55482/0 (socket says 172.21.15.31:55482) 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e11f9b700 1 -- 172.21.15.31:0/1282088907 learned_addr learned my addr 172.21.15.31:0/1282088907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1279c700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 0x7f7e0c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 -- 172.21.15.31:0/1282088907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 msgr2=0x7f7e0c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 -- 172.21.15.31:0/1282088907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 msgr2=0x7f7e0c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 0x7f7e0c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1179a700 1 -- 172.21.15.31:0/1282088907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04009d00 con 0x7f7e0c102090 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e11f9b700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.792+0000 7f7e1279c700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 0x7f7e0c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e1179a700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 0x7f7e0c197290 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e9c0 tx=0x7f7dfc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004220 con 0x7f7e0c102090 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7dfc004380 con 0x7f7e0c102090 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e08009520 con 0x7f7e0c102090 2023-10-06T21:20:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003b40 con 0x7f7e0c102090 2023-10-06T21:20:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.793+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c19f460 con 0x7f7e0c102090 2023-10-06T21:20:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.794+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c108630 con 0x7f7e0c102090 2023-10-06T21:20:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.796+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f7dfc003ca0 con 0x7f7e0c102090 2023-10-06T21:20:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.797+0000 7f7e02ffd700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 0x7f7df80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.798+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f7dfc014070 con 0x7f7e0c102090 2023-10-06T21:20:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.798+0000 7f7e11f9b700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 0x7f7df80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.798+0000 7f7e11f9b700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 0x7f7df80a37c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800f9d0 tx=0x7f7e08019040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:58.801+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7dfc096400 con 0x7f7e0c102090 2023-10-06T21:20:59.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.058+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e0c02cfe0 con 0x7f7df80a1300 2023-10-06T21:20:59.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.068+0000 7f7e02ffd700 1 -- 172.21.15.31:0/1282088907 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f7e0c02cfe0 con 0x7f7df80a1300 2023-10-06T21:20:59.071 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T21:20:57", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 msgr2=0x7f7df80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 0x7f7df80a37c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800f9d0 tx=0x7f7e08019040 comp rx=0 tx=0).stop 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 msgr2=0x7f7e0c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 0x7f7e0c197290 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e9c0 tx=0x7f7dfc00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 shutdown_connections 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0c102a50 0x7f7e0c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e0c100eb0 0x7f7e0c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f7df80a1300 0x7f7df80a37c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 --2- 172.21.15.31:0/1282088907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e0c102090 0x7f7e0c197290 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 >> 172.21.15.31:0/1282088907 conn(0x7f7e0c0fc430 msgr2=0x7f7e0c105770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:59.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 shutdown_connections 2023-10-06T21:20:59.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.070+0000 7f7e141ff700 1 -- 172.21.15.31:0/1282088907 wait complete. 2023-10-06T21:20:59.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T21:20:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: [06/Oct/2023:21:20:57] ENGINE Bus STARTING 2023-10-06T21:20:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: [06/Oct/2023:21:20:57] ENGINE Serving on https://172.21.15.31:7150 2023-10-06T21:20:59.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: [06/Oct/2023:21:20:57] ENGINE Serving on http://172.21.15.31:8765 2023-10-06T21:20:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: [06/Oct/2023:21:20:57] ENGINE Bus STARTED 2023-10-06T21:20:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: mgrmap e22: x(active, since 1.02361s), standbys: z, y 2023-10-06T21:20:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:58 smithi031 ceph-mon[98539]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: [06/Oct/2023:21:20:57] ENGINE Bus STARTING 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: [06/Oct/2023:21:20:57] ENGINE Serving on https://172.21.15.31:7150 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: [06/Oct/2023:21:20:57] ENGINE Serving on http://172.21.15.31:8765 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: [06/Oct/2023:21:20:57] ENGINE Bus STARTED 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: mgrmap e22: x(active, since 1.02361s), standbys: z, y 2023-10-06T21:20:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:59.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:58 smithi146 ceph-mon[102809]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 -- 172.21.15.31:0/924074834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 msgr2=0x7f9b20100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 --2- 172.21.15.31:0/924074834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20100540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9b100098f0 tx=0x7f9b10009c00 comp rx=0 tx=0).stop 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 -- 172.21.15.31:0/924074834 shutdown_connections 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 --2- 172.21.15.31:0/924074834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 0x7f9b20100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 --2- 172.21.15.31:0/924074834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b200ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 --2- 172.21.15.31:0/924074834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20100540 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.298+0000 7f9b273ff700 1 -- 172.21.15.31:0/924074834 >> 172.21.15.31:0/924074834 conn(0x7f9b200fb340 msgr2=0x7f9b200fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:59.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.299+0000 7f9b273ff700 1 -- 172.21.15.31:0/924074834 shutdown_connections 2023-10-06T21:20:59.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.299+0000 7f9b273ff700 1 -- 172.21.15.31:0/924074834 wait complete. 2023-10-06T21:20:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.300+0000 7f9b273ff700 1 Processor -- start 2023-10-06T21:20:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.300+0000 7f9b273ff700 1 -- start start 2023-10-06T21:20:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 0x7f9b2019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019a840 con 0x7f9b201000c0 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019a9b0 con 0x7f9b200ff760 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b273ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019ab70 con 0x7f9b20100a80 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2519b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2519b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:51220/0 (socket says 172.21.15.31:51220) 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2519b700 1 -- 172.21.15.31:0/1930298393 learned_addr learned my addr 172.21.15.31:0/1930298393 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:59.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2599c700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 0x7f9b2019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 -- 172.21.15.31:0/1930298393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 msgr2=0x7f9b2019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 0x7f9b2019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 -- 172.21.15.31:0/1930298393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 msgr2=0x7f9b20198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2499a700 1 -- 172.21.15.31:0/1930298393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10009520 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.301+0000 7f9b2519b700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b2499a700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20199390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000c00 tx=0x7f9b1000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1001c070 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2019adf0 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b1000f7b0 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100038e0 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.302+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b201a1560 con 0x7f9b201000c0 2023-10-06T21:20:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.303+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2010a540 con 0x7f9b201000c0 2023-10-06T21:20:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.308+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f9b10020030 con 0x7f9b201000c0 2023-10-06T21:20:59.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.310+0000 7f9b167fc700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 0x7f9b0c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:59.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.310+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f9b100cdd40 con 0x7f9b201000c0 2023-10-06T21:20:59.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.310+0000 7f9b2519b700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 0x7f9b0c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:59.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.311+0000 7f9b2519b700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 0x7f9b0c0a37c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b20198ba0 tx=0x7f9b1c009310 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:59.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.311+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9b10097570 con 0x7f9b201000c0 2023-10-06T21:20:59.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: [06/Oct/2023:21:20:57] ENGINE Bus STARTING 2023-10-06T21:20:59.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: [06/Oct/2023:21:20:57] ENGINE Serving on https://172.21.15.31:7150 2023-10-06T21:20:59.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: [06/Oct/2023:21:20:57] ENGINE Serving on http://172.21.15.31:8765 2023-10-06T21:20:59.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: [06/Oct/2023:21:20:57] ENGINE Bus STARTED 2023-10-06T21:20:59.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: mgrmap e22: x(active, since 1.02361s), standbys: z, y 2023-10-06T21:20:59.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:20:59.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:58 smithi142 ceph-mon[102951]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:20:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.562+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9b200612f0 con 0x7f9b0c0a1300 2023-10-06T21:20:59.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.570+0000 7f9b167fc700 1 -- 172.21.15.31:0/1930298393 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f9b200612f0 con 0x7f9b0c0a1300 2023-10-06T21:20:59.574 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 msgr2=0x7f9b0c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 0x7f9b0c0a37c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b20198ba0 tx=0x7f9b1c009310 comp rx=0 tx=0).stop 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 msgr2=0x7f9b20199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20199390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000c00 tx=0x7f9b1000b790 comp rx=0 tx=0).stop 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 shutdown_connections 2023-10-06T21:20:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b20100a80 0x7f9b2019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b200ff760 0x7f9b20198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b0c0a1300 0x7f9b0c0a37c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 --2- 172.21.15.31:0/1930298393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b201000c0 0x7f9b20199390 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 >> 172.21.15.31:0/1930298393 conn(0x7f9b200fb340 msgr2=0x7f9b200fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 shutdown_connections 2023-10-06T21:20:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:20:59.573+0000 7f9b273ff700 1 -- 172.21.15.31:0/1930298393 wait complete. 2023-10-06T21:20:59.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-06T21:20:59.671 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-06T21:20:59.683 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-06T21:20:59.697 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:20:59.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-06T21:20:59.702 INFO:teuthology.run_tasks:Running task workunit... 2023-10-06T21:20:59.724 INFO:tasks.workunit:Pulling workunits from ref 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:20:59.724 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-06T21:20:59.726 INFO:tasks.workunit:timeout=3h 2023-10-06T21:20:59.727 INFO:tasks.workunit:cleanup=True 2023-10-06T21:20:59.727 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:59.777 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:59.777 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2023-10-06T21:20:59.777 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 79h/121d Inode: 1 Links: 2 2023-10-06T21:20:59.777 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:20:59.777 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:fusefs_t:s0 2023-10-06T21:20:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 21:19:00.846695686 +0000 2023-10-06T21:20:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 21:19:00.846695686 +0000 2023-10-06T21:20:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 21:20:38.388296591 +0000 2023-10-06T21:20:59.778 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-10-06T21:20:59.778 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-06T21:20:59.778 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-06T21:20:59.869 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:20:59.918 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-06T21:21:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:59 smithi031 ceph-mon[98539]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:59 smithi031 ceph-mon[98539]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:59 smithi031 ceph-mon[98539]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:20:59 smithi031 ceph-mon[98539]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:21:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:59 smithi146 ceph-mon[102809]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:59 smithi146 ceph-mon[102809]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:59 smithi146 ceph-mon[102809]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:20:59 smithi146 ceph-mon[102809]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:21:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:59 smithi142 ceph-mon[102951]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:59 smithi142 ceph-mon[102951]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:00.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:59 smithi142 ceph-mon[102951]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:00.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:20:59 smithi142 ceph-mon[102951]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:21:00.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:21:00.573+0000 7fc33c7c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:21:00.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:21:00.573+0000 7fc33c7c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:21:00.926 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:21:00.573+0000 7fc33c7c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:21:00.926 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:21:00.573+0000 7fc33c7c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:21:00.926 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:21:00.573+0000 7fc33c7c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:21:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-mon[98539]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:00 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:00 smithi146 ceph-mon[102809]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:00 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:00 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:01.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:00 smithi142 ceph-mon[102951]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:21:01.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:00 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:01.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:00 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:01 smithi031 ceph-mon[98539]: mgrmap e24: x(active, since 5s), standbys: z, y 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:01 smithi146 ceph-mon[102809]: mgrmap e24: x(active, since 5s), standbys: z, y 2023-10-06T21:21:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:02.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:01 smithi142 ceph-mon[102951]: mgrmap e24: x(active, since 5s), standbys: z, y 2023-10-06T21:21:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:21:03.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:21:03.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:21:03.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:03 smithi031 ceph-mon[98539]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:21:03.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:03 smithi146 ceph-mon[102809]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:21:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:21:03.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:21:03.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:03.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:03.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Updating smithi031:/etc/ceph/ceph.conf 2023-10-06T21:21:03.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Updating smithi142:/etc/ceph/ceph.conf 2023-10-06T21:21:03.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: Updating smithi146:/etc/ceph/ceph.conf 2023-10-06T21:21:03.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:03 smithi142 ceph-mon[102951]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:21:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:04 smithi142 ceph-mon[102951]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:04 smithi031 ceph-mon[98539]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:05.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:04 smithi146 ceph-mon[102809]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:21:06.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 170 KiB/s wr, 93 op/s 2023-10-06T21:21:06.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:06.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:06.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:06.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:05 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 170 KiB/s wr, 93 op/s 2023-10-06T21:21:06.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:06.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:06.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:06.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:05 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: Updating smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: Updating smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: Updating smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 170 KiB/s wr, 93 op/s 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:06.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:05 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:21:08.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:07 smithi031 ceph-mon[98539]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-10-06T21:21:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:07 smithi146 ceph-mon[102809]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-10-06T21:21:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:07 smithi142 ceph-mon[102951]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-10-06T21:21:10.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:09 smithi142 ceph-mon[102951]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 109 KiB/s wr, 64 op/s 2023-10-06T21:21:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:09 smithi031 ceph-mon[98539]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 109 KiB/s wr, 64 op/s 2023-10-06T21:21:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:09 smithi146 ceph-mon[102809]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 109 KiB/s wr, 64 op/s 2023-10-06T21:21:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:11 smithi031 ceph-mon[98539]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:11 smithi146 ceph-mon[102809]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:11 smithi142 ceph-mon[102951]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 -- 172.21.15.31:0/3090784182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1134101a80 msgr2=0x7f1134105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 --2- 172.21.15.31:0/3090784182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1134101a80 0x7f1134105ac0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1128009990 tx=0x7f1128009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 -- 172.21.15.31:0/3090784182 shutdown_connections 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 --2- 172.21.15.31:0/3090784182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341061f0 0x7f113410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 --2- 172.21.15.31:0/3090784182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11341010d0 0x7f11341014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 --2- 172.21.15.31:0/3090784182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1134101a80 0x7f1134105ac0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.960+0000 7f1139dff700 1 -- 172.21.15.31:0/3090784182 >> 172.21.15.31:0/3090784182 conn(0x7f11340fc950 msgr2=0x7f11340fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.961+0000 7f1139dff700 1 -- 172.21.15.31:0/3090784182 shutdown_connections 2023-10-06T21:21:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.961+0000 7f1139dff700 1 -- 172.21.15.31:0/3090784182 wait complete. 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 Processor -- start 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 -- start start 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341010d0 0x7f1134199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f113419dcf0 0x7f113419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419e770 con 0x7f11341061f0 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419e8e0 con 0x7f113419dcf0 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1139dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419eaa0 con 0x7f11341010d0 2023-10-06T21:21:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60074/0 (socket says 172.21.15.31:60074) 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.962+0000 7f1132ffd700 1 -- 172.21.15.31:0/733027202 learned_addr learned my addr 172.21.15.31:0/733027202 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f11337fe700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341010d0 0x7f1134199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1133fff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f113419dcf0 0x7f113419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 -- 172.21.15.31:0/733027202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341010d0 msgr2=0x7f1134199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341010d0 0x7f1134199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 -- 172.21.15.31:0/733027202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f113419dcf0 msgr2=0x7f113419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f113419dcf0 0x7f113419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 -- 172.21.15.31:0/733027202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280095f0 con 0x7f11341061f0 2023-10-06T21:21:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1132ffd700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1128005740 tx=0x7f1128009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.963+0000 7f1130ff9700 1 -- 172.21.15.31:0/733027202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112801d070 con 0x7f11341061f0 2023-10-06T21:21:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.964+0000 7f1130ff9700 1 -- 172.21.15.31:0/733027202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1128003ec0 con 0x7f11341061f0 2023-10-06T21:21:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.964+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11341a5fa0 con 0x7f11341061f0 2023-10-06T21:21:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.964+0000 7f1130ff9700 1 -- 172.21.15.31:0/733027202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128022b20 con 0x7f11341061f0 2023-10-06T21:21:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.964+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11341a63c0 con 0x7f11341061f0 2023-10-06T21:21:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.967+0000 7f1130ff9700 1 -- 172.21.15.31:0/733027202 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f1128004030 con 0x7f11341061f0 2023-10-06T21:21:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.968+0000 7f114d767b80 1 Processor -- start 2023-10-06T21:21:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.968+0000 7f1130ff9700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 0x7f111c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f1130ff9700 1 -- 172.21.15.31:0/733027202 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f11280cea70 con 0x7f11341061f0 2023-10-06T21:21:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 -- start start 2023-10-06T21:21:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f11337fe700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 0x7f111c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f48c00 0x55c4c4f48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f49520 0x55c4c4f53e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4eb2400 con 0x55c4c4f543d0 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4ea3bc0 con 0x55c4c4f49520 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f114d767b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4ea14e0 con 0x55c4c4f48c00 2023-10-06T21:21:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f1133fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f1133fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60082/0 (socket says 172.21.15.31:60082) 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.969+0000 7f1133fff700 1 -- 172.21.15.31:0/3948479029 learned_addr learned my addr 172.21.15.31:0/3948479029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f11337fe700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 0x7f111c0a3c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1124007c30 tx=0x7f11240073d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f11337fe700 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f48c00 0x55c4c4f48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1132ffd700 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f49520 0x55c4c4f53e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 -- 172.21.15.31:0/3948479029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f48c00 msgr2=0x55c4c4f48fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f48c00 0x55c4c4f48fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 -- 172.21.15.31:0/3948479029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f49520 msgr2=0x55c4c4f53e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f49520 0x55c4c4f53e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 -- 172.21.15.31:0/3948479029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c4cc7d80 con 0x55c4c4f543d0 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1133fff700 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f112c003fa0 tx=0x7f112c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e61591898c56a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.970+0000 7f1119ffb700 1 -- 172.21.15.31:0/3948479029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c019070 con 0x55c4c4f543d0 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f1119ffb700 1 -- 172.21.15.31:0/3948479029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f112c00f040 con 0x55c4c4f543d0 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f1119ffb700 1 -- 172.21.15.31:0/3948479029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0079f0 con 0x55c4c4f543d0 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 -- 172.21.15.31:0/3948479029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 msgr2=0x55c4c4f567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f112c003fa0 tx=0x7f112c009ba0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 -- 172.21.15.31:0/3948479029 shutdown_connections 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f567c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f48c00 0x55c4c4f48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 --2- 172.21.15.31:0/3948479029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f49520 0x55c4c4f53e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 -- 172.21.15.31:0/3948479029 >> 172.21.15.31:0/3948479029 conn(0x55c4c4ebe200 msgr2=0x55c4c4f45080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 -- 172.21.15.31:0/3948479029 shutdown_connections 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.971+0000 7f114d767b80 1 -- 172.21.15.31:0/3948479029 wait complete. 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.972+0000 7f114d767b80 1 -- start start 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.972+0000 7f114d767b80 1 Processor -- start 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f53e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f543d0 0x55c4c4f517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4e9cdd0 con 0x55c4c4f49520 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4e396b0 con 0x55c4c4f48c00 2023-10-06T21:21:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4e39350 con 0x55c4c4f543d0 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f114d767b80 10 client.0 _ll_register_callbacks cb 0x55c4c4c967a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60094/0 (socket says 172.21.15.31:60094) 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 -- 172.21.15.31:0/3384903252 learned_addr learned my addr 172.21.15.31:0/3384903252 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f11337fe700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f53e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1133fff700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f543d0 0x55c4c4f517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 -- 172.21.15.31:0/3384903252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f543d0 msgr2=0x55c4c4f517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f543d0 0x55c4c4f517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 -- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 msgr2=0x55c4c4f53e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f53e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.973+0000 7f1132ffd700 1 -- 172.21.15.31:0/3384903252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c003bd0 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1132ffd700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f11280cc770 tx=0x7f112809bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948c5ce6441f320c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1112ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280862b0 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f112809b840 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4c4f3cd50 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112809f990 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f114d767b80 10 client.15666 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4c4cb0c00 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.974+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c4c4f3cd50 con 0x55c4c4f49520 2023-10-06T21:21:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f11280cc360 con 0x55c4c4f49520 2023-10-06T21:21:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 20 client.15666 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4f63b70 0x55c4c4f65f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 4 client.15666 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4c4e95db0 con 0x55c4c4f63b70 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.975+0000 7f11337fe700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4f63b70 0x55c4c4f65f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.976+0000 7f11337fe700 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4f63b70 0x55c4c4f65f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95875f087ec536e1 server_cookie=9176b229c97ea73 in_seq=0 out_seq=0 2023-10-06T21:21:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.976+0000 7f1112ffd700 10 client.15666 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f1112ffd700 1 -- 172.21.15.31:0/3384903252 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c4c4e95db0 con 0x55c4c4f63b70 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f1112ffd700 10 client.15666 handle_command_reply: tid=0 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 client.15666 shutdown 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 msgr2=0x55c4c4f51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f11280cc770 tx=0x7f112809bd80 comp rx=0 tx=0).stop 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 shutdown_connections 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f543d0 0x55c4c4f517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f53e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4f63b70 0x55c4c4f65f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 --2- 172.21.15.31:0/3384903252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f49520 0x55c4c4f51080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 >> 172.21.15.31:0/3384903252 conn(0x55c4c4ebe200 msgr2=0x55c4c4f44c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f1112ffd700 0 client.15666 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 shutdown_connections 2023-10-06T21:21:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 1 -- 172.21.15.31:0/3384903252 wait complete. 2023-10-06T21:21:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.977+0000 7f114d767b80 20 client.15666 trim_cache size 0 max 16384 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.993+0000 7f114d767b80 1 Processor -- start 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.993+0000 7f114d767b80 1 -- start start 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4d1d8c0 con 0x55c4c4f48c00 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4f5b450 con 0x55c4c4f543d0 2023-10-06T21:21:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f114d767b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4f4cf10 con 0x55c4c4f49520 2023-10-06T21:21:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f11337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f11337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60102/0 (socket says 172.21.15.31:60102) 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f11337fe700 1 -- 172.21.15.31:0/31819711 learned_addr learned my addr 172.21.15.31:0/31819711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f1133fff700 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.994+0000 7f1132ffd700 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 -- 172.21.15.31:0/31819711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 msgr2=0x55c4c4f539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 -- 172.21.15.31:0/31819711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f543d0 msgr2=0x55c4c4f4c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 -- 172.21.15.31:0/31819711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c4f569d0 con 0x55c4c4f48c00 2023-10-06T21:21:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f11337fe700 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1124007270 tx=0x7f1124013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e63d9a7e1745f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f111a7fc700 1 -- 172.21.15.31:0/31819711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124024040 con 0x55c4c4f48c00 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f111a7fc700 1 -- 172.21.15.31:0/31819711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1124015070 con 0x55c4c4f48c00 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.995+0000 7f111a7fc700 1 -- 172.21.15.31:0/31819711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124011440 con 0x55c4c4f48c00 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 -- 172.21.15.31:0/31819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 msgr2=0x55c4c4f5af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1124007270 tx=0x7f1124013000 comp rx=0 tx=0).stop 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 -- 172.21.15.31:0/31819711 shutdown_connections 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f48c00 0x55c4c4f5af10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 --2- 172.21.15.31:0/31819711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 -- 172.21.15.31:0/31819711 >> 172.21.15.31:0/31819711 conn(0x55c4c4ebe200 msgr2=0x55c4c4f5f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 -- 172.21.15.31:0/31819711 shutdown_connections 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.996+0000 7f114d767b80 1 -- 172.21.15.31:0/31819711 wait complete. 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 -- start start 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 Processor -- start 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f4d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4f4e330 con 0x55c4c4f543d0 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4f4e470 con 0x55c4c4f48c00 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c4f4e630 con 0x55c4c4f49520 2023-10-06T21:21:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.997+0000 7f114d767b80 10 client.0 _ll_register_callbacks cb 0x7f112809b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60114/0 (socket says 172.21.15.31:60114) 2023-10-06T21:21:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 -- 172.21.15.31:0/704632744 learned_addr learned my addr 172.21.15.31:0/704632744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1132ffd700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f4d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f11337fe700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 -- 172.21.15.31:0/704632744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 msgr2=0x55c4c4f4d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f4d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 -- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 msgr2=0x55c4c4f4d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f4d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.998+0000 7f1133fff700 1 -- 172.21.15.31:0/704632744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1124010cf0 con 0x55c4c4f543d0 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1133fff700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f112c003b90 tx=0x7f112c009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f82db3031fea680 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c019040 con 0x55c4c4f543d0 2023-10-06T21:21:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f112c00f040 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f112c007990 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112c007b50 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f114d767b80 10 client.15672 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4c4cee1f0 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f112c033080 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:12.999+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f112c02d030 con 0x55c4c4f543d0 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 20 client.15672 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4fb8440 0x55c4c4fba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 4 client.15672 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4c4e95ad0 con 0x55c4c4fb8440 2023-10-06T21:21:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.000+0000 7f11337fe700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4fb8440 0x55c4c4fba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.001+0000 7f11337fe700 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4fb8440 0x55c4c4fba860 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0c92a51ead74d1 server_cookie=adb8f0c1daab9d77 in_seq=0 out_seq=0 2023-10-06T21:21:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.001+0000 7f1112ffd700 10 client.15672 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f1112ffd700 1 -- 172.21.15.31:0/704632744 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4c4e95ad0 con 0x55c4c4fb8440 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f1112ffd700 10 client.15672 handle_command_reply: tid=0 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 client.15672 shutdown 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 msgr2=0x55c4c4f4dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f112c003b90 tx=0x7f112c009460 comp rx=0 tx=0).stop 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 shutdown_connections 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c4c4fb8440 0x55c4c4fba860 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f1112ffd700 0 client.15672 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.014+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c4c4f543d0 0x55c4c4f4dda0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4c4f48c00 0x55c4c4f4d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 1 --2- 172.21.15.31:0/704632744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4c4f49520 0x55c4c4f4d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 >> 172.21.15.31:0/704632744 conn(0x55c4c4ebe200 msgr2=0x55c4c4f4c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 shutdown_connections 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 1 -- 172.21.15.31:0/704632744 wait complete. 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f114d767b80 20 client.15672 trim_cache size 0 max 16384 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:21:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:21:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:21:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 msgr2=0x7f111c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 0x7f111c0a3c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1124007c30 tx=0x7f11240073d0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 msgr2=0x7f1134199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.015+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1128005740 tx=0x7f1128009580 comp rx=0 tx=0).stop 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 shutdown_connections 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f113419dcf0 0x7f113419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11341010d0 0x7f1134199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f111c0a1790 0x7f111c0a3c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 --2- 172.21.15.31:0/733027202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11341061f0 0x7f1134199680 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 >> 172.21.15.31:0/733027202 conn(0x7f11340fc950 msgr2=0x7f11340fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 shutdown_connections 2023-10-06T21:21:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:13.016+0000 7f1139dff700 1 -- 172.21.15.31:0/733027202 wait complete. 2023-10-06T21:21:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:13 smithi031 ceph-mon[98539]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:13 smithi146 ceph-mon[102809]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:13 smithi142 ceph-mon[102951]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 58 op/s 2023-10-06T21:21:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:15 smithi031 ceph-mon[98539]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 63 op/s 2023-10-06T21:21:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:15 smithi146 ceph-mon[102809]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 63 op/s 2023-10-06T21:21:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:15 smithi142 ceph-mon[102951]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 63 op/s 2023-10-06T21:21:17.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 -- 172.21.15.31:0/2987777058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c14106040 msgr2=0x7f8c1410dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 --2- 172.21.15.31:0/2987777058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c14106040 0x7f8c1410dbc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00b0a0 tx=0x7f8c0c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 -- 172.21.15.31:0/2987777058 shutdown_connections 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 --2- 172.21.15.31:0/2987777058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c14106040 0x7f8c1410dbc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 --2- 172.21.15.31:0/2987777058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f20 0x7f8c14101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 --2- 172.21.15.31:0/2987777058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c141018d0 0x7f8c14105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 -- 172.21.15.31:0/2987777058 >> 172.21.15.31:0/2987777058 conn(0x7f8c140fc760 msgr2=0x7f8c140feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 -- 172.21.15.31:0/2987777058 shutdown_connections 2023-10-06T21:21:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.562+0000 7f8c1919d700 1 -- 172.21.15.31:0/2987777058 wait complete. 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.563+0000 7f8c1919d700 1 Processor -- start 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.563+0000 7f8c1919d700 1 -- start start 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c14100f20 0x7f8c14198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106040 0x7f8c141994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419e3d0 con 0x7f8c1419d950 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419e540 con 0x7f8c14106040 2023-10-06T21:21:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1919d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419e700 con 0x7f8c14100f20 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60142/0 (socket says 172.21.15.31:60142) 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 -- 172.21.15.31:0/478757109 learned_addr learned my addr 172.21.15.31:0/478757109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c13fff700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c14100f20 0x7f8c14198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c137fe700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106040 0x7f8c141994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 -- 172.21.15.31:0/478757109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c14100f20 msgr2=0x7f8c14198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c14100f20 0x7f8c14198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 -- 172.21.15.31:0/478757109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106040 msgr2=0x7f8c141994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106040 0x7f8c141994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.564+0000 7f8c1899c700 1 -- 172.21.15.31:0/478757109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009d00 con 0x7f8c1419d950 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c1899c700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bd00 tx=0x7f8c0c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c117fa700 1 -- 172.21.15.31:0/478757109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00e040 con 0x7f8c1419d950 2023-10-06T21:21:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c117fa700 1 -- 172.21.15.31:0/478757109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c004e40 con 0x7f8c1419d950 2023-10-06T21:21:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c141a5c00 con 0x7f8c1419d950 2023-10-06T21:21:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c117fa700 1 -- 172.21.15.31:0/478757109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0052a0 con 0x7f8c1419d950 2023-10-06T21:21:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.565+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a60d0 con 0x7f8c1419d950 2023-10-06T21:21:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.568+0000 7f8c117fa700 1 -- 172.21.15.31:0/478757109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8c0c005400 con 0x7f8c1419d950 2023-10-06T21:21:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 Processor -- start 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c117fa700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 0x7f8bfc0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 -- start start 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c117fa700 1 -- 172.21.15.31:0/478757109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8c0c0ce960 con 0x7f8c1419d950 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c137fe700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 0x7f8bfc0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 0x560c908d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c908e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90840460 con 0x560c908d7bc0 2023-10-06T21:21:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90831c20 con 0x560c908e2a70 2023-10-06T21:21:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.570+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9082f540 con 0x560c908d72a0 2023-10-06T21:21:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 0x560c908d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60158/0 (socket says 172.21.15.31:60158) 2023-10-06T21:21:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c137fe700 1 -- 172.21.15.31:0/2389275897 learned_addr learned my addr 172.21.15.31:0/2389275897 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c1899c700 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c908e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 -- 172.21.15.31:0/2389275897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 msgr2=0x560c908e4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c908e4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 -- 172.21.15.31:0/2389275897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 msgr2=0x560c908e2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c13fff700 1 -- 172.21.15.31:0/2389275897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c90655de0 con 0x560c908d72a0 2023-10-06T21:21:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.571+0000 7f8c137fe700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 0x7f8bfc0a3cf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8c080134c0 tx=0x7f8c08013880 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.572+0000 7f8c13fff700 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 0x560c908d7680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009cb0 tx=0x7f8c04011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb25b85ee642b59e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.572+0000 7f8bfa7fc700 1 -- 172.21.15.31:0/2389275897 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04011d20 con 0x560c908d72a0 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.572+0000 7f8bfa7fc700 1 -- 172.21.15.31:0/2389275897 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c04011e80 con 0x560c908d72a0 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.572+0000 7f8bfa7fc700 1 -- 172.21.15.31:0/2389275897 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04005280 con 0x560c908d72a0 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/2389275897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 msgr2=0x560c908d7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 0x560c908d7680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009cb0 tx=0x7f8c04011440 comp rx=0 tx=0).stop 2023-10-06T21:21:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/2389275897 shutdown_connections 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c908e4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d72a0 0x560c908d7680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/2389275897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/2389275897 >> 172.21.15.31:0/2389275897 conn(0x560c9084c260 msgr2=0x560c908d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/2389275897 shutdown_connections 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/2389275897 wait complete. 2023-10-06T21:21:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.573+0000 7f8c2dd6ab80 1 -- start start 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 Processor -- start 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 0x560c908dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908e0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9082ae30 con 0x560c908d72a0 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c907c7710 con 0x560c908d7bc0 2023-10-06T21:21:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c907c73b0 con 0x560c908e2a70 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c2dd6ab80 10 client.0 _ll_register_callbacks cb 0x560c90624800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60168/0 (socket says 172.21.15.31:60168) 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c13fff700 1 -- 172.21.15.31:0/1096930037 learned_addr learned my addr 172.21.15.31:0/1096930037 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.574+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 0x560c908dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c1899c700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 -- 172.21.15.31:0/1096930037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 msgr2=0x560c908e0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908e0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 -- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 msgr2=0x560c908dfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 0x560c908dfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 -- 172.21.15.31:0/1096930037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c040098e0 con 0x560c908d72a0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 0x560c908dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005c50 tx=0x7f8c04011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3483a5340b981632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8bf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04011ea0 con 0x560c908d72a0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c90823e10 con 0x560c908d72a0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8c04004260 con 0x560c908d72a0 2023-10-06T21:21:17.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c04004420 con 0x560c908d72a0 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.575+0000 7f8c2dd6ab80 10 client.15684 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560c9063ec60 con 0x560c908d72a0 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c90823e10 con 0x560c908d72a0 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c04004f60 con 0x560c908d72a0 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 20 client.15684 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c908f2490 0x560c908f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 4 client.15684 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.576+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c90823b30 con 0x560c908f2490 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.577+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c908f2490 0x560c908f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.577+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c908f2490 0x560c908f48b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79d404e6b5f7b50 server_cookie=e4571d63953c259e in_seq=0 out_seq=0 2023-10-06T21:21:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.577+0000 7f8bf37fe700 10 client.15684 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.578+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1096930037 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c90823b30 con 0x560c908f2490 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.578+0000 7f8bf37fe700 10 client.15684 handle_command_reply: tid=0 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.578+0000 7f8c2dd6ab80 1 client.15684 shutdown 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 msgr2=0x560c908df7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005c50 tx=0x7f8c04011bd0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 shutdown_connections 2023-10-06T21:21:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908e0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c908f2490 0x560c908f48b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c908df7d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1096930037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d7bc0 0x560c908dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 >> 172.21.15.31:0/1096930037 conn(0x560c9084c260 msgr2=0x560c908d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8bf37fe700 0 client.15684 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 shutdown_connections 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1096930037 wait complete. 2023-10-06T21:21:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.579+0000 7f8c2dd6ab80 20 client.15684 trim_cache size 0 max 16384 2023-10-06T21:21:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.596+0000 7f8c2dd6ab80 1 Processor -- start 2023-10-06T21:21:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 -- start start 2023-10-06T21:21:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 0x560c908da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c906ab920 con 0x560c908d7bc0 2023-10-06T21:21:17.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c908e9b60 con 0x560c908d72a0 2023-10-06T21:21:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c908db670 con 0x560c908e2a70 2023-10-06T21:21:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60180/0 (socket says 172.21.15.31:60180) 2023-10-06T21:21:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c137fe700 1 -- 172.21.15.31:0/1071943931 learned_addr learned my addr 172.21.15.31:0/1071943931 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c1899c700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.597+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 0x560c908da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 -- 172.21.15.31:0/1071943931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 msgr2=0x560c908dfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908dfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 -- 172.21.15.31:0/1071943931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 msgr2=0x560c908da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 0x560c908da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 -- 172.21.15.31:0/1071943931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9067c250 con 0x560c908d7bc0 2023-10-06T21:21:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 0x560c908da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800bc40 tx=0x7f8c0801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8842e15e9caad821 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8bfaffd700 1 -- 172.21.15.31:0/1071943931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08025070 con 0x560c908d7bc0 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8bfaffd700 1 -- 172.21.15.31:0/1071943931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c08019070 con 0x560c908d7bc0 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.598+0000 7f8bfaffd700 1 -- 172.21.15.31:0/1071943931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0801e3d0 con 0x560c908d7bc0 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1071943931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 msgr2=0x560c908e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800bc40 tx=0x7f8c0801d000 comp rx=0 tx=0).stop 2023-10-06T21:21:17.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1071943931 shutdown_connections 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908e2a70 0x560c908dfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908d72a0 0x560c908da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1071943931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d7bc0 0x560c908e95c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1071943931 >> 172.21.15.31:0/1071943931 conn(0x560c9084c260 msgr2=0x560c908daad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1071943931 shutdown_connections 2023-10-06T21:21:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1071943931 wait complete. 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.599+0000 7f8c2dd6ab80 1 -- start start 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 Processor -- start 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d7bc0 0x560c9093eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c9093a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9093a940 con 0x560c908d72a0 2023-10-06T21:21:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9093aab0 con 0x560c908e2a70 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9093ac70 con 0x560c908d7bc0 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c2dd6ab80 10 client.0 _ll_register_callbacks cb 0x7f8c0800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60194/0 (socket says 172.21.15.31:60194) 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.600+0000 7f8c13fff700 1 -- 172.21.15.31:0/1973905402 learned_addr learned my addr 172.21.15.31:0/1973905402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d7bc0 0x560c9093eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c1899c700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c9093a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 -- 172.21.15.31:0/1973905402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d7bc0 msgr2=0x560c9093eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d7bc0 0x560c9093eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 -- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 msgr2=0x560c9093a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c9093a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 -- 172.21.15.31:0/1973905402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c080206f0 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c1899c700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c9093a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c13fff700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8c04011480 tx=0x7f8c04004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757c323183979790 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8bf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04005660 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c908cb160 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.601+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8c04011020 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c040054e0 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 10 client.15690 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560c906542b0 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c04040080 con 0x560c908d72a0 2023-10-06T21:21:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c040039b0 con 0x560c908d72a0 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 20 client.15690 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c909473d0 0x560c909497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 4 client.15690 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:17.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.602+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c90676f70 con 0x560c909473d0 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c909473d0 0x560c909497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8bf37fe700 10 client.15690 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8c137fe700 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c909473d0 0x560c909497f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3172d2f38d244ae8 server_cookie=3902f6a6ff137f0d in_seq=0 out_seq=0 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8bf37fe700 1 -- 172.21.15.31:0/1973905402 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c90676f70 con 0x560c909473d0 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8bf37fe700 10 client.15690 handle_command_reply: tid=0 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.603+0000 7f8c2dd6ab80 1 client.15690 shutdown 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 msgr2=0x560c9093e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8c04011480 tx=0x7f8c04004490 comp rx=0 tx=0).stop 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 shutdown_connections 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c908e2a70 0x560c9093a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560c909473d0 0x560c909497f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c908d72a0 0x560c9093e5f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 --2- 172.21.15.31:0/1973905402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c908d7bc0 0x560c9093eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8bf37fe700 0 client.15690 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 >> 172.21.15.31:0/1973905402 conn(0x560c9084c260 msgr2=0x560c908dfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 shutdown_connections 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 1 -- 172.21.15.31:0/1973905402 wait complete. 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c2dd6ab80 20 client.15690 trim_cache size 0 max 16384 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 msgr2=0x7f8bfc0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.604+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 0x7f8bfc0a3cf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8c080134c0 tx=0x7f8c08013880 comp rx=0 tx=0).stop 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 msgr2=0x7f8c1419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bd00 tx=0x7f8c0c00bd30 comp rx=0 tx=0).stop 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 shutdown_connections 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bfc0a1830 0x7f8bfc0a3cf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c1419d950 0x7f8c1419ddd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c14100f20 0x7f8c14198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 --2- 172.21.15.31:0/478757109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106040 0x7f8c141994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 >> 172.21.15.31:0/478757109 conn(0x7f8c140fc760 msgr2=0x7f8c1410c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 shutdown_connections 2023-10-06T21:21:17.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.605+0000 7f8c1919d700 1 -- 172.21.15.31:0/478757109 wait complete. 2023-10-06T21:21:17.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:21:17.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.827+0000 7fc9c95ff700 1 -- 172.21.15.31:0/1502854425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c4073c00 msgr2=0x7fc9c410d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.827+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/1502854425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c4073c00 0x7fc9c410d140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009990 tx=0x7fc9b8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 -- 172.21.15.31:0/1502854425 shutdown_connections 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/1502854425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c410d680 0x7fc9c410faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/1502854425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c40732e0 0x7fc9c40736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/1502854425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c4073c00 0x7fc9c410d140 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 -- 172.21.15.31:0/1502854425 >> 172.21.15.31:0/1502854425 conn(0x7fc9c40fc8d0 msgr2=0x7fc9c40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 -- 172.21.15.31:0/1502854425 shutdown_connections 2023-10-06T21:21:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.828+0000 7fc9c95ff700 1 -- 172.21.15.31:0/1502854425 wait complete. 2023-10-06T21:21:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 Processor -- start 2023-10-06T21:21:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 -- start start 2023-10-06T21:21:17.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c40732e0 0x7fc9c4100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 0x7fc9c41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a9810 con 0x7fc9c410d680 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a9980 con 0x7fc9c4073c00 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a9b40 con 0x7fc9c40732e0 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60214/0 (socket says 172.21.15.31:60214) 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.829+0000 7fc9c37fe700 1 -- 172.21.15.31:0/3131682231 learned_addr learned my addr 172.21.15.31:0/3131682231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c2ffd700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c40732e0 0x7fc9c4100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c27fc700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 0x7fc9c41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 -- 172.21.15.31:0/3131682231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c40732e0 msgr2=0x7fc9c4100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c40732e0 0x7fc9c4100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 -- 172.21.15.31:0/3131682231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 msgr2=0x7fc9c41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 0x7fc9c41012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 -- 172.21.15.31:0/3131682231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b80095f0 con 0x7fc9c410d680 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c27fc700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 0x7fc9c41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c37fe700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc007ee0 tx=0x7fc9bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc017070 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9bc00f040 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c41a9e20 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc005710 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.830+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c41aa2b0 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.831+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c4067cd0 con 0x7fc9c410d680 2023-10-06T21:21:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.833+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fc9bc005870 con 0x7fc9c410d680 2023-10-06T21:21:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.836+0000 7fc9abfff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 0x7fc9ac0a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.836+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc9bc0ccc60 con 0x7fc9c410d680 2023-10-06T21:21:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.836+0000 7fc9c2ffd700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 0x7fc9ac0a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.837+0000 7fc9c2ffd700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 0x7fc9ac0a3c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009bf0 tx=0x7fc9b4009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:17.840+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc9bc095c80 con 0x7fc9c410d680 2023-10-06T21:21:18.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.103+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c41022d0 con 0x7fc9c410d680 2023-10-06T21:21:18.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.104+0000 7fc9abfff700 1 -- 172.21.15.31:0/3131682231 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7fc9bc0992a0 con 0x7fc9c410d680 2023-10-06T21:21:18.109 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:21:18.109 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:21:18.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 msgr2=0x7fc9ac0a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 0x7fc9ac0a3c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009bf0 tx=0x7fc9b4009260 comp rx=0 tx=0).stop 2023-10-06T21:21:18.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 msgr2=0x7fc9c4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc007ee0 tx=0x7fc9bc00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 shutdown_connections 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc9ac0a1760 0x7fc9ac0a3c20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9c410d680 0x7fc9c4101820 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c40732e0 0x7fc9c4100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 --2- 172.21.15.31:0/3131682231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9c4073c00 0x7fc9c41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 >> 172.21.15.31:0/3131682231 conn(0x7fc9c40fc8d0 msgr2=0x7fc9c410f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 shutdown_connections 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.106+0000 7fc9c95ff700 1 -- 172.21.15.31:0/3131682231 wait complete. 2023-10-06T21:21:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:21:18.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T21:21:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:17 smithi031 ceph-mon[98539]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T21:21:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:17 smithi146 ceph-mon[102809]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.348+0000 7f3ae9bff700 1 -- 172.21.15.31:0/960503806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41010b0 msgr2=0x7f3ae4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.348+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/960503806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41010b0 0x7f3ae4101490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009990 tx=0x7f3ad4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 -- 172.21.15.31:0/960503806 shutdown_connections 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/960503806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae41061d0 0x7f3ae410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/960503806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41010b0 0x7f3ae4101490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/960503806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae4101a60 0x7f3ae4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 -- 172.21.15.31:0/960503806 >> 172.21.15.31:0/960503806 conn(0x7f3ae40fc950 msgr2=0x7f3ae40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 -- 172.21.15.31:0/960503806 shutdown_connections 2023-10-06T21:21:18.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.349+0000 7f3ae9bff700 1 -- 172.21.15.31:0/960503806 wait complete. 2023-10-06T21:21:18.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 Processor -- start 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 -- start start 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae41010b0 0x7f3ae40731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 0x7f3ae41a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a56f0 con 0x7f3ae41061d0 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a5860 con 0x7f3ae4073c30 2023-10-06T21:21:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.350+0000 7f3ae9bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a5a20 con 0x7f3ae41010b0 2023-10-06T21:21:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60244/0 (socket says 172.21.15.31:60244) 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1280997461 learned_addr learned my addr 172.21.15.31:0/1280997461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae41010b0 0x7f3ae40731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 0x7f3ae41a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1280997461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae41010b0 msgr2=0x7f3ae40731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae41010b0 0x7f3ae40731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1280997461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 msgr2=0x7f3ae41a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 0x7f3ae41a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1280997461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40095f0 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 0x7f3ae41a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b500 tx=0x7f3ad800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.351+0000 7f3ae0ff9700 1 -- 172.21.15.31:0/1280997461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8011830 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.352+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae41a5d00 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.352+0000 7f3ae0ff9700 1 -- 172.21.15.31:0/1280997461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ad8011e20 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.352+0000 7f3ae0ff9700 1 -- 172.21.15.31:0/1280997461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8010db0 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.352+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41a61b0 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.355+0000 7f3ae0ff9700 1 -- 172.21.15.31:0/1280997461 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f3ad8011990 con 0x7f3ae41061d0 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.356+0000 7f3afd4ecb80 1 Processor -- start 2023-10-06T21:21:18.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.356+0000 7f3afd4ecb80 1 -- start start 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.356+0000 7f3ae0ff9700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 0x7f3acc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f060c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3efbc460 con 0x558b3f0530c0 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3efadc20 con 0x558b3f05e890 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3efab540 con 0x558b3f0539e0 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae0ff9700 1 -- 172.21.15.31:0/1280997461 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ad80ccdc0 con 0x7f3ae41061d0 2023-10-06T21:21:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 0x7f3acc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60250/0 (socket says 172.21.15.31:60250) 2023-10-06T21:21:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 -- 172.21.15.31:0/846468895 learned_addr learned my addr 172.21.15.31:0/846468895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f060c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 -- 172.21.15.31:0/846468895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 msgr2=0x558b3f05e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 -- 172.21.15.31:0/846468895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 msgr2=0x558b3f060c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.357+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f060c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ae37fe700 1 -- 172.21.15.31:0/846468895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b3edd1de0 con 0x558b3f0530c0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 0x7f3acc0a3bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0127c0 tx=0x7f3adc012b80 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4005c70 tx=0x7f3ad4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513f6fb8f2be4b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ac9ffb700 1 -- 172.21.15.31:0/846468895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401d070 con 0x558b3f0530c0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ac9ffb700 1 -- 172.21.15.31:0/846468895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ad401e040 con 0x558b3f0530c0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.358+0000 7f3ac9ffb700 1 -- 172.21.15.31:0/846468895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4004500 con 0x558b3f0530c0 2023-10-06T21:21:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/846468895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 msgr2=0x558b3f0534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4005c70 tx=0x7f3ad4005040 comp rx=0 tx=0).stop 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/846468895 shutdown_connections 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f060c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f0534a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/846468895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/846468895 >> 172.21.15.31:0/846468895 conn(0x558b3efc8260 msgr2=0x558b3f04f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/846468895 shutdown_connections 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.359+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/846468895 wait complete. 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 -- start start 2023-10-06T21:21:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 Processor -- start 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3efa6e30 con 0x558b3f0530c0 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3ef43710 con 0x558b3f05e890 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.360+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3ef433b0 con 0x558b3f0539e0 2023-10-06T21:21:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3afd4ecb80 10 client.0 _ll_register_callbacks cb 0x558b3eda0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60260/0 (socket says 172.21.15.31:60260) 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 -- 172.21.15.31:0/2864959767 learned_addr learned my addr 172.21.15.31:0/2864959767 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 -- 172.21.15.31:0/2864959767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 msgr2=0x558b3f05b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 -- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 msgr2=0x558b3f05bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 -- 172.21.15.31:0/2864959767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae410e5c0 con 0x558b3f0530c0 2023-10-06T21:21:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009990 tx=0x7f3ad400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f50a43251d2d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.361+0000 7f3ac2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401d030 con 0x558b3f0530c0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b3f046f10 con 0x558b3f0530c0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ad4004700 con 0x558b3f0530c0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ad40056f0 con 0x558b3f0530c0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3afd4ecb80 10 client.15708 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b3edbac60 con 0x558b3f0530c0 2023-10-06T21:21:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558b3f046f10 con 0x558b3f0530c0 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ad4003c90 con 0x558b3f0530c0 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3afd4ecb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.362+0000 7f3afd4ecb80 10 client.15708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3afd4ecb80 20 client.15708 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f06dd70 0x558b3f070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3afd4ecb80 4 client.15708 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b3ef9fe10 con 0x558b3f06dd70 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f06dd70 0x558b3f070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f06dd70 0x558b3f070160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=404df337f9fefd19 server_cookie=225d08b68a5a8ecd in_seq=0 out_seq=0 2023-10-06T21:21:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.363+0000 7f3ac2ffd700 10 client.15708 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:18.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/2864959767 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b3ef9fe10 con 0x558b3f06dd70 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3ac2ffd700 10 client.15708 handle_command_reply: tid=0 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3afd4ecb80 1 client.15708 shutdown 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 msgr2=0x558b3f05e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009990 tx=0x7f3ad400b9d0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 shutdown_connections 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.364+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f06dd70 0x558b3f070160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f05e290 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/2864959767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f05b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3ac2ffd700 0 client.15708 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 >> 172.21.15.31:0/2864959767 conn(0x558b3efc8260 msgr2=0x558b3f04efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 shutdown_connections 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/2864959767 wait complete. 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.365+0000 7f3afd4ecb80 20 client.15708 trim_cache size 0 max 16384 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.378+0000 7f3afd4ecb80 1 Processor -- start 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 -- start start 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f06a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3ee27920 con 0x558b3f0530c0 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3f0572f0 con 0x558b3f05e890 2023-10-06T21:21:18.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3f05c0b0 con 0x558b3f0539e0 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60262/0 (socket says 172.21.15.31:60262) 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 -- 172.21.15.31:0/3286949365 learned_addr learned my addr 172.21.15.31:0/3286949365 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 -- 172.21.15.31:0/3286949365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 msgr2=0x558b3f06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f06a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 -- 172.21.15.31:0/3286949365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 msgr2=0x558b3f05ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae37fe700 1 -- 172.21.15.31:0/3286949365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b3edf8250 con 0x558b3f0530c0 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.379+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009350 tx=0x7f3ad40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34d698671642015 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3aca7fc700 1 -- 172.21.15.31:0/3286949365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401d030 con 0x558b3f0530c0 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3aca7fc700 1 -- 172.21.15.31:0/3286949365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ad4005670 con 0x558b3f0530c0 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3aca7fc700 1 -- 172.21.15.31:0/3286949365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400faa0 con 0x558b3f0530c0 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/3286949365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 msgr2=0x558b3f04dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.380+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009350 tx=0x7f3ad40042e0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/3286949365 shutdown_connections 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f05ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0530c0 0x558b3f04dbc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/3286949365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0539e0 0x558b3f06a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/3286949365 >> 172.21.15.31:0/3286949365 conn(0x558b3efc8260 msgr2=0x558b3f074aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/3286949365 shutdown_connections 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/3286949365 wait complete. 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 -- start start 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.381+0000 7f3afd4ecb80 1 Processor -- start 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0530c0 0x558b3f063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f0c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3f0c1090 con 0x558b3f0539e0 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3f0c11d0 con 0x558b3f05e890 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3f0c1310 con 0x558b3f0530c0 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3afd4ecb80 10 client.0 _ll_register_callbacks cb 0x7f3ad80106e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60268/0 (socket says 172.21.15.31:60268) 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1959194455 learned_addr learned my addr 172.21.15.31:0/1959194455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f0c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1959194455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0530c0 msgr2=0x558b3f063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0530c0 0x558b3f063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 msgr2=0x558b3f0c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f0c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae2ffd700 1 -- 172.21.15.31:0/1959194455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b3edbac60 con 0x558b3f0539e0 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.382+0000 7f3ae3fff700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f0c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ae2ffd700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8099b40 tx=0x7f3ad80995c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69727aae0fe034b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80b2460 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ad8084040 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ad401e040 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ad8011c40 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3afd4ecb80 10 client.15714 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b3edd02b0 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ad8063020 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.383+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ad8008b90 con 0x558b3f0539e0 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 10 client.15714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 20 client.15714 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f0c2ad0 0x558b3f0c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 4 client.15714 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b3ef9fb30 con 0x558b3f0c2ad0 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f0c2ad0 0x558b3f0c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3ae37fe700 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f0c2ad0 0x558b3f0c4ef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea73b8087ca34e95 server_cookie=2fe7df30803d09f5 in_seq=0 out_seq=0 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.384+0000 7f3ac2ffd700 10 client.15714 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3ac2ffd700 1 -- 172.21.15.31:0/1959194455 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b3ef9fb30 con 0x558b3f0c2ad0 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3ac2ffd700 10 client.15714 handle_command_reply: tid=0 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 client.15714 shutdown 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 msgr2=0x558b3f063670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8099b40 tx=0x7f3ad80995c0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 shutdown_connections 2023-10-06T21:21:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x558b3f0c2ad0 0x558b3f0c4ef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b3f05e890 0x558b3f0c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b3f0530c0 0x558b3f063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3ac2ffd700 0 client.15714 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 --2- 172.21.15.31:0/1959194455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3f0539e0 0x558b3f063670 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 >> 172.21.15.31:0/1959194455 conn(0x558b3efc8260 msgr2=0x558b3f06a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 shutdown_connections 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 1 -- 172.21.15.31:0/1959194455 wait complete. 2023-10-06T21:21:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.385+0000 7f3afd4ecb80 20 client.15714 trim_cache size 0 max 16384 2023-10-06T21:21:18.400 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 msgr2=0x7f3acc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 0x7f3acc0a3bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0127c0 tx=0x7f3adc012b80 comp rx=0 tx=0).stop 2023-10-06T21:21:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 msgr2=0x7f3ae40736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b500 tx=0x7f3ad800b810 comp rx=0 tx=0).stop 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 shutdown_connections 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae4073c30 0x7f3ae41a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ae41010b0 0x7f3ae40731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3acc0a1710 0x7f3acc0a3bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 --2- 172.21.15.31:0/1280997461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae41061d0 0x7f3ae40736f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 >> 172.21.15.31:0/1280997461 conn(0x7f3ae40fc950 msgr2=0x7f3ae40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 shutdown_connections 2023-10-06T21:21:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.386+0000 7f3ae9bff700 1 -- 172.21.15.31:0/1280997461 wait complete. 2023-10-06T21:21:18.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:17 smithi142 ceph-mon[102951]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T21:21:18.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 -- 172.21.15.31:0/3930287619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1061d0 msgr2=0x7f95ac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 --2- 172.21.15.31:0/3930287619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1061d0 0x7f95ac10dd50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f95a400b0a0 tx=0x7f95a400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 -- 172.21.15.31:0/3930287619 shutdown_connections 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 --2- 172.21.15.31:0/3930287619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1061d0 0x7f95ac10dd50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 --2- 172.21.15.31:0/3930287619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac1010b0 0x7f95ac101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 --2- 172.21.15.31:0/3930287619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac101a60 0x7f95ac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 -- 172.21.15.31:0/3930287619 >> 172.21.15.31:0/3930287619 conn(0x7f95ac0fc950 msgr2=0x7f95ac0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 -- 172.21.15.31:0/3930287619 shutdown_connections 2023-10-06T21:21:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.611+0000 7f95b35ff700 1 -- 172.21.15.31:0/3930287619 wait complete. 2023-10-06T21:21:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 Processor -- start 2023-10-06T21:21:18.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 -- start start 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 0x7f95ac197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac19b960 0x7f95ac19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19c3e0 con 0x7f95ac1010b0 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19c550 con 0x7f95ac1061d0 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.612+0000 7f95b35ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19c710 con 0x7f95ac19b960 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60294/0 (socket says 172.21.15.31:60294) 2023-10-06T21:21:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 -- 172.21.15.31:0/1350342920 learned_addr learned my addr 172.21.15.31:0/1350342920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac19b960 0x7f95ac19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 0x7f95ac197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 -- 172.21.15.31:0/1350342920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac19b960 msgr2=0x7f95ac19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac19b960 0x7f95ac19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 -- 172.21.15.31:0/1350342920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 msgr2=0x7f95ac197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 0x7f95ac197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 -- 172.21.15.31:0/1350342920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4009d00 con 0x7f95ac1010b0 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 0x7f95ac197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.613+0000 7f95b139b700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e9a0 tx=0x7f95a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.614+0000 7f95a27fc700 1 -- 172.21.15.31:0/1350342920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8009790 con 0x7f95ac1010b0 2023-10-06T21:21:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.614+0000 7f95a27fc700 1 -- 172.21.15.31:0/1350342920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95a800cbd0 con 0x7f95ac1010b0 2023-10-06T21:21:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.614+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1a3c70 con 0x7f95ac1010b0 2023-10-06T21:21:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.614+0000 7f95a27fc700 1 -- 172.21.15.31:0/1350342920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8018680 con 0x7f95ac1010b0 2023-10-06T21:21:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.614+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1a41a0 con 0x7f95ac1010b0 2023-10-06T21:21:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.617+0000 7f95a27fc700 1 -- 172.21.15.31:0/1350342920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f95a800cd40 con 0x7f95ac1010b0 2023-10-06T21:21:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.618+0000 7f95c6e35b80 1 Processor -- start 2023-10-06T21:21:18.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.618+0000 7f95c6e35b80 1 -- start start 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c38d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c398350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c2f6460 con 0x56022c398890 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c2e7c20 con 0x56022c38d9e0 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c2e5540 con 0x56022c38d0c0 2023-10-06T21:21:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95a27fc700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 0x7f95980a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60300/0 (socket says 172.21.15.31:60300) 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b0b9a700 1 -- 172.21.15.31:0/1419317500 learned_addr learned my addr 172.21.15.31:0/1419317500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b139b700 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c398350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c38d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95a27fc700 1 -- 172.21.15.31:0/1350342920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f95a8014070 con 0x7f95ac1010b0 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.619+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 0x7f95980a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 -- 172.21.15.31:0/1419317500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 msgr2=0x56022c38d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c38d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 -- 172.21.15.31:0/1419317500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 msgr2=0x56022c398350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c398350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 -- 172.21.15.31:0/1419317500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56022c10bde0 con 0x56022c398890 2023-10-06T21:21:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f959c009d10 tx=0x7f959c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faf6aeaac6cf426 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95977fe700 1 -- 172.21.15.31:0/1419317500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00fe50 con 0x56022c398890 2023-10-06T21:21:18.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95977fe700 1 -- 172.21.15.31:0/1419317500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f959c009e60 con 0x56022c398890 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.620+0000 7f95977fe700 1 -- 172.21.15.31:0/1419317500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c011440 con 0x56022c398890 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1419317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 msgr2=0x56022c39ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 0x7f95980a3c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009fd0 tx=0x7f95a4006210 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f959c009d10 tx=0x7f959c00f640 comp rx=0 tx=0).stop 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1419317500 shutdown_connections 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39ac80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c38d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1419317500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c398350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1419317500 >> 172.21.15.31:0/1419317500 conn(0x56022c302260 msgr2=0x56022c389450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1419317500 shutdown_connections 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.621+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1419317500 wait complete. 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 -- start start 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 Processor -- start 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c398330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c2e0e30 con 0x56022c398890 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c27d710 con 0x56022c38d9e0 2023-10-06T21:21:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c27d3b0 con 0x56022c38d0c0 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95c6e35b80 10 client.0 _ll_register_callbacks cb 0x56022c0da800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.622+0000 7f95b1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60306/0 (socket says 172.21.15.31:60306) 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 -- 172.21.15.31:0/1182123696 learned_addr learned my addr 172.21.15.31:0/1182123696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b139b700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c398330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 -- 172.21.15.31:0/1182123696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 msgr2=0x56022c398330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c398330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 -- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 msgr2=0x56022c3955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 -- 172.21.15.31:0/1182123696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c009940 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95a400bcd0 tx=0x7f95a40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d22d9a92189bc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.623+0000 7f9594ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4027070 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56022c380f10 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f95a4003c90 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a400e040 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 10 client.15726 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56022c0f4c60 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95a4037020 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f95a4005040 con 0x56022c398890 2023-10-06T21:21:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 10 client.15726 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 20 client.15726 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3a83e0 0x56022c3aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 4 client.15726 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.624+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56022c2d9e10 con 0x56022c3a83e0 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.625+0000 7f95b139b700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3a83e0 0x56022c3aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.625+0000 7f95b139b700 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3a83e0 0x56022c3aa800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4dbb5d0fd8c96cf5 server_cookie=4f27245f68eeb9e5 in_seq=0 out_seq=0 2023-10-06T21:21:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.625+0000 7f9594ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f9594ff9700 1 -- 172.21.15.31:0/1182123696 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56022c2d9e10 con 0x56022c3a83e0 2023-10-06T21:21:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f9594ff9700 10 client.15726 handle_command_reply: tid=0 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 client.15726 shutdown 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 msgr2=0x56022c395cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95a400bcd0 tx=0x7f95a40099e0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 shutdown_connections 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c395cf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d0c0 0x56022c398330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3a83e0 0x56022c3aa800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1182123696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 >> 172.21.15.31:0/1182123696 conn(0x56022c302260 msgr2=0x56022c389040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.626+0000 7f9594ff9700 0 client.15726 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.627+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 shutdown_connections 2023-10-06T21:21:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.627+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1182123696 wait complete. 2023-10-06T21:21:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.627+0000 7f95c6e35b80 20 client.15726 trim_cache size 0 max 16384 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 Processor -- start 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 -- start start 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c398890 0x56022c390e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c161920 con 0x56022c38d0c0 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c388300 con 0x56022c38d9e0 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.640+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c391480 con 0x56022c398890 2023-10-06T21:21:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60316/0 (socket says 172.21.15.31:60316) 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 -- 172.21.15.31:0/1410535086 learned_addr learned my addr 172.21.15.31:0/1410535086 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 -- 172.21.15.31:0/1410535086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c398890 msgr2=0x56022c390e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c398890 0x56022c390e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 -- 172.21.15.31:0/1410535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 msgr2=0x56022c3908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 -- 172.21.15.31:0/1410535086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56022c132250 con 0x56022c38d0c0 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.641+0000 7f95b139b700 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e970 tx=0x7f95a806d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1330090d249707 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f9597fff700 1 -- 172.21.15.31:0/1410535086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8065b60 con 0x56022c38d0c0 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f9597fff700 1 -- 172.21.15.31:0/1410535086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95a809a9f0 con 0x56022c38d0c0 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f9597fff700 1 -- 172.21.15.31:0/1410535086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8098720 con 0x56022c38d0c0 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1410535086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 msgr2=0x56022c390060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e970 tx=0x7f95a806d260 comp rx=0 tx=0).stop 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1410535086 shutdown_connections 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c398890 0x56022c390e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c38d0c0 0x56022c390060 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e970 tx=0x7f95a806d260 comp rx=0 tx=0).stop 2023-10-06T21:21:18.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/1410535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c38d9e0 0x56022c3908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1410535086 >> 172.21.15.31:0/1410535086 conn(0x56022c302260 msgr2=0x56022c3aa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1410535086 shutdown_connections 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.642+0000 7f95c6e35b80 1 -- 172.21.15.31:0/1410535086 wait complete. 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.643+0000 7f95c6e35b80 1 -- start start 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.643+0000 7f95c6e35b80 1 Processor -- start 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d9e0 0x56022c39f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c39ba30 0x56022c39beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c39f9b0 con 0x56022c398890 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c3fc240 con 0x56022c39ba30 2023-10-06T21:21:18.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022c3fc400 con 0x56022c38d9e0 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95c6e35b80 10 client.0 _ll_register_callbacks cb 0x7f95a4007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60328/0 (socket says 172.21.15.31:60328) 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 -- 172.21.15.31:0/2795078214 learned_addr learned my addr 172.21.15.31:0/2795078214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b1b9c700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c39ba30 0x56022c39beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b139b700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d9e0 0x56022c39f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 -- 172.21.15.31:0/2795078214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d9e0 msgr2=0x56022c39f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d9e0 0x56022c39f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 -- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c39ba30 msgr2=0x56022c39beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.644+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c39ba30 0x56022c39beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f95b0b9a700 1 -- 172.21.15.31:0/2795078214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1a41a0 con 0x56022c398890 2023-10-06T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f95b0b9a700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c00ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879209836b17611e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f9594ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c021040 con 0x56022c398890 2023-10-06T21:21:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f959c005590 con 0x56022c398890 2023-10-06T21:21:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f959c009e60 con 0x56022c398890 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f959c003680 con 0x56022c398890 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f95c6e35b80 10 client.15732 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.645+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56022c10a2b0 con 0x56022c398890 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f959c009e60 con 0x56022c398890 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f959c019030 con 0x56022c398890 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 10 client.15732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 20 client.15732 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3fd980 0x56022c3ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 4 client.15732 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56022c2d9b30 con 0x56022c3fd980 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.646+0000 7f95b139b700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3fd980 0x56022c3ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95b139b700 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3fd980 0x56022c3ffda0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f64e8663617b920c server_cookie=f220c8c0443853a5 in_seq=0 out_seq=0 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f9594ff9700 10 client.15732 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f9594ff9700 1 -- 172.21.15.31:0/2795078214 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56022c2d9b30 con 0x56022c3fd980 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f9594ff9700 10 client.15732 handle_command_reply: tid=0 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 client.15732 shutdown 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 msgr2=0x56022c39b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c00ffc0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 shutdown_connections 2023-10-06T21:21:18.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56022c3fd980 0x56022c3ffda0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56022c39ba30 0x56022c39beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56022c38d9e0 0x56022c39f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f9594ff9700 0 client.15732 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 --2- 172.21.15.31:0/2795078214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56022c398890 0x56022c39b4f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 >> 172.21.15.31:0/2795078214 conn(0x56022c302260 msgr2=0x56022c388660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 shutdown_connections 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.647+0000 7f95c6e35b80 1 -- 172.21.15.31:0/2795078214 wait complete. 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95c6e35b80 20 client.15732 trim_cache size 0 max 16384 2023-10-06T21:21:18.654 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 msgr2=0x7f95980a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 0x7f95980a3c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009fd0 tx=0x7f95a4006210 comp rx=0 tx=0).stop 2023-10-06T21:21:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 msgr2=0x7f95ac196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e9a0 tx=0x7f95a800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 shutdown_connections 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac19b960 0x7f95ac19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f95980a1790 0x7f95980a3c50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95ac1010b0 0x7f95ac196fc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 --2- 172.21.15.31:0/1350342920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ac1061d0 0x7f95ac197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 >> 172.21.15.31:0/1350342920 conn(0x7f95ac0fc950 msgr2=0x7f95ac0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 shutdown_connections 2023-10-06T21:21:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.648+0000 7f95b35ff700 1 -- 172.21.15.31:0/1350342920 wait complete. 2023-10-06T21:21:18.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 -- 172.21.15.31:0/114566339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e8101a60 msgr2=0x7f80e8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 --2- 172.21.15.31:0/114566339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e8101a60 0x7f80e8105aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 -- 172.21.15.31:0/114566339 shutdown_connections 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 --2- 172.21.15.31:0/114566339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81061d0 0x7f80e810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 --2- 172.21.15.31:0/114566339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e81010b0 0x7f80e8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 --2- 172.21.15.31:0/114566339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e8101a60 0x7f80e8105aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 -- 172.21.15.31:0/114566339 >> 172.21.15.31:0/114566339 conn(0x7f80e80fc950 msgr2=0x7f80e80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 -- 172.21.15.31:0/114566339 shutdown_connections 2023-10-06T21:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.879+0000 7f80effff700 1 -- 172.21.15.31:0/114566339 wait complete. 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.880+0000 7f80effff700 1 Processor -- start 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.880+0000 7f80effff700 1 -- start start 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81010b0 0x7f80e8196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e819b8a0 0x7f80e819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819c320 con 0x7f80e81061d0 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819c490 con 0x7f80e819b8a0 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80effff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819c650 con 0x7f80e81010b0 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60346/0 (socket says 172.21.15.31:60346) 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 -- 172.21.15.31:0/3730598027 learned_addr learned my addr 172.21.15.31:0/3730598027 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80edd9b700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81010b0 0x7f80e8196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ee59c700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e819b8a0 0x7f80e819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 -- 172.21.15.31:0/3730598027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81010b0 msgr2=0x7f80e8196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81010b0 0x7f80e8196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 -- 172.21.15.31:0/3730598027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e819b8a0 msgr2=0x7f80e819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e819b8a0 0x7f80e819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.881+0000 7f80ed59a700 1 -- 172.21.15.31:0/3730598027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80095f0 con 0x7f80e81061d0 2023-10-06T21:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005910 tx=0x7f80d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80deffd700 1 -- 172.21.15.31:0/3730598027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d801d070 con 0x7f80e81061d0 2023-10-06T21:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e81a3b50 con 0x7f80e81061d0 2023-10-06T21:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80deffd700 1 -- 172.21.15.31:0/3730598027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d8022470 con 0x7f80e81061d0 2023-10-06T21:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80deffd700 1 -- 172.21.15.31:0/3730598027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8017650 con 0x7f80e81061d0 2023-10-06T21:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.882+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81a3f70 con 0x7f80e81061d0 2023-10-06T21:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.885+0000 7f80deffd700 1 -- 172.21.15.31:0/3730598027 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f80d80225e0 con 0x7f80e81061d0 2023-10-06T21:21:18.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.887+0000 7f810397bb80 1 Processor -- start 2023-10-06T21:21:18.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.887+0000 7f810397bb80 1 -- start start 2023-10-06T21:21:18.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.887+0000 7f80deffd700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 0x7f80d40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.887+0000 7f80deffd700 1 -- 172.21.15.31:0/3730598027 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f80d80ce990 con 0x7f80e81061d0 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e479e0 0x565182e52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e52890 0x565182e54c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182db0460 con 0x565182e52890 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182da1c20 con 0x565182e479e0 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f810397bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182d9f540 con 0x565182e470c0 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80ee59c700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 0x7f80d40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43754/0 (socket says 172.21.15.31:43754) 2023-10-06T21:21:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 -- 172.21.15.31:0/2233137568 learned_addr learned my addr 172.21.15.31:0/2233137568 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80ee59c700 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e52890 0x565182e54c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80ed59a700 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e479e0 0x565182e52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 -- 172.21.15.31:0/2233137568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e479e0 msgr2=0x565182e52350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e479e0 0x565182e52350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 -- 172.21.15.31:0/2233137568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e52890 msgr2=0x565182e54c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e52890 0x565182e54c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.888+0000 7f80edd9b700 1 -- 172.21.15.31:0/2233137568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565182bc5de0 con 0x565182e470c0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.889+0000 7f80ee59c700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 0x7f80d40a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80e00075c0 tx=0x7f80e0007980 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.889+0000 7f80edd9b700 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d80 tx=0x7f80e4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f40e3aa45422ed87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.889+0000 7f80d3fff700 1 -- 172.21.15.31:0/2233137568 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4004230 con 0x565182e470c0 2023-10-06T21:21:18.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.889+0000 7f80d3fff700 1 -- 172.21.15.31:0/2233137568 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e4004390 con 0x565182e470c0 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.889+0000 7f80d3fff700 1 -- 172.21.15.31:0/2233137568 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40197e0 con 0x565182e470c0 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 -- 172.21.15.31:0/2233137568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 msgr2=0x565182e474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d80 tx=0x7f80e4011720 comp rx=0 tx=0).stop 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 -- 172.21.15.31:0/2233137568 shutdown_connections 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e52890 0x565182e54c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e474a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 --2- 172.21.15.31:0/2233137568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e479e0 0x565182e52350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 -- 172.21.15.31:0/2233137568 >> 172.21.15.31:0/2233137568 conn(0x565182dbc260 msgr2=0x565182e43450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 -- 172.21.15.31:0/2233137568 shutdown_connections 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.890+0000 7f810397bb80 1 -- 172.21.15.31:0/2233137568 wait complete. 2023-10-06T21:21:18.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 -- start start 2023-10-06T21:21:18.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 Processor -- start 2023-10-06T21:21:18.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e470c0 0x565182e52230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e52890 0x565182e4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182d9ae30 con 0x565182e479e0 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182d37710 con 0x565182e470c0 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182d373b0 con 0x565182e52890 2023-10-06T21:21:18.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.891+0000 7f810397bb80 10 client.0 _ll_register_callbacks cb 0x565182b94800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60362/0 (socket says 172.21.15.31:60362) 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 -- 172.21.15.31:0/4286373953 learned_addr learned my addr 172.21.15.31:0/4286373953 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ee59c700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e52890 0x565182e4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80edd9b700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e470c0 0x565182e52230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 -- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e52890 msgr2=0x565182e4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e52890 0x565182e4fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 -- 172.21.15.31:0/4286373953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e470c0 msgr2=0x565182e52230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e470c0 0x565182e52230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 -- 172.21.15.31:0/4286373953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e40099e0 con 0x565182e479e0 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80ed59a700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f80d809b400 tx=0x7f80d809bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe564495aa0c6c62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80b5060 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f80d80228c0 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.892+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565182e3af10 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80d809be00 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 10 client.15744 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565182baec60 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d8065020 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f80d80cc190 con 0x565182e479e0 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 20 client.15744 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182e61ec0 0x565182e642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 4 client.15744 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.893+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565182d93e10 con 0x565182e61ec0 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.894+0000 7f80edd9b700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182e61ec0 0x565182e642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.894+0000 7f80edd9b700 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182e61ec0 0x565182e642b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1768d402e8cb2b88 server_cookie=e33569cf02993e08 in_seq=0 out_seq=0 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.894+0000 7f80d17fa700 10 client.15744 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f80d17fa700 1 -- 172.21.15.31:0/4286373953 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565182d93e10 con 0x565182e61ec0 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f80d17fa700 10 client.15744 handle_command_reply: tid=0 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f810397bb80 1 client.15744 shutdown 2023-10-06T21:21:18.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 msgr2=0x565182e4f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f80d809b400 tx=0x7f80d809bc40 comp rx=0 tx=0).stop 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 shutdown_connections 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.895+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182e61ec0 0x565182e642b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e52890 0x565182e4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e470c0 0x565182e52230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 --2- 172.21.15.31:0/4286373953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e4f4b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 >> 172.21.15.31:0/4286373953 conn(0x565182dbc260 msgr2=0x565182e42fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f80d17fa700 0 client.15744 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 shutdown_connections 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 1 -- 172.21.15.31:0/4286373953 wait complete. 2023-10-06T21:21:18.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.896+0000 7f810397bb80 20 client.15744 trim_cache size 0 max 16384 2023-10-06T21:21:18.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.909+0000 7f810397bb80 1 Processor -- start 2023-10-06T21:21:18.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 -- start start 2023-10-06T21:21:18.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e51d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182c1b920 con 0x565182e479e0 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182e3fa20 con 0x565182e52890 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f810397bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182e440e0 con 0x565182e470c0 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f80ed59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60366/0 (socket says 172.21.15.31:60366) 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f80ed59a700 1 -- 172.21.15.31:0/3827979631 learned_addr learned my addr 172.21.15.31:0/3827979631 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.910+0000 7f80ee59c700 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e51d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 -- 172.21.15.31:0/3827979631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 msgr2=0x565182e49f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 -- 172.21.15.31:0/3827979631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 msgr2=0x565182e51d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e51d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 -- 172.21.15.31:0/3827979631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565182e54e50 con 0x565182e479e0 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ee59c700 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e51d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80ed59a700 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f80d8098170 tx=0x7f80d8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f27824079136c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80dcff9700 1 -- 172.21.15.31:0/3827979631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80cbdf0 con 0x565182e479e0 2023-10-06T21:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80dcff9700 1 -- 172.21.15.31:0/3827979631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d8065070 con 0x565182e479e0 2023-10-06T21:21:18.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.911+0000 7f80dcff9700 1 -- 172.21.15.31:0/3827979631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d809b700 con 0x565182e479e0 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- 172.21.15.31:0/3827979631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 msgr2=0x565182e593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f80d8098170 tx=0x7f80d8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- 172.21.15.31:0/3827979631 shutdown_connections 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e51d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e470c0 0x565182e49f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 --2- 172.21.15.31:0/3827979631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e479e0 0x565182e593d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- 172.21.15.31:0/3827979631 >> 172.21.15.31:0/3827979631 conn(0x565182dbc260 msgr2=0x565182e4a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- 172.21.15.31:0/3827979631 shutdown_connections 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- 172.21.15.31:0/3827979631 wait complete. 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.912+0000 7f810397bb80 1 -- start start 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 Processor -- start 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e479e0 0x565182e4d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e4b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182e4bcb0 con 0x565182e470c0 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182e4be20 con 0x565182e52890 2023-10-06T21:21:18.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565182e4bfe0 con 0x565182e479e0 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f810397bb80 10 client.0 _ll_register_callbacks cb 0x7f80d8097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60374/0 (socket says 172.21.15.31:60374) 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f80edd9b700 1 -- 172.21.15.31:0/367141198 learned_addr learned my addr 172.21.15.31:0/367141198 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f80ed59a700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e479e0 0x565182e4d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.913+0000 7f80ee59c700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e4b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 -- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e479e0 msgr2=0x565182e4d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e479e0 0x565182e4d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 -- 172.21.15.31:0/367141198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 msgr2=0x565182e4b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:18.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e4b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 -- 172.21.15.31:0/367141198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565182baec60 con 0x565182e470c0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80ee59c700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80edd9b700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b677473c47f15558 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402b250 con 0x565182e470c0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e4004390 con 0x565182e470c0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f80e402b3b0 con 0x565182e470c0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.914+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e402b570 con 0x565182e470c0 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 10 client.15750 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565182bec250 con 0x565182e470c0 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e4004390 con 0x565182e470c0 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f80e4005040 con 0x565182e470c0 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 10 client.15750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 20 client.15750 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182ebaf50 0x565182ebd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 4 client.15750 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.915+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565182d93b30 con 0x565182ebaf50 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f80ed59a700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182ebaf50 0x565182ebd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f80ed59a700 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182ebaf50 0x565182ebd370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1269076111266f1 server_cookie=bdf713a9d87e7d30 in_seq=0 out_seq=0 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f80d17fa700 10 client.15750 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f80d17fa700 1 -- 172.21.15.31:0/367141198 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565182d93b30 con 0x565182ebaf50 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f80d17fa700 10 client.15750 handle_command_reply: tid=0 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f810397bb80 1 client.15750 shutdown 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 msgr2=0x565182e4cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e4009f20 comp rx=0 tx=0).stop 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.916+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 shutdown_connections 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x565182ebaf50 0x565182ebd370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565182e52890 0x565182e4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565182e470c0 0x565182e4cac0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80d17fa700 0 client.15750 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 --2- 172.21.15.31:0/367141198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565182e479e0 0x565182e4d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 >> 172.21.15.31:0/367141198 conn(0x565182dbc260 msgr2=0x565182e4ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 shutdown_connections 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 1 -- 172.21.15.31:0/367141198 wait complete. 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f810397bb80 20 client.15750 trim_cache size 0 max 16384 2023-10-06T21:21:18.926 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 msgr2=0x7f80d40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 0x7f80d40a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80e00075c0 tx=0x7f80e0007980 comp rx=0 tx=0).stop 2023-10-06T21:21:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 msgr2=0x7f80e8197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005910 tx=0x7f80d8005740 comp rx=0 tx=0).stop 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 shutdown_connections 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80e819b8a0 0x7f80e819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80e81010b0 0x7f80e8196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.917+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80d40a1710 0x7f80d40a3bd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.918+0000 7f80effff700 1 --2- 172.21.15.31:0/3730598027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e81061d0 0x7f80e8197440 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.918+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 >> 172.21.15.31:0/3730598027 conn(0x7f80e80fc950 msgr2=0x7f80e80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.918+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 shutdown_connections 2023-10-06T21:21:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:18.918+0000 7f80effff700 1 -- 172.21.15.31:0/3730598027 wait complete. 2023-10-06T21:21:18.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 -- 172.21.15.31:0/2314443466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c0101a60 msgr2=0x7fc2c0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/2314443466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c0101a60 0x7fc2c0105aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009990 tx=0x7fc2b4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 -- 172.21.15.31:0/2314443466 shutdown_connections 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/2314443466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 0x7fc2c010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/2314443466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c01010b0 0x7fc2c0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/2314443466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c0101a60 0x7fc2c0105aa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.144+0000 7fc2c5bff700 1 -- 172.21.15.31:0/2314443466 >> 172.21.15.31:0/2314443466 conn(0x7fc2c00fc950 msgr2=0x7fc2c00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.145+0000 7fc2c5bff700 1 -- 172.21.15.31:0/2314443466 shutdown_connections 2023-10-06T21:21:19.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.145+0000 7fc2c5bff700 1 -- 172.21.15.31:0/2314443466 wait complete. 2023-10-06T21:21:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.145+0000 7fc2c5bff700 1 Processor -- start 2023-10-06T21:21:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 -- start start 2023-10-06T21:21:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 0x7fc2c01995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c019da10 0x7fc2c019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019e490 con 0x7fc2c01010b0 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019e600 con 0x7fc2c019da10 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2c5bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019e7c0 con 0x7fc2c01061d0 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60400/0 (socket says 172.21.15.31:60400) 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/634554575 learned_addr learned my addr 172.21.15.31:0/634554575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.146+0000 7fc2bffff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c019da10 0x7fc2c019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2beffd700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 0x7fc2c01995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/634554575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 msgr2=0x7fc2c01995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 0x7fc2c01995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/634554575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c019da10 msgr2=0x7fc2c019de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c019da10 0x7fc2c019de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/634554575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40095f0 con 0x7fc2c01010b0 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c9b0 tx=0x7fc2b000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bcff9700 1 -- 172.21.15.31:0/634554575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0007b00 con 0x7fc2c01010b0 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bcff9700 1 -- 172.21.15.31:0/634554575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2b0007c60 con 0x7fc2c01010b0 2023-10-06T21:21:19.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2bcff9700 1 -- 172.21.15.31:0/634554575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0018660 con 0x7fc2c01010b0 2023-10-06T21:21:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.147+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c01a5d20 con 0x7fc2c01010b0 2023-10-06T21:21:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.148+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c01a61b0 con 0x7fc2c01010b0 2023-10-06T21:21:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.151+0000 7fc2bcff9700 1 -- 172.21.15.31:0/634554575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fc2b000f450 con 0x7fc2c01010b0 2023-10-06T21:21:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 Processor -- start 2023-10-06T21:21:19.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 -- start start 2023-10-06T21:21:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db7352e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73497460 con 0x55db7352e9e0 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73488c20 con 0x55db73539890 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.152+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73486540 con 0x55db7352e0c0 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bcff9700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 0x7fc2a80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60416/0 (socket says 172.21.15.31:60416) 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1221715649 learned_addr learned my addr 172.21.15.31:0/1221715649 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2beffd700 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bffff700 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db7352e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bffff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 0x7fc2a80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bcff9700 1 -- 172.21.15.31:0/634554575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2b00cd440 con 0x7fc2c01010b0 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1221715649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 msgr2=0x55db7352e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db7352e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1221715649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 msgr2=0x55db7353bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.153+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1221715649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db732acde0 con 0x55db7352e9e0 2023-10-06T21:21:19.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.154+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0096b90 tx=0x7fc2b000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a701ff044c561c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.154+0000 7fc2bffff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 0x7fc2a80a3bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8007b90 tx=0x7fc2b8009100 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.154+0000 7fc2a5ffb700 1 -- 172.21.15.31:0/1221715649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000fe60 con 0x55db7352e9e0 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.154+0000 7fc2a5ffb700 1 -- 172.21.15.31:0/1221715649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2b009a120 con 0x55db7352e9e0 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.154+0000 7fc2a5ffb700 1 -- 172.21.15.31:0/1221715649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00a24b0 con 0x55db7352e9e0 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1221715649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 msgr2=0x55db73539350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0096b90 tx=0x7fc2b000fb70 comp rx=0 tx=0).stop 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1221715649 shutdown_connections 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db7352e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1221715649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e9e0 0x55db73539350 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1221715649 >> 172.21.15.31:0/1221715649 conn(0x55db734a3260 msgr2=0x55db7352a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1221715649 shutdown_connections 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1221715649 wait complete. 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.155+0000 7fc2d9562b80 1 -- start start 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 Processor -- start 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db73536bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db73537130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73481e30 con 0x55db7352e0c0 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7341e710 con 0x55db73539890 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7341e3b0 con 0x55db7352e9e0 2023-10-06T21:21:19.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2d9562b80 10 client.0 _ll_register_callbacks cb 0x55db7327b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.156+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60420/0 (socket says 172.21.15.31:60420) 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db73537130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/3898396888 learned_addr learned my addr 172.21.15.31:0/3898396888 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2beffd700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db73536bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 msgr2=0x55db73536bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db73536bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/3898396888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 msgr2=0x55db73537130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db73537130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/3898396888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c01a61b0 con 0x55db7352e0c0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2b009a0b0 tx=0x7fc2b00ca9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84db89cabefac7db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.157+0000 7fc29effd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b009a380 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db73521f10 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2b009e270 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b005fea0 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 10 client.15762 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db73295c60 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2b0063020 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2b00b2d00 con 0x55db7352e0c0 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:21:19.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 20 client.15762 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db73548ee0 0x55db7354b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 4 client.15762 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.158+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db7347ae10 con 0x55db73548ee0 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.159+0000 7fc2beffd700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db73548ee0 0x55db7354b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.159+0000 7fc2beffd700 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db73548ee0 0x55db7354b2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=430b295a97648f07 server_cookie=100241304802fb98 in_seq=0 out_seq=0 2023-10-06T21:21:19.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.159+0000 7fc29effd700 10 client.15762 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.160+0000 7fc29effd700 1 -- 172.21.15.31:0/3898396888 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55db7347ae10 con 0x55db73548ee0 2023-10-06T21:21:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.160+0000 7fc29effd700 10 client.15762 handle_command_reply: tid=0 2023-10-06T21:21:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.160+0000 7fc2d9562b80 1 client.15762 shutdown 2023-10-06T21:21:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.160+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 msgr2=0x55db735366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.160+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2b009a0b0 tx=0x7fc2b00ca9b0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 shutdown_connections 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db73548ee0 0x55db7354b2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db73537130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db735366b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/3898396888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db73536bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc29effd700 0 client.15762 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 >> 172.21.15.31:0/3898396888 conn(0x55db734a3260 msgr2=0x55db73529fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 shutdown_connections 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 1 -- 172.21.15.31:0/3898396888 wait complete. 2023-10-06T21:21:19.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.161+0000 7fc2d9562b80 20 client.15762 trim_cache size 0 max 16384 2023-10-06T21:21:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.174+0000 7fc2d9562b80 1 Processor -- start 2023-10-06T21:21:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.174+0000 7fc2d9562b80 1 -- start start 2023-10-06T21:21:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db73531010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db7352e9e0 0x55db73536f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73302920 con 0x55db73539890 2023-10-06T21:21:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73545d90 con 0x55db7352e9e0 2023-10-06T21:21:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7354b930 con 0x55db7352e0c0 2023-10-06T21:21:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60428/0 (socket says 172.21.15.31:60428) 2023-10-06T21:21:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 -- 172.21.15.31:0/2865900458 learned_addr learned my addr 172.21.15.31:0/2865900458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2beffd700 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db7352e9e0 0x55db73536f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db73531010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 -- 172.21.15.31:0/2865900458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 msgr2=0x55db73531010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db73531010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 -- 172.21.15.31:0/2865900458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db7352e9e0 msgr2=0x55db73536f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db7352e9e0 0x55db73536f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.175+0000 7fc2bffff700 1 -- 172.21.15.31:0/2865900458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db73526140 con 0x55db73539890 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2bffff700 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8003a30 tx=0x7fc2b800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bdec394cc87199 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2865900458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8022050 con 0x55db73539890 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2865900458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2b8023040 con 0x55db73539890 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2865900458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004850 con 0x55db73539890 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2d9562b80 1 -- 172.21.15.31:0/2865900458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 msgr2=0x55db73535320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.176+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8003a30 tx=0x7fc2b800d000 comp rx=0 tx=0).stop 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 -- 172.21.15.31:0/2865900458 shutdown_connections 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db73539890 0x55db73535320 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e0c0 0x55db73531010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/2865900458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db7352e9e0 0x55db73536f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 -- 172.21.15.31:0/2865900458 >> 172.21.15.31:0/2865900458 conn(0x55db734a3260 msgr2=0x55db73534c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 -- 172.21.15.31:0/2865900458 shutdown_connections 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 -- 172.21.15.31:0/2865900458 wait complete. 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 -- start start 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.177+0000 7fc2d9562b80 1 Processor -- start 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db7353e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7353efb0 con 0x55db7352e0c0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7353f120 con 0x55db73539890 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7353f2e0 con 0x55db7352e9e0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2d9562b80 10 client.0 _ll_register_callbacks cb 0x7fc2b4009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60444/0 (socket says 172.21.15.31:60444) 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1983334311 learned_addr learned my addr 172.21.15.31:0/1983334311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2beffd700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db7353e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bffff700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 msgr2=0x55db7353e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db7353e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1983334311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 msgr2=0x55db7353ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/1983334311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b8011040 con 0x55db7352e0c0 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.178+0000 7fc2bffff700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0096b40 tx=0x7fc2b00998f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d3a12810a7dad4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc29effd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b005fea0 con 0x55db7352e0c0 2023-10-06T21:21:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2b009a120 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2b0064080 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b0096460 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc2d9562b80 10 client.15768 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db732d3250 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.179+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2b009a120 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2b00b80d0 con 0x55db7352e0c0 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 10 client.15768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 20 client.15768 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db7359dc90 0x55db735a0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 4 client.15768 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db7347ab30 con 0x55db7359dc90 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.180+0000 7fc2beffd700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db7359dc90 0x55db735a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.181+0000 7fc2beffd700 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db7359dc90 0x55db735a0080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbdc2e6ab9699df1 server_cookie=9bd5d840797bef89 in_seq=0 out_seq=0 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.181+0000 7fc29effd700 10 client.15768 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.181+0000 7fc29effd700 1 -- 172.21.15.31:0/1983334311 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db7347ab30 con 0x55db7359dc90 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.181+0000 7fc29effd700 10 client.15768 handle_command_reply: tid=0 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.181+0000 7fc2d9562b80 1 client.15768 shutdown 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 msgr2=0x55db73540200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0096b40 tx=0x7fc2b00998f0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 shutdown_connections 2023-10-06T21:21:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x55db7359dc90 0x55db735a0080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55db73539890 0x55db7353ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc29effd700 0 client.15768 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db7352e0c0 0x55db73540200 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 --2- 172.21.15.31:0/1983334311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db7352e9e0 0x55db7353e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 >> 172.21.15.31:0/1983334311 conn(0x55db734a3260 msgr2=0x55db735387e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 shutdown_connections 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 1 -- 172.21.15.31:0/1983334311 wait complete. 2023-10-06T21:21:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2d9562b80 20 client.15768 trim_cache size 0 max 16384 2023-10-06T21:21:19.192 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 msgr2=0x7fc2a80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 0x7fc2a80a3bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8007b90 tx=0x7fc2b8009100 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 msgr2=0x7fc2c0199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.182+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c9b0 tx=0x7fc2b000ccc0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 shutdown_connections 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2c019da10 0x7fc2c019de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2a80a1710 0x7fc2a80a3bd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2c01010b0 0x7fc2c0199070 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 --2- 172.21.15.31:0/634554575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2c01061d0 0x7fc2c01995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 >> 172.21.15.31:0/634554575 conn(0x7fc2c00fc950 msgr2=0x7fc2c00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 shutdown_connections 2023-10-06T21:21:19.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.183+0000 7fc2c5bff700 1 -- 172.21.15.31:0/634554575 wait complete. 2023-10-06T21:21:19.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:18 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3131682231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:19.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:21:19.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:18 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3131682231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:18 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3131682231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 -- 172.21.15.31:0/901876806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c1010b0 msgr2=0x7fb01c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 --2- 172.21.15.31:0/901876806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c1010b0 0x7fb01c101490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009990 tx=0x7fb00c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 -- 172.21.15.31:0/901876806 shutdown_connections 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 --2- 172.21.15.31:0/901876806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1061d0 0x7fb01c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 --2- 172.21.15.31:0/901876806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c1010b0 0x7fb01c101490 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 --2- 172.21.15.31:0/901876806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c101a60 0x7fb01c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 -- 172.21.15.31:0/901876806 >> 172.21.15.31:0/901876806 conn(0x7fb01c0fc950 msgr2=0x7fb01c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 -- 172.21.15.31:0/901876806 shutdown_connections 2023-10-06T21:21:19.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.407+0000 7fb021dff700 1 -- 172.21.15.31:0/901876806 wait complete. 2023-10-06T21:21:19.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.408+0000 7fb021dff700 1 Processor -- start 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.408+0000 7fb021dff700 1 -- start start 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1010b0 0x7fb01c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 0x7fb01c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e530 con 0x7fb01c19dab0 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e6a0 con 0x7fb01c1061d0 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb021dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e860 con 0x7fb01c1010b0 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb013fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 0x7fb01c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60470/0 (socket says 172.21.15.31:60470) 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1010b0 0x7fb01c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 -- 172.21.15.31:0/2815440216 learned_addr learned my addr 172.21.15.31:0/2815440216 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 -- 172.21.15.31:0/2815440216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1010b0 msgr2=0x7fb01c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1010b0 0x7fb01c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 -- 172.21.15.31:0/2815440216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 msgr2=0x7fb01c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 0x7fb01c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb01bfff700 1 -- 172.21.15.31:0/2815440216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0095f0 con 0x7fb01c19dab0 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.409+0000 7fb013fff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 0x7fb01c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:19.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb014007ee0 tx=0x7fb01400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb0197fa700 1 -- 172.21.15.31:0/2815440216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400dd20 con 0x7fb01c19dab0 2023-10-06T21:21:19.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c1a5dc0 con 0x7fb01c19dab0 2023-10-06T21:21:19.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb0197fa700 1 -- 172.21.15.31:0/2815440216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb01400f040 con 0x7fb01c19dab0 2023-10-06T21:21:19.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb0197fa700 1 -- 172.21.15.31:0/2815440216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014004750 con 0x7fb01c19dab0 2023-10-06T21:21:19.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.410+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c1a6250 con 0x7fb01c19dab0 2023-10-06T21:21:19.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.413+0000 7fb0197fa700 1 -- 172.21.15.31:0/2815440216 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fb014003a00 con 0x7fb01c19dab0 2023-10-06T21:21:19.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 Processor -- start 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 -- start start 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb0197fa700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 0x7fb0080a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b5816434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb0197fa700 1 -- 172.21.15.31:0/2815440216 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fb0140ccc60 con 0x7fb01c19dab0 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5815ac460 con 0x55b58164e890 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58159dc20 con 0x55b5816430c0 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb035630b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58159b540 con 0x55b5816439e0 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.415+0000 7fb013fff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 0x7fb0080a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60484/0 (socket says 172.21.15.31:60484) 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 -- 172.21.15.31:0/3505480444 learned_addr learned my addr 172.21.15.31:0/3505480444 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb013fff700 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b5816434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 -- 172.21.15.31:0/3505480444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 msgr2=0x55b58164e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 -- 172.21.15.31:0/3505480444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 msgr2=0x55b5816434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b5816434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 -- 172.21.15.31:0/3505480444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5813c1de0 con 0x55b58164e890 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb01bfff700 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb0140963b0 tx=0x7fb014099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dab645407e3c56e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb013fff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 0x7fb0080a3bd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb01c19a720 tx=0x7fb004013c90 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb011ffb700 1 -- 172.21.15.31:0/3505480444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400de80 con 0x55b58164e890 2023-10-06T21:21:19.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb011ffb700 1 -- 172.21.15.31:0/3505480444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb014009410 con 0x55b58164e890 2023-10-06T21:21:19.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.416+0000 7fb011ffb700 1 -- 172.21.15.31:0/3505480444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b9320 con 0x55b58164e890 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 -- 172.21.15.31:0/3505480444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 msgr2=0x55b581650c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb0140963b0 tx=0x7fb014099550 comp rx=0 tx=0).stop 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 -- 172.21.15.31:0/3505480444 shutdown_connections 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b581650c80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b5816434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 --2- 172.21.15.31:0/3505480444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 -- 172.21.15.31:0/3505480444 >> 172.21.15.31:0/3505480444 conn(0x55b5815b8260 msgr2=0x55b58163f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 -- 172.21.15.31:0/3505480444 shutdown_connections 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.417+0000 7fb035630b80 1 -- 172.21.15.31:0/3505480444 wait complete. 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.418+0000 7fb035630b80 1 -- start start 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.418+0000 7fb035630b80 1 Processor -- start 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.418+0000 7fb035630b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b58164b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581596e30 con 0x55b58164e890 2023-10-06T21:21:19.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581533710 con 0x55b5816430c0 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5815333b0 con 0x55b5816439e0 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb035630b80 10 client.0 _ll_register_callbacks cb 0x55b581390800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b58164b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60500/0 (socket says 172.21.15.31:60500) 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 -- 172.21.15.31:0/54437832 learned_addr learned my addr 172.21.15.31:0/54437832 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b58164b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:44396/0 (socket says 172.21.15.31:44396) 2023-10-06T21:21:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb013fff700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 -- 172.21.15.31:0/54437832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 msgr2=0x55b58164bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 -- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 msgr2=0x55b58164b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b58164b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb01bfff700 1 -- 172.21.15.31:0/54437832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c1a6250 con 0x55b58164e890 2023-10-06T21:21:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.419+0000 7fb013fff700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7fb01bfff700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb014099640 tx=0x7fb014099060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c451e2a0a6ca1fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:19.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b2220 con 0x55b58164e890 2023-10-06T21:21:19.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b581636f10 con 0x55b58164e890 2023-10-06T21:21:19.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fb0140b7e40 con 0x55b58164e890 2023-10-06T21:21:19.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb01400de80 con 0x55b58164e890 2023-10-06T21:21:19.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7fb035630b80 10 client.15780 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5813aac60 con 0x55b58164e890 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b581636f10 con 0x55b58164e890 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0140a1840 con 0x55b58164e890 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.420+0000 7fb035630b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb035630b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb035630b80 20 client.15780 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:19.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58165e000 0x55b581660420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb035630b80 4 client.15780 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:19.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b58158fe10 con 0x55b58165e000 2023-10-06T21:21:19.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.421+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58165e000 0x55b581660420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.422+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58165e000 0x55b581660420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b7bd3060ede218 server_cookie=4e67fc9ce054b368 in_seq=0 out_seq=0 2023-10-06T21:21:19.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.422+0000 7faffaffd700 10 client.15780 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:19.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7faffaffd700 1 -- 172.21.15.31:0/54437832 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b58158fe10 con 0x55b58165e000 2023-10-06T21:21:19.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7faffaffd700 10 client.15780 handle_command_reply: tid=0 2023-10-06T21:21:19.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 client.15780 shutdown 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 msgr2=0x55b58164c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb014099640 tx=0x7fb014099060 comp rx=0 tx=0).stop 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 shutdown_connections 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58165e000 0x55b581660420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.423+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164c090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816430c0 0x55b58164b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7faffaffd700 0 client.15780 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 1 --2- 172.21.15.31:0/54437832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816439e0 0x55b58164bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 >> 172.21.15.31:0/54437832 conn(0x55b5815b8260 msgr2=0x55b58163efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 shutdown_connections 2023-10-06T21:21:19.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 1 -- 172.21.15.31:0/54437832 wait complete. 2023-10-06T21:21:19.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.424+0000 7fb035630b80 20 client.15780 trim_cache size 0 max 16384 2023-10-06T21:21:19.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.437+0000 7fb035630b80 1 Processor -- start 2023-10-06T21:21:19.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.437+0000 7fb035630b80 1 -- start start 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.437+0000 7fb035630b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581646030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.437+0000 7fb035630b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 0x55b58164bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb035630b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb035630b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581417920 con 0x55b58164e890 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb035630b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581636de0 con 0x55b5816439e0 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb035630b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581647490 con 0x55b5816430c0 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60506/0 (socket says 172.21.15.31:60506) 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 -- 172.21.15.31:0/2788983904 learned_addr learned my addr 172.21.15.31:0/2788983904 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb013fff700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 0x55b58164bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581646030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 -- 172.21.15.31:0/2788983904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 msgr2=0x55b581646030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581646030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 -- 172.21.15.31:0/2788983904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 msgr2=0x55b58164bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 0x55b58164bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 -- 172.21.15.31:0/2788983904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58163b140 con 0x55b58164e890 2023-10-06T21:21:19.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb013fff700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 0x55b58164bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.438+0000 7fb01bfff700 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb0140994a0 tx=0x7fb014099180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d1ac22c3dc022 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb0127fc700 1 -- 172.21.15.31:0/2788983904 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b2220 con 0x55b58164e890 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb0127fc700 1 -- 172.21.15.31:0/2788983904 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb01400de80 con 0x55b58164e890 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb0127fc700 1 -- 172.21.15.31:0/2788983904 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01409da90 con 0x55b58164e890 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 -- 172.21.15.31:0/2788983904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 msgr2=0x55b58164a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb0140994a0 tx=0x7fb014099180 comp rx=0 tx=0).stop 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 -- 172.21.15.31:0/2788983904 shutdown_connections 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58164e890 0x55b58164a280 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581646030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 --2- 172.21.15.31:0/2788983904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5816439e0 0x55b58164bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 -- 172.21.15.31:0/2788983904 >> 172.21.15.31:0/2788983904 conn(0x55b5815b8260 msgr2=0x55b581649b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 -- 172.21.15.31:0/2788983904 shutdown_connections 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.439+0000 7fb035630b80 1 -- 172.21.15.31:0/2788983904 wait complete. 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.440+0000 7fb035630b80 1 -- start start 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.440+0000 7fb035630b80 1 Processor -- start 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581653560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58164e890 0x55b581654a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b581655070 con 0x55b5816439e0 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5816551e0 con 0x55b58164e890 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5816553a0 con 0x55b5816430c0 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb035630b80 10 client.0 _ll_register_callbacks cb 0x7fb0140997a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60522/0 (socket says 172.21.15.31:60522) 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 -- 172.21.15.31:0/1862879739 learned_addr learned my addr 172.21.15.31:0/1862879739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb01bfff700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58164e890 0x55b581654a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581653560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 -- 172.21.15.31:0/1862879739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 msgr2=0x55b581653560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581653560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 -- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58164e890 msgr2=0x55b581654a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58164e890 0x55b581654a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.441+0000 7fb013fff700 1 -- 172.21.15.31:0/1862879739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5813aac60 con 0x55b5816439e0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7fb013fff700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb004009fd0 tx=0x7fb00401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfed1625ccf4286 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7faffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004004a30 con 0x55b5816439e0 2023-10-06T21:21:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fb004004b90 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb014009410 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb004004d50 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7fb035630b80 10 client.15786 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5813e8250 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.442+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb004035080 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb00402f030 con 0x55b5816439e0 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 20 client.15786 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b5816b3160 0x55b5816b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 4 client.15786 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b58158fb30 con 0x55b5816b3160 2023-10-06T21:21:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.443+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b5816b3160 0x55b5816b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb01b7fe700 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b5816b3160 0x55b5816b5580 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70cb45b89fa519d server_cookie=4c776c9f371faf2 in_seq=0 out_seq=0 2023-10-06T21:21:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7faffaffd700 10 client.15786 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7faffaffd700 1 -- 172.21.15.31:0/1862879739 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b58158fb30 con 0x55b5816b3160 2023-10-06T21:21:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7faffaffd700 10 client.15786 handle_command_reply: tid=0 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 client.15786 shutdown 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 msgr2=0x55b581653aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb004009fd0 tx=0x7fb00401b000 comp rx=0 tx=0).stop 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 shutdown_connections 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58164e890 0x55b581654a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5816430c0 0x55b581653560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b5816b3160 0x55b5816b5580 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 --2- 172.21.15.31:0/1862879739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b5816439e0 0x55b581653aa0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7faffaffd700 0 client.15786 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.444+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 >> 172.21.15.31:0/1862879739 conn(0x55b5815b8260 msgr2=0x55b58164d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 shutdown_connections 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb035630b80 1 -- 172.21.15.31:0/1862879739 wait complete. 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb035630b80 20 client.15786 trim_cache size 0 max 16384 2023-10-06T21:21:19.454 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 msgr2=0x7fb0080a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 0x7fb0080a3bd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb01c19a720 tx=0x7fb004013c90 comp rx=0 tx=0).stop 2023-10-06T21:21:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 msgr2=0x7fb01c19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb014007ee0 tx=0x7fb01400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 shutdown_connections 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0080a1710 0x7fb0080a3bd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb01c19dab0 0x7fb01c19df30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c1010b0 0x7fb01c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 --2- 172.21.15.31:0/2815440216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb01c1061d0 0x7fb01c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.445+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 >> 172.21.15.31:0/2815440216 conn(0x7fb01c0fc950 msgr2=0x7fb01c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.446+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 shutdown_connections 2023-10-06T21:21:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:19.446+0000 7fb021dff700 1 -- 172.21.15.31:0/2815440216 wait complete. 2023-10-06T21:21:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:19 smithi031 ceph-mon[98539]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2023-10-06T21:21:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:19 smithi146 ceph-mon[102809]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2023-10-06T21:21:20.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:19 smithi142 ceph-mon[102951]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2023-10-06T21:21:20.467 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:21:20.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 -- 172.21.15.31:0/2408942422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1010b0 msgr2=0x7f53dc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 --2- 172.21.15.31:0/2408942422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1010b0 0x7f53dc101490 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009990 tx=0x7f53cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 -- 172.21.15.31:0/2408942422 shutdown_connections 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 --2- 172.21.15.31:0/2408942422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc1061d0 0x7f53dc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 --2- 172.21.15.31:0/2408942422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1010b0 0x7f53dc101490 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 --2- 172.21.15.31:0/2408942422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc101a60 0x7f53dc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 -- 172.21.15.31:0/2408942422 >> 172.21.15.31:0/2408942422 conn(0x7f53dc0fc950 msgr2=0x7f53dc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 -- 172.21.15.31:0/2408942422 shutdown_connections 2023-10-06T21:21:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.684+0000 7f53e29ff700 1 -- 172.21.15.31:0/2408942422 wait complete. 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.685+0000 7f53e29ff700 1 Processor -- start 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 -- start start 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc1010b0 0x7f53dc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc19da90 0x7f53dc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc19e510 con 0x7f53dc1061d0 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc19e680 con 0x7f53dc19da90 2023-10-06T21:21:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e29ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc19e840 con 0x7f53dc1010b0 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60540/0 (socket says 172.21.15.31:60540) 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53db7fe700 1 -- 172.21.15.31:0/3940436103 learned_addr learned my addr 172.21.15.31:0/3940436103 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc19da90 0x7f53dc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.686+0000 7f53dbfff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc1010b0 0x7f53dc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 -- 172.21.15.31:0/3940436103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc1010b0 msgr2=0x7f53dc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc1010b0 0x7f53dc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 -- 172.21.15.31:0/3940436103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc19da90 msgr2=0x7f53dc19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc19da90 0x7f53dc19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 -- 172.21.15.31:0/3940436103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0095f0 con 0x7f53dc1061d0 2023-10-06T21:21:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53db7fe700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d980 tx=0x7f53d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53d97fa700 1 -- 172.21.15.31:0/3940436103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000f890 con 0x7f53dc1061d0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53d97fa700 1 -- 172.21.15.31:0/3940436103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53d000fe80 con 0x7f53dc1061d0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53d97fa700 1 -- 172.21.15.31:0/3940436103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000e3b0 con 0x7f53dc1061d0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.687+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc1a5da0 con 0x7f53dc1061d0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.688+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc1a61d0 con 0x7f53dc1061d0 2023-10-06T21:21:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.691+0000 7f53d97fa700 1 -- 172.21.15.31:0/3940436103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f53d000f9f0 con 0x7f53dc1061d0 2023-10-06T21:21:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53f62b6b80 1 Processor -- start 2023-10-06T21:21:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53f62b6b80 1 -- start start 2023-10-06T21:21:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53d97fa700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 0x7f53c40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53d97fa700 1 -- 172.21.15.31:0/3940436103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f53d00ccea0 con 0x7f53dc1061d0 2023-10-06T21:21:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.692+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 0x7f53c40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2a60 0x559276cad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276c0b4e0 con 0x559276cad910 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276bfcca0 con 0x559276ca2140 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276bfa5c0 con 0x559276ca2a60 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:44444/0 (socket says 172.21.15.31:44444) 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 -- 172.21.15.31:0/49337776 learned_addr learned my addr 172.21.15.31:0/49337776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60552/0 (socket says 172.21.15.31:60552) 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53db7fe700 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2a60 0x559276cad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 -- 172.21.15.31:0/49337776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2a60 msgr2=0x559276cad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2a60 0x559276cad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 -- 172.21.15.31:0/49337776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 msgr2=0x559276cafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53dbfff700 1 -- 172.21.15.31:0/49337776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559276a20e60 con 0x559276ca2140 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.693+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.694+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 0x7f53c40a3bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53d400f870 tx=0x7f53d400fc30 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.694+0000 7f53dbfff700 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f53cc015000 tx=0x7f53cc005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80021481911a80c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.694+0000 7f53c27fc700 1 -- 172.21.15.31:0/49337776 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc01d070 con 0x559276ca2140 2023-10-06T21:21:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.694+0000 7f53c27fc700 1 -- 172.21.15.31:0/49337776 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53cc01e040 con 0x559276ca2140 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.694+0000 7f53c27fc700 1 -- 172.21.15.31:0/49337776 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004760 con 0x559276ca2140 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 -- 172.21.15.31:0/49337776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 msgr2=0x559276ca2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f53cc015000 tx=0x7f53cc005f00 comp rx=0 tx=0).stop 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 -- 172.21.15.31:0/49337776 shutdown_connections 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276cafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2140 0x559276ca2520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/49337776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2a60 0x559276cad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 -- 172.21.15.31:0/49337776 >> 172.21.15.31:0/49337776 conn(0x559276c172e0 msgr2=0x559276c9e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 -- 172.21.15.31:0/49337776 shutdown_connections 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.695+0000 7f53f62b6b80 1 -- 172.21.15.31:0/49337776 wait complete. 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 -- start start 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 Processor -- start 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2a60 0x559276caa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276bf5eb0 con 0x559276cad910 2023-10-06T21:21:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276b92790 con 0x559276ca2a60 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276b92430 con 0x559276ca2140 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.696+0000 7f53f62b6b80 10 client.0 _ll_register_callbacks cb 0x5592769ef880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60562/0 (socket says 172.21.15.31:60562) 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 -- 172.21.15.31:0/1234401857 learned_addr learned my addr 172.21.15.31:0/1234401857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53db7fe700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2a60 0x559276caa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53dbfff700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 -- 172.21.15.31:0/1234401857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 msgr2=0x559276cad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 -- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2a60 msgr2=0x559276caa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2a60 0x559276caa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 -- 172.21.15.31:0/1234401857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d400fdb0 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f53d40060b0 tx=0x7f53d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902d1aa0c1ac74f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.697+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400e040 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559276c95f90 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f53d4018e60 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d40093f0 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 10 client.15798 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f53dc10e5c0 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559276c95f90 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f53d4024980 con 0x559276cad910 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 20 client.15798 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276cbd2b0 0x559276cbf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 4 client.15798 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.698+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559276beee90 con 0x559276cbd2b0 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.699+0000 7f53dbfff700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276cbd2b0 0x559276cbf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.699+0000 7f53dbfff700 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276cbd2b0 0x559276cbf6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1358fff622b5349b server_cookie=ef41b456c84a6db4 in_seq=0 out_seq=0 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.699+0000 7f53bb7fe700 10 client.15798 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.700+0000 7f53bb7fe700 1 -- 172.21.15.31:0/1234401857 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559276beee90 con 0x559276cbd2b0 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.700+0000 7f53bb7fe700 10 client.15798 handle_command_reply: tid=0 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.700+0000 7f53f62b6b80 1 client.15798 shutdown 2023-10-06T21:21:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 msgr2=0x559276caac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f53d40060b0 tx=0x7f53d4004b40 comp rx=0 tx=0).stop 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 shutdown_connections 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276cbd2b0 0x559276cbf6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276cad910 0x559276caac10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53bb7fe700 0 client.15798 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/1234401857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276ca2a60 0x559276caa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 >> 172.21.15.31:0/1234401857 conn(0x559276c172e0 msgr2=0x559276c9e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 shutdown_connections 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 1 -- 172.21.15.31:0/1234401857 wait complete. 2023-10-06T21:21:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.701+0000 7f53f62b6b80 20 client.15798 trim_cache size 0 max 16384 2023-10-06T21:21:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.718+0000 7f53f62b6b80 1 Processor -- start 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 -- start start 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cb4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276ca9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276a769a0 con 0x559276ca2a60 2023-10-06T21:21:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276c9aaa0 con 0x559276cad910 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276c9f300 con 0x559276ca2140 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60564/0 (socket says 172.21.15.31:60564) 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53db7fe700 1 -- 172.21.15.31:0/2359237823 learned_addr learned my addr 172.21.15.31:0/2359237823 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.719+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276ca9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53dbfff700 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cb4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 -- 172.21.15.31:0/2359237823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 msgr2=0x559276cb4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cb4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 -- 172.21.15.31:0/2359237823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 msgr2=0x559276ca9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276ca9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 -- 172.21.15.31:0/2359237823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559276cc4300 con 0x559276ca2a60 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53db7fe700 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f53d000bfd0 tx=0x7f53d00995b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21b503135f1a92b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53c2ffd700 1 -- 172.21.15.31:0/2359237823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d009a280 con 0x559276ca2a60 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53c2ffd700 1 -- 172.21.15.31:0/2359237823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53d0099b80 con 0x559276ca2a60 2023-10-06T21:21:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.720+0000 7f53c2ffd700 1 -- 172.21.15.31:0/2359237823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00b2470 con 0x559276ca2a60 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- 172.21.15.31:0/2359237823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 msgr2=0x559276cac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f53d000bfd0 tx=0x7f53d00995b0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- 172.21.15.31:0/2359237823 shutdown_connections 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276ca9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276cb4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/2359237823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276cac5b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- 172.21.15.31:0/2359237823 >> 172.21.15.31:0/2359237823 conn(0x559276c172e0 msgr2=0x559276cabea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- 172.21.15.31:0/2359237823 shutdown_connections 2023-10-06T21:21:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- 172.21.15.31:0/2359237823 wait complete. 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 -- start start 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.721+0000 7f53f62b6b80 1 Processor -- start 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276d0b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276cb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276cb3630 con 0x559276ca2a60 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276cb3770 con 0x559276cad910 2023-10-06T21:21:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276cb3930 con 0x559276ca2140 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53f62b6b80 10 client.0 _ll_register_callbacks cb 0x7f53d4009040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60576/0 (socket says 172.21.15.31:60576) 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 -- 172.21.15.31:0/248695371 learned_addr learned my addr 172.21.15.31:0/248695371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53dbfff700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276d0b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276cb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 -- 172.21.15.31:0/248695371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 msgr2=0x559276d0b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276d0b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 -- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 msgr2=0x559276cb3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276cb3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.722+0000 7f53db7fe700 1 -- 172.21.15.31:0/248695371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d0084040 con 0x559276ca2a60 2023-10-06T21:21:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53dbfff700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276d0b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:20.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53e0f9c700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276cb3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:20.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53db7fe700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f53d0099880 tx=0x7f53d00ca360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee9e01d9d4c9139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d009a280 con 0x559276ca2a60 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53cc01e040 con 0x559276ca2a60 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f53d00a1f10 con 0x559276ca2a60 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d00b2210 con 0x559276ca2a60 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53f62b6b80 10 client.15804 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559276cb9ca0 con 0x559276ca2a60 2023-10-06T21:21:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.723+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53d007f020 con 0x559276ca2a60 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f53d00a1a80 con 0x559276ca2a60 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 20 client.15804 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276d12e80 0x559276d152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 4 client.15804 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:21:20.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559276beebb0 con 0x559276d12e80 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.724+0000 7f53dbfff700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276d12e80 0x559276d152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.725+0000 7f53dbfff700 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276d12e80 0x559276d152a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3e9948be891716 server_cookie=6c822b4df9cae48 in_seq=0 out_seq=0 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.725+0000 7f53bb7fe700 10 client.15804 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53bb7fe700 1 -- 172.21.15.31:0/248695371 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559276beebb0 con 0x559276d12e80 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53bb7fe700 10 client.15804 handle_command_reply: tid=0 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 client.15804 shutdown 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 msgr2=0x559276d0b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f53d0099880 tx=0x7f53d00ca360 comp rx=0 tx=0).stop 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 shutdown_connections 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559276cad910 0x559276cb3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559276ca2140 0x559276d0b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559276d12e80 0x559276d152a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 --2- 172.21.15.31:0/248695371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559276ca2a60 0x559276d0b9d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 >> 172.21.15.31:0/248695371 conn(0x559276c172e0 msgr2=0x559276cac730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.726+0000 7f53bb7fe700 0 client.15804 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 shutdown_connections 2023-10-06T21:21:20.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53f62b6b80 1 -- 172.21.15.31:0/248695371 wait complete. 2023-10-06T21:21:20.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53f62b6b80 20 client.15804 trim_cache size 0 max 16384 2023-10-06T21:21:20.742 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-10-06T21:21:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "f29e4ae7-7206-4447-aa9a-ffddc6179c81", 2023-10-06T21:21:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-10-06T21:21:20.744 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 msgr2=0x7f53c40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 0x7f53c40a3bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53d400f870 tx=0x7f53d400fc30 comp rx=0 tx=0).stop 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 msgr2=0x7f53dc199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d980 tx=0x7f53d000dc90 comp rx=0 tx=0).stop 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 shutdown_connections 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53dc19da90 0x7f53dc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53dc1010b0 0x7f53dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f53c40a1710 0x7f53c40a3bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 --2- 172.21.15.31:0/3940436103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53dc1061d0 0x7f53dc199630 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.727+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 >> 172.21.15.31:0/3940436103 conn(0x7f53dc0fc950 msgr2=0x7f53dc10d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.728+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 shutdown_connections 2023-10-06T21:21:20.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.728+0000 7f53e29ff700 1 -- 172.21.15.31:0/3940436103 wait complete. 2023-10-06T21:21:20.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.954+0000 7f39ecdff700 1 -- 172.21.15.31:0/420420281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 msgr2=0x7f39e8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.954+0000 7f39ecdff700 1 --2- 172.21.15.31:0/420420281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e8101490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009990 tx=0x7f39d8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 -- 172.21.15.31:0/420420281 shutdown_connections 2023-10-06T21:21:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 --2- 172.21.15.31:0/420420281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e81061d0 0x7f39e810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 --2- 172.21.15.31:0/420420281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e8101490 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 --2- 172.21.15.31:0/420420281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e8101a60 0x7f39e8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 -- 172.21.15.31:0/420420281 >> 172.21.15.31:0/420420281 conn(0x7f39e80fc950 msgr2=0x7f39e80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 -- 172.21.15.31:0/420420281 shutdown_connections 2023-10-06T21:21:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.955+0000 7f39ecdff700 1 -- 172.21.15.31:0/420420281 wait complete. 2023-10-06T21:21:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 Processor -- start 2023-10-06T21:21:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 -- start start 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e81061d0 0x7f39e80736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 0x7f39e81a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e81a5730 con 0x7f39e81010b0 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e81a58a0 con 0x7f39e81061d0 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39ecdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e81a5a60 con 0x7f39e8073c30 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60588/0 (socket says 172.21.15.31:60588) 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.956+0000 7f39e659c700 1 -- 172.21.15.31:0/796975772 learned_addr learned my addr 172.21.15.31:0/796975772 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 0x7f39e81a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e81061d0 0x7f39e80736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 -- 172.21.15.31:0/796975772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 msgr2=0x7f39e81a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 0x7f39e81a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 -- 172.21.15.31:0/796975772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e81061d0 msgr2=0x7f39e80736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e81061d0 0x7f39e80736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 -- 172.21.15.31:0/796975772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80095f0 con 0x7f39e81010b0 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 0x7f39e81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39e659c700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005740 tx=0x7f39d800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39d77fe700 1 -- 172.21.15.31:0/796975772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d801d070 con 0x7f39e81010b0 2023-10-06T21:21:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39d77fe700 1 -- 172.21.15.31:0/796975772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d8022470 con 0x7f39e81010b0 2023-10-06T21:21:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.957+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e81a5ce0 con 0x7f39e81010b0 2023-10-06T21:21:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.958+0000 7f39d77fe700 1 -- 172.21.15.31:0/796975772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8017650 con 0x7f39e81010b0 2023-10-06T21:21:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.958+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e81a6100 con 0x7f39e81010b0 2023-10-06T21:21:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.961+0000 7f39d77fe700 1 -- 172.21.15.31:0/796975772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f39d80225e0 con 0x7f39e81010b0 2023-10-06T21:21:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.962+0000 7f3a0078cb80 1 Processor -- start 2023-10-06T21:21:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.962+0000 7f39d77fe700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 0x7f39d00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.962+0000 7f39d77fe700 1 -- 172.21.15.31:0/796975772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f39d80ce9b0 con 0x7f39e81010b0 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.962+0000 7f3a0078cb80 1 -- start start 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.962+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 0x7f39d00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b62350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 0x55ab53b64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53ac0460 con 0x55ab53b570c0 2023-10-06T21:21:20.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53ab1c20 con 0x55ab53b62890 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53aaf540 con 0x55ab53b579e0 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 0x7f39d00a3bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e80749a0 tx=0x7f39dc009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60600/0 (socket says 172.21.15.31:60600) 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e659c700 1 -- 172.21.15.31:0/253430060 learned_addr learned my addr 172.21.15.31:0/253430060 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 0x55ab53b64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 -- 172.21.15.31:0/253430060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 msgr2=0x55ab53b62350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.963+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b62350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b62350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 -- 172.21.15.31:0/253430060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 msgr2=0x55ab53b64c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 0x55ab53b64c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 -- 172.21.15.31:0/253430060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab538d5de0 con 0x55ab53b570c0 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 0x55ab53b64c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b62350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39e659c700 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f39d80a34f0 tx=0x7f39d809b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae94024e33f954cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39d4ff9700 1 -- 172.21.15.31:0/253430060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d809be20 con 0x55ab53b570c0 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39d4ff9700 1 -- 172.21.15.31:0/253430060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d809b720 con 0x55ab53b570c0 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.964+0000 7f39d4ff9700 1 -- 172.21.15.31:0/253430060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d809bbe0 con 0x55ab53b570c0 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 -- 172.21.15.31:0/253430060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 msgr2=0x55ab53b574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f39d80a34f0 tx=0x7f39d809b260 comp rx=0 tx=0).stop 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 -- 172.21.15.31:0/253430060 shutdown_connections 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b62890 0x55ab53b64c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b574a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/253430060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b62350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 -- 172.21.15.31:0/253430060 >> 172.21.15.31:0/253430060 conn(0x55ab53acc260 msgr2=0x55ab53b53450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 -- 172.21.15.31:0/253430060 shutdown_connections 2023-10-06T21:21:20.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.965+0000 7f3a0078cb80 1 -- 172.21.15.31:0/253430060 wait complete. 2023-10-06T21:21:20.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 -- start start 2023-10-06T21:21:20.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 Processor -- start 2023-10-06T21:21:20.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53b631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b62890 0x55ab53b636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53aaae30 con 0x55ab53b570c0 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53a47710 con 0x55ab53b579e0 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53a473b0 con 0x55ab53b62890 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.966+0000 7f3a0078cb80 10 client.0 _ll_register_callbacks cb 0x55ab538a4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:20.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60604/0 (socket says 172.21.15.31:60604) 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 -- 172.21.15.31:0/1350758548 learned_addr learned my addr 172.21.15.31:0/1350758548 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b62890 0x55ab53b636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53b631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 -- 172.21.15.31:0/1350758548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b62890 msgr2=0x55ab53b636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b62890 0x55ab53b636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 -- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 msgr2=0x55ab53b631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53b631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 -- 172.21.15.31:0/1350758548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e81a6100 con 0x55ab53b570c0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53b631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39e659c700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39d8097c20 tx=0x7f39d809f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c15de664f27433 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.967+0000 7f39cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d809bb80 con 0x55ab53b570c0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab53b4af10 con 0x55ab53b570c0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f39d80a3590 con 0x55ab53b570c0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d80a3750 con 0x55ab53b570c0 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 10 client.15816 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:20.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ab538bec60 con 0x55ab53b570c0 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d807e020 con 0x55ab53b570c0 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39d800fcd0 con 0x55ab53b570c0 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 20 client.15816 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53b71a50 0x55ab53b73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 4 client.15816 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:20.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.968+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab53aa3e10 con 0x55ab53b71a50 2023-10-06T21:21:20.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.969+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53b71a50 0x55ab53b73e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.969+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53b71a50 0x55ab53b73e70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de603da58775393 server_cookie=6b79dcbb6d2f7730 in_seq=0 out_seq=0 2023-10-06T21:21:20.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.969+0000 7f39cdffb700 10 client.15816 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f39cdffb700 1 -- 172.21.15.31:0/1350758548 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab53aa3e10 con 0x55ab53b71a50 2023-10-06T21:21:20.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f39cdffb700 10 client.15816 handle_command_reply: tid=0 2023-10-06T21:21:20.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 client.15816 shutdown 2023-10-06T21:21:20.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 msgr2=0x55ab53b62c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39d8097c20 tx=0x7f39d809f920 comp rx=0 tx=0).stop 2023-10-06T21:21:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 shutdown_connections 2023-10-06T21:21:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b62890 0x55ab53b636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53b71a50 0x55ab53b73e70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b570c0 0x55ab53b62c70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f39cdffb700 0 client.15816 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/1350758548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53b631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.970+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 >> 172.21.15.31:0/1350758548 conn(0x55ab53acc260 msgr2=0x55ab53b52f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.971+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 shutdown_connections 2023-10-06T21:21:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.971+0000 7f3a0078cb80 1 -- 172.21.15.31:0/1350758548 wait complete. 2023-10-06T21:21:20.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.971+0000 7f3a0078cb80 20 client.15816 trim_cache size 0 max 16384 2023-10-06T21:21:20.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.988+0000 7f3a0078cb80 1 Processor -- start 2023-10-06T21:21:20.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.988+0000 7f3a0078cb80 1 -- start start 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 0x55ab53b66ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b6e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53b5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5392b920 con 0x55ab53b62890 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53b671e0 con 0x55ab53b570c0 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53b5d7c0 con 0x55ab53b579e0 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53b5ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53b5ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60608/0 (socket says 172.21.15.31:60608) 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 0x55ab53b66ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e6d9d700 1 -- 172.21.15.31:0/624403953 learned_addr learned my addr 172.21.15.31:0/624403953 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b6e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 -- 172.21.15.31:0/624403953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 msgr2=0x55ab53b6e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b6e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 -- 172.21.15.31:0/624403953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 msgr2=0x55ab53b5ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53b5ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.989+0000 7f39e659c700 1 -- 172.21.15.31:0/624403953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab53b5c5d0 con 0x55ab53b570c0 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f39e659c700 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 0x55ab53b66ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f39d809b6b0 tx=0x7f39d8099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c615c012639616e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f39d57fa700 1 -- 172.21.15.31:0/624403953 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80b9d90 con 0x55ab53b570c0 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f39d57fa700 1 -- 172.21.15.31:0/624403953 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d807e070 con 0x55ab53b570c0 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f39d57fa700 1 -- 172.21.15.31:0/624403953 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d809b980 con 0x55ab53b570c0 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f3a0078cb80 1 -- 172.21.15.31:0/624403953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 msgr2=0x55ab53b66ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.990+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 0x55ab53b66ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f39d809b6b0 tx=0x7f39d8099cd0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 -- 172.21.15.31:0/624403953 shutdown_connections 2023-10-06T21:21:20.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53b5ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b570c0 0x55ab53b66ca0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/624403953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b579e0 0x55ab53b6e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 -- 172.21.15.31:0/624403953 >> 172.21.15.31:0/624403953 conn(0x55ab53acc260 msgr2=0x55ab53b61210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 -- 172.21.15.31:0/624403953 shutdown_connections 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 -- 172.21.15.31:0/624403953 wait complete. 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 -- start start 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.991+0000 7f3a0078cb80 1 Processor -- start 2023-10-06T21:21:20.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53bb9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53bba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:20.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53bc4970 con 0x55ab53b62890 2023-10-06T21:21:20.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53bc4ab0 con 0x55ab53b579e0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53bc4c70 con 0x55ab53b570c0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f3a0078cb80 10 client.0 _ll_register_callbacks cb 0x7f39d809b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f39e659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f39e659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43962/0 (socket says 172.21.15.31:43962) 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f39e659c700 1 -- 172.21.15.31:0/3286432019 learned_addr learned my addr 172.21.15.31:0/3286432019 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53bba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.992+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53bb9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 -- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 msgr2=0x55ab53bb9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53bb9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 -- 172.21.15.31:0/3286432019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 msgr2=0x55ab53bba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53bba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 -- 172.21.15.31:0/3286432019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab538bec60 con 0x55ab53b570c0 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e6d9d700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53bba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53bb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39e659c700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005f50 tx=0x7f39d809b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67b058d336ad25e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8097780 con 0x55ab53b570c0 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f39d807f080 con 0x55ab53b570c0 2023-10-06T21:21:21.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.993+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39d809b720 con 0x55ab53b570c0 2023-10-06T21:21:21.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d8097560 con 0x55ab53b570c0 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 10 client.25522 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ab538fc250 con 0x55ab53b570c0 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d809b720 con 0x55ab53b570c0 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39d809a680 con 0x55ab53b570c0 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 20 client.25522 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53bc5ed0 0x55ab53bc82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 4 client.25522 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.998+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab53aa3b30 con 0x55ab53bc5ed0 2023-10-06T21:21:21.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.999+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53bc5ed0 0x55ab53bc82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.999+0000 7f39e5d9b700 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53bc5ed0 0x55ab53bc82c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afeee24a18dd0fb server_cookie=e24e6676091bba61 in_seq=0 out_seq=0 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:20.999+0000 7f39cdffb700 10 client.25522 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f39cdffb700 1 -- 172.21.15.31:0/3286432019 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ab53aa3b30 con 0x55ab53bc5ed0 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f39cdffb700 10 client.25522 handle_command_reply: tid=0 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 client.25522 shutdown 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 msgr2=0x55ab53bb9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005f50 tx=0x7f39d809b260 comp rx=0 tx=0).stop 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 shutdown_connections 2023-10-06T21:21:21.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ab53bc5ed0 0x55ab53bc82c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab53b62890 0x55ab53bba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ab53b570c0 0x55ab53bb9810 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f39cdffb700 0 client.25522 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 --2- 172.21.15.31:0/3286432019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab53b579e0 0x55ab53bb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 >> 172.21.15.31:0/3286432019 conn(0x55ab53acc260 msgr2=0x55ab53b5ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 shutdown_connections 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 1 -- 172.21.15.31:0/3286432019 wait complete. 2023-10-06T21:21:21.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.000+0000 7f3a0078cb80 20 client.25522 trim_cache size 0 max 16384 2023-10-06T21:21:21.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:21:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "f29e4ae7-7206-4447-aa9a-ffddc6179c81": { 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "f29e4ae7-7206-4447-aa9a-ffddc6179c81", 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:21:21.009 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 msgr2=0x7f39d00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 0x7f39d00a3bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e80749a0 tx=0x7f39dc009260 comp rx=0 tx=0).stop 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 msgr2=0x7f39e80731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005740 tx=0x7f39d800bfa0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 shutdown_connections 2023-10-06T21:21:21.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39e8073c30 0x7f39e81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f39d00a1710 0x7f39d00a3bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e81010b0 0x7f39e80731b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 --2- 172.21.15.31:0/796975772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e81061d0 0x7f39e80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 >> 172.21.15.31:0/796975772 conn(0x7f39e80fc950 msgr2=0x7f39e80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 shutdown_connections 2023-10-06T21:21:21.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:21.001+0000 7f39ecdff700 1 -- 172.21.15.31:0/796975772 wait complete. 2023-10-06T21:21:21.020 INFO:tasks.cephfs.filesystem:scrub status for tag:f29e4ae7-7206-4447-aa9a-ffddc6179c81 - {'path': '/', 'tag': 'f29e4ae7-7206-4447-aa9a-ffddc6179c81', 'options': 'recursive,force'} 2023-10-06T21:21:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:21 smithi031 ceph-mon[98539]: scrub queued for path: / 2023-10-06T21:21:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:21 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:21 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:21:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:21 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:21 smithi031 ceph-mon[98539]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:21 smithi146 ceph-mon[102809]: scrub queued for path: / 2023-10-06T21:21:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:21 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:21 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:21:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:21 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:21 smithi146 ceph-mon[102809]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:21 smithi142 ceph-mon[102951]: scrub queued for path: / 2023-10-06T21:21:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:21 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:21 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:21:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:21 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:21 smithi142 ceph-mon[102951]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:24.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:24 smithi031 ceph-mon[98539]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:24.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:24 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:21:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:24 smithi146 ceph-mon[102809]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:24 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:21:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:24 smithi142 ceph-mon[102951]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-06T21:21:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:24 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:21:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 -- 172.21.15.31:0/2714069136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87281061f0 msgr2=0x7f872810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 --2- 172.21.15.31:0/2714069136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87281061f0 0x7f872810dd70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f872000b0a0 tx=0x7f872000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 -- 172.21.15.31:0/2714069136 shutdown_connections 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 --2- 172.21.15.31:0/2714069136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87281061f0 0x7f872810dd70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 --2- 172.21.15.31:0/2714069136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281010d0 0x7f87281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 --2- 172.21.15.31:0/2714069136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8728101a80 0x7f8728105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 -- 172.21.15.31:0/2714069136 >> 172.21.15.31:0/2714069136 conn(0x7f87280fc930 msgr2=0x7f87280fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 -- 172.21.15.31:0/2714069136 shutdown_connections 2023-10-06T21:21:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.177+0000 7f872c9ff700 1 -- 172.21.15.31:0/2714069136 wait complete. 2023-10-06T21:21:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.178+0000 7f872c9ff700 1 Processor -- start 2023-10-06T21:21:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.178+0000 7f872c9ff700 1 -- start start 2023-10-06T21:21:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281010d0 0x7f87281991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 0x7f8728199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819e810 con 0x7f872819dd90 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819e980 con 0x7f87281010d0 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819eb40 con 0x7f87281061f0 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f8726d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f8726d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33768/0 (socket says 172.21.15.31:33768) 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f8726d9d700 1 -- 172.21.15.31:0/1455216962 learned_addr learned my addr 172.21.15.31:0/1455216962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.179+0000 7f872659c700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281010d0 0x7f87281991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f871ffff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 0x7f8728199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 -- 172.21.15.31:0/1455216962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 msgr2=0x7f8728199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 0x7f8728199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 -- 172.21.15.31:0/1455216962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281010d0 msgr2=0x7f87281991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281010d0 0x7f87281991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 -- 172.21.15.31:0/1455216962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720009d00 con 0x7f872819dd90 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f871ffff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 0x7f8728199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f8726d9d700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8720007990 tx=0x7f87200079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f871f7fe700 1 -- 172.21.15.31:0/1455216962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000e040 con 0x7f872819dd90 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f871f7fe700 1 -- 172.21.15.31:0/1455216962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f872001d910 con 0x7f872819dd90 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87281a6040 con 0x7f872819dd90 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.180+0000 7f871f7fe700 1 -- 172.21.15.31:0/1455216962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872001b810 con 0x7f872819dd90 2023-10-06T21:21:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.181+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87281a64c0 con 0x7f872819dd90 2023-10-06T21:21:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.185+0000 7f871f7fe700 1 -- 172.21.15.31:0/1455216962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f872001d480 con 0x7f872819dd90 2023-10-06T21:21:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.186+0000 7f8740339b80 1 Processor -- start 2023-10-06T21:21:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.186+0000 7f8740339b80 1 -- start start 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6950c00 0x562fa6950fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa695e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa68ba400 con 0x562fa695c3d0 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa68abbc0 con 0x562fa6951520 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8740339b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa68a94e0 con 0x562fa6950c00 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f871f7fe700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 0x7f87140a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f871ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6950c00 0x562fa6950fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f871ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa695e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f872659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f872659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55720/0 (socket says 172.21.15.31:55720) 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f872659c700 1 -- 172.21.15.31:0/2103302912 learned_addr learned my addr 172.21.15.31:0/2103302912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f871f7fe700 1 -- 172.21.15.31:0/1455216962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f87200cef20 con 0x7f872819dd90 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.187+0000 7f8726d9d700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 0x7f87140a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 -- 172.21.15.31:0/2103302912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6950c00 msgr2=0x562fa6950fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6950c00 0x562fa6950fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 -- 172.21.15.31:0/2103302912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 msgr2=0x562fa695e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa695e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 -- 172.21.15.31:0/2103302912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fa66cfd80 con 0x562fa6951520 2023-10-06T21:21:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f8726d9d700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 0x7f87140a3bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87200cc4a0 tx=0x7f8720009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f872659c700 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8718007cf0 tx=0x7f87180103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863ebdabe9342cf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f871cff9700 1 -- 172.21.15.31:0/2103302912 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718010d10 con 0x562fa6951520 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f871cff9700 1 -- 172.21.15.31:0/2103302912 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8718015070 con 0x562fa6951520 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.188+0000 7f871cff9700 1 -- 172.21.15.31:0/2103302912 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718019630 con 0x562fa6951520 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 -- 172.21.15.31:0/2103302912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 msgr2=0x562fa695be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8718007cf0 tx=0x7f87180103d0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 -- 172.21.15.31:0/2103302912 shutdown_connections 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa695e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6950c00 0x562fa6950fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 --2- 172.21.15.31:0/2103302912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6951520 0x562fa695be90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 -- 172.21.15.31:0/2103302912 >> 172.21.15.31:0/2103302912 conn(0x562fa68c6200 msgr2=0x562fa694d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 -- 172.21.15.31:0/2103302912 shutdown_connections 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.189+0000 7f8740339b80 1 -- 172.21.15.31:0/2103302912 wait complete. 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.190+0000 7f8740339b80 1 -- start start 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.190+0000 7f8740339b80 1 Processor -- start 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.190+0000 7f8740339b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa6959050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa695c3d0 0x562fa6959660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa68a4dd0 con 0x562fa6950c00 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa68416b0 con 0x562fa695c3d0 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa6841350 con 0x562fa6951520 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f8740339b80 10 client.0 _ll_register_callbacks cb 0x562fa669e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33784/0 (socket says 172.21.15.31:33784) 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 -- 172.21.15.31:0/1969322207 learned_addr learned my addr 172.21.15.31:0/1969322207 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f872659c700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa6959050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa695c3d0 0x562fa6959660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 -- 172.21.15.31:0/1969322207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 msgr2=0x562fa6959050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa6959050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 -- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa695c3d0 msgr2=0x562fa6959660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa695c3d0 0x562fa6959660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.191+0000 7f871ffff700 1 -- 172.21.15.31:0/1969322207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8718007920 con 0x562fa6950c00 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f871ffff700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8710014f40 tx=0x7f87100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c7a373de5b0b8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f870dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000fe30 con 0x562fa6950c00 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fa6944d50 con 0x562fa6950c00 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8710014110 con 0x562fa6950c00 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87100142d0 con 0x562fa6950c00 2023-10-06T21:21:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f8740339b80 10 client.15834 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.192+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562fa66b8c00 con 0x562fa6950c00 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871002d080 con 0x562fa6950c00 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8710027030 con 0x562fa6950c00 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 20 client.15834 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa696be70 0x562fa696e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 4 client.15834 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fa689ddb0 con 0x562fa696be70 2023-10-06T21:21:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.193+0000 7f872659c700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa696be70 0x562fa696e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.194+0000 7f872659c700 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa696be70 0x562fa696e290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50b3dc675e02bf5 server_cookie=7e97e339512fe538 in_seq=0 out_seq=0 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.194+0000 7f870dffb700 10 client.15834 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f870dffb700 1 -- 172.21.15.31:0/1969322207 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562fa689ddb0 con 0x562fa696be70 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f870dffb700 10 client.15834 handle_command_reply: tid=0 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 client.15834 shutdown 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 msgr2=0x562fa695bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8710014f40 tx=0x7f87100047c0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 shutdown_connections 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa695c3d0 0x562fa6959660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa696be70 0x562fa696e290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6950c00 0x562fa695bd60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 --2- 172.21.15.31:0/1969322207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa6959050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f870dffb700 0 client.15834 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 >> 172.21.15.31:0/1969322207 conn(0x562fa68c6200 msgr2=0x562fa694cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 shutdown_connections 2023-10-06T21:21:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.195+0000 7f8740339b80 1 -- 172.21.15.31:0/1969322207 wait complete. 2023-10-06T21:21:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.196+0000 7f8740339b80 20 client.15834 trim_cache size 0 max 16384 2023-10-06T21:21:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 Processor -- start 2023-10-06T21:21:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 -- start start 2023-10-06T21:21:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa6958810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa695b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.211+0000 7f8740339b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa67258c0 con 0x562fa695c3d0 2023-10-06T21:21:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f8740339b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa69472f0 con 0x562fa6950c00 2023-10-06T21:21:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f8740339b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa695a5c0 con 0x562fa6951520 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33788/0 (socket says 172.21.15.31:33788) 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 -- 172.21.15.31:0/2464440240 learned_addr learned my addr 172.21.15.31:0/2464440240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f872659c700 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa695b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa6958810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 -- 172.21.15.31:0/2464440240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 msgr2=0x562fa695b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa695b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 -- 172.21.15.31:0/2464440240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 msgr2=0x562fa6958810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa6958810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 -- 172.21.15.31:0/2464440240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fa66f61f0 con 0x562fa695c3d0 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.212+0000 7f871ffff700 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8710003b50 tx=0x7f8710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12760113b8e3100 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f871d7fa700 1 -- 172.21.15.31:0/2464440240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000e470 con 0x562fa695c3d0 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f871d7fa700 1 -- 172.21.15.31:0/2464440240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562fa6944d50 con 0x562fa695c3d0 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f871d7fa700 1 -- 172.21.15.31:0/2464440240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87100038e0 con 0x562fa695c3d0 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 -- 172.21.15.31:0/2464440240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 msgr2=0x562fa6954390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8710003b50 tx=0x7f8710004750 comp rx=0 tx=0).stop 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 -- 172.21.15.31:0/2464440240 shutdown_connections 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa695c3d0 0x562fa6954390 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa6958810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 --2- 172.21.15.31:0/2464440240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa6951520 0x562fa695b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 -- 172.21.15.31:0/2464440240 >> 172.21.15.31:0/2464440240 conn(0x562fa68c6200 msgr2=0x562fa69680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 -- 172.21.15.31:0/2464440240 shutdown_connections 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.213+0000 7f8740339b80 1 -- 172.21.15.31:0/2464440240 wait complete. 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 -- start start 2023-10-06T21:21:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 Processor -- start 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa695f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 0x562fa69b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa69b81a0 con 0x562fa6951520 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa69b8310 con 0x562fa6950c00 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.214+0000 7f8740339b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa69b84d0 con 0x562fa695c3d0 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f8740339b80 10 client.0 _ll_register_callbacks cb 0x7f8710014a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33790/0 (socket says 172.21.15.31:33790) 2023-10-06T21:21:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 -- 172.21.15.31:0/2090450721 learned_addr learned my addr 172.21.15.31:0/2090450721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f871ffff700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa695f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 -- 172.21.15.31:0/2090450721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 msgr2=0x562fa69b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f871ffff700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 0x562fa69b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 0x562fa69b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 -- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 msgr2=0x562fa695f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa695f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 -- 172.21.15.31:0/2090450721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fa66b8c00 con 0x562fa6951520 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f871ffff700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 0x562fa69b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f872659c700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8718010720 tx=0x7f8718010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8455d0ea2c188d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.215+0000 7f870dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718005240 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8718015070 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f87180053a0 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87180055c0 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f8740339b80 10 client.15840 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562fa66ce250 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871803f080 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8718039030 con 0x562fa6951520 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f8740339b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.216+0000 7f8740339b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.217+0000 7f8740339b80 20 client.15840 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.217+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa69c0e80 0x562fa69c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.217+0000 7f8740339b80 4 client.15840 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.217+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562fa689dad0 con 0x562fa69c0e80 2023-10-06T21:21:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.217+0000 7f871ffff700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa69c0e80 0x562fa69c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.218+0000 7f871ffff700 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa69c0e80 0x562fa69c32a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428dad483ca139ad server_cookie=aaf7805e123df1bb in_seq=0 out_seq=0 2023-10-06T21:21:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.218+0000 7f870dffb700 10 client.15840 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f870dffb700 1 -- 172.21.15.31:0/2090450721 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562fa689dad0 con 0x562fa69c0e80 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f870dffb700 10 client.15840 handle_command_reply: tid=0 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 client.15840 shutdown 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 msgr2=0x562fa695f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8718010720 tx=0x7f8718010dd0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 shutdown_connections 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562fa695c3d0 0x562fa69b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fa6950c00 0x562fa695f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562fa69c0e80 0x562fa69c32a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 --2- 172.21.15.31:0/2090450721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562fa6951520 0x562fa695f890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f870dffb700 0 client.15840 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 >> 172.21.15.31:0/2090450721 conn(0x562fa68c6200 msgr2=0x562fa6954390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 shutdown_connections 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 1 -- 172.21.15.31:0/2090450721 wait complete. 2023-10-06T21:21:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.222+0000 7f8740339b80 20 client.15840 trim_cache size 0 max 16384 2023-10-06T21:21:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:21:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:21:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:21:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 msgr2=0x7f87140a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 0x7f87140a3bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87200cc4a0 tx=0x7f8720009040 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 msgr2=0x7f872819e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8720007990 tx=0x7f87200079c0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 shutdown_connections 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f87140a1710 0x7f87140a3bd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f872819dd90 0x7f872819e210 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281010d0 0x7f87281991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 --2- 172.21.15.31:0/1455216962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87281061f0 0x7f8728199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 >> 172.21.15.31:0/1455216962 conn(0x7f87280fc930 msgr2=0x7f87280fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 shutdown_connections 2023-10-06T21:21:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:26.223+0000 7f872c9ff700 1 -- 172.21.15.31:0/1455216962 wait complete. 2023-10-06T21:21:26.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:26 smithi031 ceph-mon[98539]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2023-10-06T21:21:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:26 smithi146 ceph-mon[102809]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2023-10-06T21:21:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:26 smithi142 ceph-mon[102951]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2023-10-06T21:21:28.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:28 smithi031 ceph-mon[98539]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T21:21:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:28 smithi146 ceph-mon[102809]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T21:21:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:28 smithi142 ceph-mon[102951]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T21:21:30.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:29 smithi142 ceph-mon[102951]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 18 op/s 2023-10-06T21:21:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:29 smithi031 ceph-mon[98539]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 18 op/s 2023-10-06T21:21:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:29 smithi146 ceph-mon[102809]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 18 op/s 2023-10-06T21:21:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:31 smithi031 ceph-mon[98539]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:31 smithi146 ceph-mon[102809]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:31 smithi142 ceph-mon[102951]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:33 smithi031 ceph-mon[98539]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:33 smithi146 ceph-mon[102809]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:33 smithi142 ceph-mon[102951]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:36.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:35 smithi031 ceph-mon[98539]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:21:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:35 smithi146 ceph-mon[102809]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:21:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:35 smithi142 ceph-mon[102951]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:21:38.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:37 smithi031 ceph-mon[98539]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:37 smithi146 ceph-mon[102809]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:37 smithi142 ceph-mon[102951]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.374+0000 7f1c659ff700 1 -- 172.21.15.31:0/2111071856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60108440 msgr2=0x7f1c6010ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.374+0000 7f1c659ff700 1 --2- 172.21.15.31:0/2111071856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60108440 0x7f1c6010ffc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b0a0 tx=0x7f1c5800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 -- 172.21.15.31:0/2111071856 shutdown_connections 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 --2- 172.21.15.31:0/2111071856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60108440 0x7f1c6010ffc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 --2- 172.21.15.31:0/2111071856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60103320 0x7f1c60103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 --2- 172.21.15.31:0/2111071856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60103cd0 0x7f1c60107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 -- 172.21.15.31:0/2111071856 >> 172.21.15.31:0/2111071856 conn(0x7f1c600feb60 msgr2=0x7f1c60100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 -- 172.21.15.31:0/2111071856 shutdown_connections 2023-10-06T21:21:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.375+0000 7f1c659ff700 1 -- 172.21.15.31:0/2111071856 wait complete. 2023-10-06T21:21:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.376+0000 7f1c659ff700 1 Processor -- start 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 -- start start 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60108440 0x7f1c60199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 0x7f1c6019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6019e870 con 0x7f1c60103320 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6019e9e0 con 0x7f1c6019ddf0 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.377+0000 7f1c659ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6019eba0 con 0x7f1c60108440 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49758/0 (socket says 172.21.15.31:49758) 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 -- 172.21.15.31:0/802500825 learned_addr learned my addr 172.21.15.31:0/802500825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60108440 0x7f1c60199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 0x7f1c6019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 -- 172.21.15.31:0/802500825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60108440 msgr2=0x7f1c60199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60108440 0x7f1c60199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 -- 172.21.15.31:0/802500825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 msgr2=0x7f1c6019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 0x7f1c6019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5effd700 1 -- 172.21.15.31:0/802500825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009d00 con 0x7f1c60103320 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.378+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 0x7f1c6019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c5effd700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000e9a0 tx=0x7f1c5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c649fd700 1 -- 172.21.15.31:0/802500825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50009790 con 0x7f1c60103320 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c649fd700 1 -- 172.21.15.31:0/802500825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c5000cbd0 con 0x7f1c60103320 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c601a6100 con 0x7f1c60103320 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c649fd700 1 -- 172.21.15.31:0/802500825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50018680 con 0x7f1c60103320 2023-10-06T21:21:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.379+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c601a65e0 con 0x7f1c60103320 2023-10-06T21:21:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.385+0000 7f1c649fd700 1 -- 172.21.15.31:0/802500825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f1c5000cd40 con 0x7f1c60103320 2023-10-06T21:21:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.386+0000 7f1c79305b80 1 Processor -- start 2023-10-06T21:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.386+0000 7f1c79305b80 1 -- start start 2023-10-06T21:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d868bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d85df400 con 0x5637d8675c00 2023-10-06T21:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d85d0bc0 con 0x5637d8676520 2023-10-06T21:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c79305b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d85ce4e0 con 0x5637d8689be0 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c5f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c649fd700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 0x7f1c480a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c5f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49768/0 (socket says 172.21.15.31:49768) 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/2742910381 learned_addr learned my addr 172.21.15.31:0/2742910381 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.387+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d868bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c649fd700 1 -- 172.21.15.31:0/802500825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c50014070 con 0x7f1c60103320 2023-10-06T21:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5effd700 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/2742910381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 msgr2=0x5637d868bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d868bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/2742910381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 msgr2=0x5637d86896a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86896a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/2742910381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d83f4d80 con 0x5637d8675c00 2023-10-06T21:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 0x7f1c480a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.388+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b780 tx=0x7f1c58009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f378241addec2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.389+0000 7f1c45ffb700 1 -- 172.21.15.31:0/2742910381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800e070 con 0x5637d8675c00 2023-10-06T21:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.389+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 0x7f1c480a3bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c58028000 tx=0x7f1c58025000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.389+0000 7f1c45ffb700 1 -- 172.21.15.31:0/2742910381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c5800bd10 con 0x5637d8675c00 2023-10-06T21:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.389+0000 7f1c45ffb700 1 -- 172.21.15.31:0/2742910381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5801c860 con 0x5637d8675c00 2023-10-06T21:21:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 -- 172.21.15.31:0/2742910381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 msgr2=0x5637d8675fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b780 tx=0x7f1c58009390 comp rx=0 tx=0).stop 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 -- 172.21.15.31:0/2742910381 shutdown_connections 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d868bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8675fe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2742910381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86896a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 -- 172.21.15.31:0/2742910381 >> 172.21.15.31:0/2742910381 conn(0x5637d85eb200 msgr2=0x5637d8672080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.390+0000 7f1c79305b80 1 -- 172.21.15.31:0/2742910381 shutdown_connections 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.391+0000 7f1c79305b80 1 -- 172.21.15.31:0/2742910381 wait complete. 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.391+0000 7f1c79305b80 1 -- start start 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.392+0000 7f1c79305b80 1 Processor -- start 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.392+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8689680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.392+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.392+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d8687040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c79305b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d85c9dd0 con 0x5637d8675c00 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c79305b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d85666b0 con 0x5637d8676520 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c79305b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d8566350 con 0x5637d8689be0 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c79305b80 10 client.0 _ll_register_callbacks cb 0x5637d83c37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c5e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45490/0 (socket says 172.21.15.31:45490) 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c5e7fc700 1 -- 172.21.15.31:0/4085817907 learned_addr learned my addr 172.21.15.31:0/4085817907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d8687040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.393+0000 7f1c5effd700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8689680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 -- 172.21.15.31:0/4085817907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 msgr2=0x5637d8687040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d8687040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 -- 172.21.15.31:0/4085817907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 msgr2=0x5637d8689680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8689680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 -- 172.21.15.31:0/4085817907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60110830 con 0x5637d8676520 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5effd700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8689680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c54012700 tx=0x7f1c54012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b73d5f575fa4057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.394+0000 7f1c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017070 con 0x5637d8676520 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c5400f480 con 0x5637d8676520 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637d8669d50 con 0x5637d8676520 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5400f680 con 0x5637d8676520 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c79305b80 10 client.25589 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.395+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5637d83ddc00 con 0x5637d8676520 2023-10-06T21:21:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637d8669d50 con 0x5637d8676520 2023-10-06T21:21:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1c54026740 con 0x5637d8676520 2023-10-06T21:21:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c79305b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c79305b80 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c79305b80 20 client.25589 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.396+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d867cd60 0x5637d867f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.397+0000 7f1c79305b80 4 client.25589 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.397+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637d85c2db0 con 0x5637d867cd60 2023-10-06T21:21:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.397+0000 7f1c5effd700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d867cd60 0x5637d867f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.398+0000 7f1c5effd700 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d867cd60 0x5637d867f180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b9a73974408f42 server_cookie=7d4aa010e77cc0ad in_seq=0 out_seq=0 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.398+0000 7f1c3effd700 10 client.25589 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c3effd700 1 -- 172.21.15.31:0/4085817907 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637d85c2db0 con 0x5637d867cd60 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c3effd700 10 client.25589 handle_command_reply: tid=0 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 client.25589 shutdown 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 msgr2=0x5637d8686900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c54012700 tx=0x7f1c54012a10 comp rx=0 tx=0).stop 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 shutdown_connections 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d8687040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d867cd60 0x5637d867f180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8689680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 --2- 172.21.15.31:0/4085817907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d8686900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c3effd700 0 client.25589 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 >> 172.21.15.31:0/4085817907 conn(0x5637d85eb200 msgr2=0x5637d8671c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 shutdown_connections 2023-10-06T21:21:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.399+0000 7f1c79305b80 1 -- 172.21.15.31:0/4085817907 wait complete. 2023-10-06T21:21:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.400+0000 7f1c79305b80 20 client.25589 trim_cache size 0 max 16384 2023-10-06T21:21:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.415+0000 7f1c79305b80 1 Processor -- start 2023-10-06T21:21:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 -- start start 2023-10-06T21:21:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d86782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d844a8c0 con 0x5637d8675c00 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d8672dc0 con 0x5637d8676520 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.416+0000 7f1c79305b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d8682770 con 0x5637d8689be0 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d86782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:58012/0 (socket says 172.21.15.31:58012) 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/3682701403 learned_addr learned my addr 172.21.15.31:0/3682701403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/3682701403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 msgr2=0x5637d86787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/3682701403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 msgr2=0x5637d86782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d86782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.417+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/3682701403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d841b1f0 con 0x5637d8689be0 2023-10-06T21:21:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.418+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b070 tx=0x7f1c58009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd41db3fe8600c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.418+0000 7f1c467fc700 1 -- 172.21.15.31:0/3682701403 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800e040 con 0x5637d8689be0 2023-10-06T21:21:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.418+0000 7f1c467fc700 1 -- 172.21.15.31:0/3682701403 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c5800be80 con 0x5637d8689be0 2023-10-06T21:21:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.418+0000 7f1c467fc700 1 -- 172.21.15.31:0/3682701403 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5801cab0 con 0x5637d8689be0 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- 172.21.15.31:0/3682701403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 msgr2=0x5637d86860c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b070 tx=0x7f1c58009bf0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- 172.21.15.31:0/3682701403 shutdown_connections 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8689be0 0x5637d86860c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d86782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 --2- 172.21.15.31:0/3682701403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8676520 0x5637d86787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- 172.21.15.31:0/3682701403 >> 172.21.15.31:0/3682701403 conn(0x5637d85eb200 msgr2=0x5637d8677b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- 172.21.15.31:0/3682701403 shutdown_connections 2023-10-06T21:21:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- 172.21.15.31:0/3682701403 wait complete. 2023-10-06T21:21:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.419+0000 7f1c79305b80 1 -- start start 2023-10-06T21:21:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.420+0000 7f1c79305b80 1 Processor -- start 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.420+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.420+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8676520 0x5637d8682fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c79305b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 0x5637d8683510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c79305b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d8683c10 con 0x5637d8675c00 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c79305b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d8683d80 con 0x5637d8689be0 2023-10-06T21:21:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c79305b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d86f5870 con 0x5637d8676520 2023-10-06T21:21:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c79305b80 10 client.0 _ll_register_callbacks cb 0x7f1c54009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49780/0 (socket says 172.21.15.31:49780) 2023-10-06T21:21:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5effd700 1 -- 172.21.15.31:0/2398896715 learned_addr learned my addr 172.21.15.31:0/2398896715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8676520 0x5637d8682fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 0x5637d8683510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.421+0000 7f1c5effd700 1 -- 172.21.15.31:0/2398896715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8676520 msgr2=0x5637d8682fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5effd700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8676520 0x5637d8682fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5effd700 1 -- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 msgr2=0x5637d8683510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5effd700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 0x5637d8683510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5effd700 1 -- 172.21.15.31:0/2398896715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5802c070 con 0x5637d8675c00 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5f7fe700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 0x5637d8683510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c5effd700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1c50099f80 tx=0x7f1c5000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6acde3e83bf234 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5009a980 con 0x5637d8675c00 2023-10-06T21:21:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c5009a280 con 0x5637d8675c00 2023-10-06T21:21:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.422+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637d85c2ad0 con 0x5637d8675c00 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5009e3d0 con 0x5637d8675c00 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 10 client.15858 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5637d83f3250 con 0x5637d8675c00 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c5007a030 con 0x5637d8675c00 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1c500b8020 con 0x5637d8675c00 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 20 client.15858 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d86f6f30 0x5637d86f9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 4 client.15858 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.423+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637d8415f10 con 0x5637d86f6f30 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.424+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d86f6f30 0x5637d86f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.425+0000 7f1c5e7fc700 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d86f6f30 0x5637d86f9350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dde724376e9d92 server_cookie=8ccf099ea5c73f4b in_seq=0 out_seq=0 2023-10-06T21:21:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.425+0000 7f1c3effd700 10 client.15858 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c3effd700 1 -- 172.21.15.31:0/2398896715 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637d8415f10 con 0x5637d86f6f30 2023-10-06T21:21:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c3effd700 10 client.15858 handle_command_reply: tid=0 2023-10-06T21:21:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 client.15858 shutdown 2023-10-06T21:21:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 msgr2=0x5637d8690730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1c50099f80 tx=0x7f1c5000ef70 comp rx=0 tx=0).stop 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 shutdown_connections 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637d8689be0 0x5637d8683510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5637d86f6f30 0x5637d86f9350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637d8675c00 0x5637d8690730 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 --2- 172.21.15.31:0/2398896715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5637d8676520 0x5637d8682fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c3effd700 0 client.15858 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.429+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 >> 172.21.15.31:0/2398896715 conn(0x5637d85eb200 msgr2=0x5637d8678380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.430+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 shutdown_connections 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.430+0000 7f1c79305b80 1 -- 172.21.15.31:0/2398896715 wait complete. 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.430+0000 7f1c79305b80 20 client.15858 trim_cache size 0 max 16384 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:21:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:21:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 msgr2=0x7f1c480a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 0x7f1c480a3bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c58028000 tx=0x7f1c58025000 comp rx=0 tx=0).stop 2023-10-06T21:21:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 msgr2=0x7f1c60199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000e9a0 tx=0x7f1c5000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 shutdown_connections 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c6019ddf0 0x7f1c6019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c480a1710 0x7f1c480a3bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c60103320 0x7f1c60199240 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 --2- 172.21.15.31:0/802500825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c60108440 0x7f1c60199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 >> 172.21.15.31:0/802500825 conn(0x7f1c600feb60 msgr2=0x7f1c600ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 shutdown_connections 2023-10-06T21:21:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:39.431+0000 7f1c659ff700 1 -- 172.21.15.31:0/802500825 wait complete. 2023-10-06T21:21:40.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:39 smithi142 ceph-mon[102951]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:39 smithi031 ceph-mon[98539]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:39 smithi146 ceph-mon[102809]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:42.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:41 smithi031 ceph-mon[98539]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:21:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:41 smithi146 ceph-mon[102809]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:21:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:41 smithi142 ceph-mon[102951]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T21:21:44.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:43 smithi031 ceph-mon[98539]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-06T21:21:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:43 smithi146 ceph-mon[102809]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-06T21:21:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:43 smithi142 ceph-mon[102951]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-06T21:21:46.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:45 smithi031 ceph-mon[98539]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:45 smithi146 ceph-mon[102809]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:45 smithi142 ceph-mon[102951]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:48.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:47 smithi031 ceph-mon[98539]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:47 smithi146 ceph-mon[102809]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:47 smithi142 ceph-mon[102951]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:49 smithi142 ceph-mon[102951]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:49 smithi031 ceph-mon[98539]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:49 smithi146 ceph-mon[102809]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T21:21:51.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.367+0000 7f1c21fff700 1 -- 172.21.15.31:0/2135298915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c101a60 msgr2=0x7f1c1c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.367+0000 7f1c21fff700 1 --2- 172.21.15.31:0/2135298915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c101a60 0x7f1c1c105aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009990 tx=0x7f1c10009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 -- 172.21.15.31:0/2135298915 shutdown_connections 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 --2- 172.21.15.31:0/2135298915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 --2- 172.21.15.31:0/2135298915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c1010b0 0x7f1c1c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 --2- 172.21.15.31:0/2135298915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c101a60 0x7f1c1c105aa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 -- 172.21.15.31:0/2135298915 >> 172.21.15.31:0/2135298915 conn(0x7f1c1c0fc950 msgr2=0x7f1c1c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 -- 172.21.15.31:0/2135298915 shutdown_connections 2023-10-06T21:21:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.368+0000 7f1c21fff700 1 -- 172.21.15.31:0/2135298915 wait complete. 2023-10-06T21:21:51.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.369+0000 7f1c21fff700 1 Processor -- start 2023-10-06T21:21:51.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.369+0000 7f1c21fff700 1 -- start start 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1010b0 0x7f1c1c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c10ae00 0x7f1c1c10b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c10e0b0 con 0x7f1c1c1061d0 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c1a9dd0 con 0x7f1c1c1010b0 2023-10-06T21:21:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c21fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c1a9f40 con 0x7f1c1c10ae00 2023-10-06T21:21:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.370+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44526/0 (socket says 172.21.15.31:44526) 2023-10-06T21:21:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 -- 172.21.15.31:0/681181891 learned_addr learned my addr 172.21.15.31:0/681181891 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c10ae00 0x7f1c1c10b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1010b0 0x7f1c1c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 -- 172.21.15.31:0/681181891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c10ae00 msgr2=0x7f1c1c10b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c10ae00 0x7f1c1c10b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 -- 172.21.15.31:0/681181891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1010b0 msgr2=0x7f1c1c10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1010b0 0x7f1c1c10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 -- 172.21.15.31:0/681181891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100095f0 con 0x7f1c1c1061d0 2023-10-06T21:21:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.371+0000 7f1c1affd700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005910 tx=0x7f1c10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.372+0000 7f1c18ff9700 1 -- 172.21.15.31:0/681181891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1001d070 con 0x7f1c1c1061d0 2023-10-06T21:21:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.372+0000 7f1c18ff9700 1 -- 172.21.15.31:0/681181891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c10022470 con 0x7f1c1c1061d0 2023-10-06T21:21:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.372+0000 7f1c18ff9700 1 -- 172.21.15.31:0/681181891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10017650 con 0x7f1c1c1061d0 2023-10-06T21:21:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.372+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c1aa1c0 con 0x7f1c1c1061d0 2023-10-06T21:21:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.372+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c1aa590 con 0x7f1c1c1061d0 2023-10-06T21:21:51.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.377+0000 7f1c18ff9700 1 -- 172.21.15.31:0/681181891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f1c100225e0 con 0x7f1c1c1061d0 2023-10-06T21:21:51.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.379+0000 7f1c35a05b80 1 Processor -- start 2023-10-06T21:21:51.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.379+0000 7f1c35a05b80 1 -- start start 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.379+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x556507809c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556507765460 con 0x5565077fc9e0 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556507756c20 con 0x5565077fc0c0 2023-10-06T21:21:51.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556507754540 con 0x556507807890 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x556507809c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44530/0 (socket says 172.21.15.31:44530) 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56554/0 (socket says 172.21.15.31:56554) 2023-10-06T21:21:51.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.380+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/1248342584 learned_addr learned my addr 172.21.15.31:0/1248342584 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1248342584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 msgr2=0x556507809c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x556507809c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1248342584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 msgr2=0x556507807350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1248342584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650757ade0 con 0x5565077fc0c0 2023-10-06T21:21:51.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003e30 tx=0x7f1c14009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da8b71d2879c993 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.381+0000 7f1c1affd700 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x556507809c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.382+0000 7f1c01ffb700 1 -- 172.21.15.31:0/1248342584 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14019070 con 0x5565077fc0c0 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.382+0000 7f1c01ffb700 1 -- 172.21.15.31:0/1248342584 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c1400f040 con 0x5565077fc0c0 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.382+0000 7f1c01ffb700 1 -- 172.21.15.31:0/1248342584 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c140078c0 con 0x5565077fc0c0 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.382+0000 7f1c18ff9700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 0x7f1c040a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.382+0000 7f1c18ff9700 1 -- 172.21.15.31:0/681181891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c100ce960 con 0x7f1c1c1061d0 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1248342584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 msgr2=0x5565077fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003e30 tx=0x7f1c14009c20 comp rx=0 tx=0).stop 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 0x7f1c040a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1248342584 shutdown_connections 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x556507809c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x5565077fc4a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1248342584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507807350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.383+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1248342584 >> 172.21.15.31:0/1248342584 conn(0x556507771260 msgr2=0x5565077f8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.384+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1248342584 shutdown_connections 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.384+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1248342584 wait complete. 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.384+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 0x7f1c040a3c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012560 tx=0x7f1c0c009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.384+0000 7f1c35a05b80 1 -- start start 2023-10-06T21:21:51.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 Processor -- start 2023-10-06T21:21:51.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507804510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x556507804b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650774fe30 con 0x5565077fc9e0 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.385+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565076ec710 con 0x556507807890 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565076ec3b0 con 0x5565077fc0c0 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c35a05b80 10 client.0 _ll_register_callbacks cb 0x556507549800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52308/0 (socket says 172.21.15.31:52308) 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/2370527918 learned_addr learned my addr 172.21.15.31:0/2370527918 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.386+0000 7f1c1affd700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507804510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 msgr2=0x556507804b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x556507804b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/2370527918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 msgr2=0x556507804510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507804510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/2370527918 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c009940 con 0x5565077fc0c0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0045e0 tx=0x7f1c0c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d4053f7bf924879 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.387+0000 7f1bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017070 con 0x5565077fc0c0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c0c012e70 con 0x5565077fc0c0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565077eff10 con 0x5565077fc0c0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c0c003ea0 con 0x5565077fc0c0 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1c35a05b80 10 client.25564 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:51.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.388+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1c14003a60 con 0x5565077fc0c0 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.389+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5565077eff10 con 0x5565077fc0c0 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1c0c030620 con 0x5565077fc0c0 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 10 client.25564 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 20 client.25564 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556507817270 0x556507819690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 4 client.25564 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:51.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.390+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556507748e10 con 0x556507817270 2023-10-06T21:21:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.391+0000 7f1c1affd700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556507817270 0x556507819690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.391+0000 7f1c1affd700 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556507817270 0x556507819690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6afa898ea0d164ab server_cookie=6c0c608010f71c13 in_seq=0 out_seq=0 2023-10-06T21:21:51.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.391+0000 7f1bfaffd700 10 client.25564 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:51.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.392+0000 7f1bfaffd700 1 -- 172.21.15.31:0/2370527918 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556507748e10 con 0x556507817270 2023-10-06T21:21:51.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.392+0000 7f1bfaffd700 10 client.25564 handle_command_reply: tid=0 2023-10-06T21:21:51.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.392+0000 7f1c35a05b80 1 client.25564 shutdown 2023-10-06T21:21:51.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.392+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 msgr2=0x556507807290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.392+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0045e0 tx=0x7f1c0c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 shutdown_connections 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x556507804b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc0c0 0x556507807290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556507817270 0x556507819690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/2370527918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x556507804510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 >> 172.21.15.31:0/2370527918 conn(0x556507771260 msgr2=0x5565077f7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1bfaffd700 0 client.25564 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 shutdown_connections 2023-10-06T21:21:51.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 1 -- 172.21.15.31:0/2370527918 wait complete. 2023-10-06T21:21:51.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.393+0000 7f1c35a05b80 20 client.25564 trim_cache size 0 max 16384 2023-10-06T21:21:51.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.427+0000 7f1c35a05b80 1 Processor -- start 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 -- start start 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc9e0 0x5565077fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x55650780e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565075d0920 con 0x5565077fc0c0 2023-10-06T21:21:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556507817090 con 0x556507807890 2023-10-06T21:21:51.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.428+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556507805b10 con 0x5565077fc9e0 2023-10-06T21:21:51.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44548/0 (socket says 172.21.15.31:44548) 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/192785306 learned_addr learned my addr 172.21.15.31:0/192785306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x55650780e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1affd700 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc9e0 0x5565077fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/192785306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc9e0 msgr2=0x5565077fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc9e0 0x5565077fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/192785306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 msgr2=0x55650780e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x55650780e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.429+0000 7f1c1b7fe700 1 -- 172.21.15.31:0/192785306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650781e2e0 con 0x5565077fc0c0 2023-10-06T21:21:51.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.430+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012d30 tx=0x7f1c0c012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c94fce8207c8417 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.430+0000 7f1c027fc700 1 -- 172.21.15.31:0/192785306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c030620 con 0x5565077fc0c0 2023-10-06T21:21:51.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.430+0000 7f1c027fc700 1 -- 172.21.15.31:0/192785306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c0c030780 con 0x5565077fc0c0 2023-10-06T21:21:51.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.430+0000 7f1c027fc700 1 -- 172.21.15.31:0/192785306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c042430 con 0x5565077fc0c0 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 -- 172.21.15.31:0/192785306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 msgr2=0x5565077f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012d30 tx=0x7f1c0c012e70 comp rx=0 tx=0).stop 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 -- 172.21.15.31:0/192785306 shutdown_connections 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556507807890 0x55650780e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc0c0 0x5565077f6e20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/192785306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565077fc9e0 0x5565077fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 -- 172.21.15.31:0/192785306 >> 172.21.15.31:0/192785306 conn(0x556507771260 msgr2=0x5565077ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.431+0000 7f1c35a05b80 1 -- 172.21.15.31:0/192785306 shutdown_connections 2023-10-06T21:21:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.432+0000 7f1c35a05b80 1 -- 172.21.15.31:0/192785306 wait complete. 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.432+0000 7f1c35a05b80 1 -- start start 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.432+0000 7f1c35a05b80 1 Processor -- start 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x55650780c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x55650780ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650786a2a0 con 0x5565077fc9e0 2023-10-06T21:21:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650786a410 con 0x5565077fc0c0 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650786a5d0 con 0x556507807890 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c35a05b80 10 client.0 _ll_register_callbacks cb 0x7f1c0c0043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.433+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44562/0 (socket says 172.21.15.31:44562) 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1affd700 1 -- 172.21.15.31:0/1500113650 learned_addr learned my addr 172.21.15.31:0/1500113650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x55650780ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x55650780c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 msgr2=0x55650780c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x55650780c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1500113650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 msgr2=0x55650780c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 -- 172.21.15.31:0/1500113650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c14003a60 con 0x556507807890 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1affd700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.434+0000 7f1c1bfff700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x55650780ce60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009c60 tx=0x7f1c140077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579c86916c1cbb06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14019070 con 0x556507807890 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c1400f040 con 0x556507807890 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f1c14007a00 con 0x556507807890 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c14004ea0 con 0x556507807890 2023-10-06T21:21:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1c35a05b80 10 client.25570 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:51.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.435+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556507563c60 con 0x556507807890 2023-10-06T21:21:51.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.436+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c1400f040 con 0x556507807890 2023-10-06T21:21:51.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1c14015cb0 con 0x556507807890 2023-10-06T21:21:51.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 10 client.25570 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:51.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:51.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 20 client.25570 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650786bd80 0x55650786e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 4 client.25570 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556507748b30 con 0x55650786bd80 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.437+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650786bd80 0x55650786e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.438+0000 7f1c1b7fe700 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650786bd80 0x55650786e1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa48dfe083dc83e server_cookie=e643509381b48377 in_seq=0 out_seq=0 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.438+0000 7f1bfaffd700 10 client.25570 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.438+0000 7f1bfaffd700 1 -- 172.21.15.31:0/1500113650 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556507748b30 con 0x55650786bd80 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.438+0000 7f1bfaffd700 10 client.25570 handle_command_reply: tid=0 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.438+0000 7f1c35a05b80 1 client.25570 shutdown 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 msgr2=0x55650780ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x55650780ce60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009c60 tx=0x7f1c140077e0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 shutdown_connections 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556507807890 0x55650780ce60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565077fc0c0 0x55650780c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650786bd80 0x55650786e1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 --2- 172.21.15.31:0/1500113650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565077fc9e0 0x55650780c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 >> 172.21.15.31:0/1500113650 conn(0x556507771260 msgr2=0x5565077f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1bfaffd700 0 client.25570 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 shutdown_connections 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 1 -- 172.21.15.31:0/1500113650 wait complete. 2023-10-06T21:21:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.439+0000 7f1c35a05b80 20 client.25570 trim_cache size 0 max 16384 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 msgr2=0x7f1c040a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 0x7f1c040a3c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012560 tx=0x7f1c0c009260 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 msgr2=0x7f1c1c10a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005910 tx=0x7f1c10005740 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 shutdown_connections 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c1c10ae00 0x7f1c1c10b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c1c1010b0 0x7f1c1c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1c040a1790 0x7f1c040a3c50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 --2- 172.21.15.31:0/681181891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c1c1061d0 0x7f1c1c10a8c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.440+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 >> 172.21.15.31:0/681181891 conn(0x7f1c1c0fc950 msgr2=0x7f1c1c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.441+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 shutdown_connections 2023-10-06T21:21:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.441+0000 7f1c21fff700 1 -- 172.21.15.31:0/681181891 wait complete. 2023-10-06T21:21:51.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:21:51.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:21:51.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1307354638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1061d0 msgr2=0x7fe3bc10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1307354638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc10dd50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400b0a0 tx=0x7fe3b400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1307354638 shutdown_connections 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1307354638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc10dd50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1307354638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1010b0 0x7fe3bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1307354638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc101a60 0x7fe3bc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.817+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1307354638 >> 172.21.15.31:0/1307354638 conn(0x7fe3bc0fc8d0 msgr2=0x7fe3bc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.818+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1307354638 shutdown_connections 2023-10-06T21:21:51.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.818+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1307354638 wait complete. 2023-10-06T21:21:51.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.819+0000 7fe3c15ff700 1 Processor -- start 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.819+0000 7fe3c15ff700 1 -- start start 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1010b0 0x7fe3bc107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 0x7fe3bc1a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a9cb0 con 0x7fe3bc1010b0 2023-10-06T21:21:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a9df0 con 0x7fe3bc108850 2023-10-06T21:21:51.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.820+0000 7fe3c15ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a9f60 con 0x7fe3bc1061d0 2023-10-06T21:21:51.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 0x7fe3bc1a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 0x7fe3bc1a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56612/0 (socket says 172.21.15.31:56612) 2023-10-06T21:21:51.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3bb7fe700 1 -- 172.21.15.31:0/1588519583 learned_addr learned my addr 172.21.15.31:0/1588519583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3baffd700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1010b0 0x7fe3bc107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 -- 172.21.15.31:0/1588519583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 msgr2=0x7fe3bc1a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 0x7fe3bc1a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 -- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1010b0 msgr2=0x7fe3bc107dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1010b0 0x7fe3bc107dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:51.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.821+0000 7fe3ba7fc700 1 -- 172.21.15.31:0/1588519583 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009d00 con 0x7fe3bc1061d0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3ba7fc700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc108310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e9c0 tx=0x7fe3ac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00cbd0 con 0x7fe3bc1061d0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3ac00eeb0 con 0x7fe3bc1061d0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac018680 con 0x7fe3bc1061d0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc1aa240 con 0x7fe3bc1061d0 2023-10-06T21:21:51.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.822+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1aa700 con 0x7fe3bc1061d0 2023-10-06T21:21:51.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.824+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc067cd0 con 0x7fe3bc1061d0 2023-10-06T21:21:51.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.828+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fe3ac010430 con 0x7fe3bc1061d0 2023-10-06T21:21:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.830+0000 7fe3b3fff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 0x7fe3a80a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.831+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fe3ac014070 con 0x7fe3bc1061d0 2023-10-06T21:21:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.831+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe3ac09a1d0 con 0x7fe3bc1061d0 2023-10-06T21:21:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.831+0000 7fe3baffd700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 0x7fe3a80a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:51.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:51.833+0000 7fe3baffd700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 0x7fe3a80a3ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009bf0 tx=0x7fe3a4009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:51 smithi031 ceph-mon[98539]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:51 smithi146 ceph-mon[102809]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.342+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3bc101c70 con 0x7fe3bc1061d0 2023-10-06T21:21:52.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.344+0000 7fe3b3fff700 1 -- 172.21.15.31:0/1588519583 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11374 (secure 0 0 0) 0x7fe3ac099d60 con 0x7fe3bc1061d0 2023-10-06T21:21:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:21:52.347 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:19:01.835058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:21:52.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 msgr2=0x7fe3a80a3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 0x7fe3a80a3ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009bf0 tx=0x7fe3a4009260 comp rx=0 tx=0).stop 2023-10-06T21:21:52.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 msgr2=0x7fe3bc108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc108310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e9c0 tx=0x7fe3ac00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 shutdown_connections 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc108850 0x7fe3bc1a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe3a80a17e0 0x7fe3a80a3ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3bc1010b0 0x7fe3bc107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 --2- 172.21.15.31:0/1588519583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3bc1061d0 0x7fe3bc108310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.347+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 >> 172.21.15.31:0/1588519583 conn(0x7fe3bc0fc8d0 msgr2=0x7fe3bc0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.348+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 shutdown_connections 2023-10-06T21:21:52.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.348+0000 7fe3c15ff700 1 -- 172.21.15.31:0/1588519583 wait complete. 2023-10-06T21:21:52.352 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-10-06T21:21:52.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T21:21:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:51 smithi142 ceph-mon[102951]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 -- 172.21.15.31:0/3592219682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c068c90 msgr2=0x7f8b0c10dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 --2- 172.21.15.31:0/3592219682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c068c90 0x7f8b0c10dd00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b0a0 tx=0x7f8b0400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 -- 172.21.15.31:0/3592219682 shutdown_connections 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 --2- 172.21.15.31:0/3592219682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c068c90 0x7f8b0c10dd00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 --2- 172.21.15.31:0/3592219682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c106f60 0x7f8b0c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 --2- 172.21.15.31:0/3592219682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c0682d0 0x7f8b0c068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.668+0000 7f8b10bff700 1 -- 172.21.15.31:0/3592219682 >> 172.21.15.31:0/3592219682 conn(0x7f8b0c076b70 msgr2=0x7f8b0c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.669+0000 7f8b10bff700 1 -- 172.21.15.31:0/3592219682 shutdown_connections 2023-10-06T21:21:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.669+0000 7f8b10bff700 1 -- 172.21.15.31:0/3592219682 wait complete. 2023-10-06T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.670+0000 7f8b10bff700 1 Processor -- start 2023-10-06T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.670+0000 7f8b10bff700 1 -- start start 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c0682d0 0x7f8b0c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c068c90 0x7f8b0c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19c060 con 0x7f8b0c0682d0 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19c1d0 con 0x7f8b0c068c90 2023-10-06T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.671+0000 7f8b10bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19c390 con 0x7f8b0c106f60 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52364/0 (socket says 172.21.15.31:52364) 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 -- 172.21.15.31:0/2460341812 learned_addr learned my addr 172.21.15.31:0/2460341812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c0682d0 0x7f8b0c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c068c90 0x7f8b0c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 -- 172.21.15.31:0/2460341812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c068c90 msgr2=0x7f8b0c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c068c90 0x7f8b0c1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 -- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c0682d0 msgr2=0x7f8b0c196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c0682d0 0x7f8b0c196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.672+0000 7f8b0ad9d700 1 -- 172.21.15.31:0/2460341812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009d00 con 0x7f8b0c106f60 2023-10-06T21:21:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8b0ad9d700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b780 tx=0x7f8b040078a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8afb7fe700 1 -- 172.21.15.31:0/2460341812 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400e040 con 0x7f8b0c106f60 2023-10-06T21:21:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8afb7fe700 1 -- 172.21.15.31:0/2460341812 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b04003e90 con 0x7f8b0c106f60 2023-10-06T21:21:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8afb7fe700 1 -- 172.21.15.31:0/2460341812 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04004480 con 0x7f8b0c106f60 2023-10-06T21:21:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c19c610 con 0x7f8b0c106f60 2023-10-06T21:21:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.673+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c100150 con 0x7f8b0c106f60 2023-10-06T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.680+0000 7f8afb7fe700 1 -- 172.21.15.31:0/2460341812 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8b04004c60 con 0x7f8b0c106f60 2023-10-06T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.680+0000 7f8b244a6b80 1 Processor -- start 2023-10-06T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.680+0000 7f8b244a6b80 1 -- start start 2023-10-06T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82b4be0 0x55d7b82b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b820a400 con 0x55d7b82b4be0 2023-10-06T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b81fbbc0 con 0x55d7b82a9410 2023-10-06T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.681+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b81f94e0 con 0x55d7b82a9d30 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.682+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.682+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56626/0 (socket says 172.21.15.31:56626) 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.682+0000 7f8b09d9b700 1 -- 172.21.15.31:0/936206336 learned_addr learned my addr 172.21.15.31:0/936206336 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.682+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.682+0000 7f8afb7fe700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 0x7f8af40a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82b4be0 0x55d7b82b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 -- 172.21.15.31:0/936206336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 msgr2=0x55d7b82b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 -- 172.21.15.31:0/936206336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82b4be0 msgr2=0x55d7b82b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82b4be0 0x55d7b82b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8afb7fe700 1 -- 172.21.15.31:0/2460341812 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8b040cdd70 con 0x7f8b0c106f60 2023-10-06T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 -- 172.21.15.31:0/936206336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b801fd80 con 0x55d7b82a9410 2023-10-06T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b0ad9d700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 0x7f8af40a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.683+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0125f0 tx=0x7f8afc009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2ef692b7c1418d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.684+0000 7f8af8ff9700 1 -- 172.21.15.31:0/936206336 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc004250 con 0x55d7b82a9410 2023-10-06T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.684+0000 7f8af8ff9700 1 -- 172.21.15.31:0/936206336 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8afc0043b0 con 0x55d7b82a9410 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.684+0000 7f8af8ff9700 1 -- 172.21.15.31:0/936206336 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f870 con 0x55d7b82a9410 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 -- 172.21.15.31:0/936206336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 msgr2=0x55d7b82a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0125f0 tx=0x7f8afc009820 comp rx=0 tx=0).stop 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b0ad9d700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 0x7f8af40a3cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cb2f0 tx=0x7f8b040099e0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 -- 172.21.15.31:0/936206336 shutdown_connections 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82b4be0 0x55d7b82b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82a9410 0x55d7b82a97f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/936206336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.685+0000 7f8b244a6b80 1 -- 172.21.15.31:0/936206336 >> 172.21.15.31:0/936206336 conn(0x55d7b8216200 msgr2=0x55d7b829cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.686+0000 7f8b244a6b80 1 -- 172.21.15.31:0/936206336 shutdown_connections 2023-10-06T21:21:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.686+0000 7f8b244a6b80 1 -- 172.21.15.31:0/936206336 wait complete. 2023-10-06T21:21:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.687+0000 7f8b244a6b80 1 -- start start 2023-10-06T21:21:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.687+0000 7f8b244a6b80 1 Processor -- start 2023-10-06T21:21:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b81f4dd0 con 0x55d7b82a9410 2023-10-06T21:21:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b81916b0 con 0x55d7b82b4be0 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8191350 con 0x55d7b82a9d30 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.688+0000 7f8b244a6b80 10 client.0 _ll_register_callbacks cb 0x55d7b7fee7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.689+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.689+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44618/0 (socket says 172.21.15.31:44618) 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.689+0000 7f8b09d9b700 1 -- 172.21.15.31:0/1332899301 learned_addr learned my addr 172.21.15.31:0/1332899301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.689+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.689+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 -- 172.21.15.31:0/1332899301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 msgr2=0x55d7b82b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 -- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 msgr2=0x55d7b82b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 -- 172.21.15.31:0/1332899301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc009d20 con 0x55d7b82a9410 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8afc003f20 tx=0x7f8afc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88385732351fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f6f0 con 0x55d7b82a9410 2023-10-06T21:21:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8afc00f850 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.690+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7b8294d50 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.691+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8afc00fa10 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.691+0000 7f8b244a6b80 10 client.15894 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.691+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7b8008c00 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d7b8294d50 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8afc01e8d0 con 0x55d7b82a9410 2023-10-06T21:21:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 20 client.15894 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b82c4600 0x55d7b82c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 4 client.15894 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7b81eddb0 con 0x55d7b82c4600 2023-10-06T21:21:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.692+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b82c4600 0x55d7b82c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.693+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b82c4600 0x55d7b82c6a20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db099578b8459125 server_cookie=54920550aaef181d in_seq=0 out_seq=0 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.693+0000 7f8af1ffb700 10 client.15894 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.694+0000 7f8af1ffb700 1 -- 172.21.15.31:0/1332899301 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7b81eddb0 con 0x55d7b82c4600 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.694+0000 7f8af1ffb700 10 client.15894 handle_command_reply: tid=0 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.694+0000 7f8b244a6b80 1 client.15894 shutdown 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 msgr2=0x55d7b82b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8afc003f20 tx=0x7f8afc0057f0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 shutdown_connections 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b82c4600 0x55d7b82c6a20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82b1950 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/1332899301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8af1ffb700 0 client.15894 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 >> 172.21.15.31:0/1332899301 conn(0x55d7b8216200 msgr2=0x55d7b829cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.695+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 shutdown_connections 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.696+0000 7f8b244a6b80 1 -- 172.21.15.31:0/1332899301 wait complete. 2023-10-06T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.696+0000 7f8b244a6b80 20 client.15894 trim_cache size 0 max 16384 2023-10-06T21:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.725+0000 7f8b244a6b80 1 Processor -- start 2023-10-06T21:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 -- start start 2023-10-06T21:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b80758c0 con 0x55d7b82a9410 2023-10-06T21:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8294c00 con 0x55d7b82b4be0 2023-10-06T21:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.726+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b82bbc70 con 0x55d7b82a9d30 2023-10-06T21:21:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.727+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.727+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44644/0 (socket says 172.21.15.31:44644) 2023-10-06T21:21:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.727+0000 7f8b09d9b700 1 -- 172.21.15.31:0/2688665937 learned_addr learned my addr 172.21.15.31:0/2688665937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.727+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 -- 172.21.15.31:0/2688665937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 msgr2=0x55d7b82aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 -- 172.21.15.31:0/2688665937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 msgr2=0x55d7b82ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 -- 172.21.15.31:0/2688665937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b80461f0 con 0x55d7b82a9d30 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82bb730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d7b82c49e0 tx=0x7f8b0000d7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b3e9f50ebc5068 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8af97fa700 1 -- 172.21.15.31:0/2688665937 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00018040 con 0x55d7b82a9d30 2023-10-06T21:21:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8af97fa700 1 -- 172.21.15.31:0/2688665937 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b000054b0 con 0x55d7b82a9d30 2023-10-06T21:21:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.728+0000 7f8af97fa700 1 -- 172.21.15.31:0/2688665937 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00012450 con 0x55d7b82a9d30 2023-10-06T21:21:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 -- 172.21.15.31:0/1004360154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1061d0 msgr2=0x7ff80c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 --2- 172.21.15.31:0/1004360154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1061d0 0x7ff80c10dd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff80400b0a0 tx=0x7ff80400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 -- 172.21.15.31:0/1004360154 shutdown_connections 2023-10-06T21:21:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 --2- 172.21.15.31:0/1004360154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1061d0 0x7ff80c10dd50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 --2- 172.21.15.31:0/1004360154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c1010b0 0x7ff80c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 --2- 172.21.15.31:0/1004360154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c101a60 0x7ff80c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 -- 172.21.15.31:0/1004360154 >> 172.21.15.31:0/1004360154 conn(0x7ff80c0fc950 msgr2=0x7ff80c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.729+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2688665937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 msgr2=0x55d7b82bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.729+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82bb730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d7b82c49e0 tx=0x7f8b0000d7d0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.729+0000 7ff8113ff700 1 -- 172.21.15.31:0/1004360154 shutdown_connections 2023-10-06T21:21:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2688665937 shutdown_connections 2023-10-06T21:21:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b82aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9410 0x55d7b82ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2688665937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9d30 0x55d7b82bb730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2688665937 >> 172.21.15.31:0/2688665937 conn(0x55d7b8216200 msgr2=0x55d7b82acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.730+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2688665937 shutdown_connections 2023-10-06T21:21:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.730+0000 7ff8113ff700 1 -- 172.21.15.31:0/1004360154 wait complete. 2023-10-06T21:21:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.731+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2688665937 wait complete. 2023-10-06T21:21:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.731+0000 7ff8113ff700 1 Processor -- start 2023-10-06T21:21:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.731+0000 7f8b244a6b80 1 -- start start 2023-10-06T21:21:52.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.731+0000 7ff8113ff700 1 -- start start 2023-10-06T21:21:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.732+0000 7f8b244a6b80 1 Processor -- start 2023-10-06T21:21:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.732+0000 7ff8113ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.732+0000 7ff8113ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.732+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52396/0 (socket says 172.21.15.31:52396) 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff80affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff80affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56662/0 (socket says 172.21.15.31:56662) 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff8113ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff8113ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c19e460 con 0x7ff80c19d9e0 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff8113ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c19e5d0 con 0x7ff80c1010b0 2023-10-06T21:21:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff8113ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c19e790 con 0x7ff80c1061d0 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9d30 0x55d7b8311be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b8310430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8310970 con 0x55d7b82a9d30 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8310ae0 con 0x55d7b82b4be0 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8310ca0 con 0x55d7b82a9410 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.733+0000 7f8b244a6b80 10 client.0 _ll_register_callbacks cb 0x7f8afc012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.734+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b8310430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.734+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.734+0000 7f8b09d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52398/0 (socket says 172.21.15.31:52398) 2023-10-06T21:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.734+0000 7f8b09d9b700 1 -- 172.21.15.31:0/2341799282 learned_addr learned my addr 172.21.15.31:0/2341799282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.733+0000 7ff80b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44654/0 (socket says 172.21.15.31:44654) 2023-10-06T21:21:52.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 -- 172.21.15.31:0/2479698343 learned_addr learned my addr 172.21.15.31:0/2479698343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.734+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9d30 0x55d7b8311be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 -- 172.21.15.31:0/2479698343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 msgr2=0x7ff80c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 -- 172.21.15.31:0/2479698343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 msgr2=0x7ff80c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80b7fe700 1 -- 172.21.15.31:0/2479698343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009d00 con 0x7ff80c19d9e0 2023-10-06T21:21:52.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.734+0000 7ff80affd700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 -- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 msgr2=0x55d7b8310430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b8310430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 -- 172.21.15.31:0/2341799282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9d30 msgr2=0x55d7b8311be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9d30 0x55d7b8311be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 -- 172.21.15.31:0/2341799282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7b8008c00 con 0x55d7b82a9410 2023-10-06T21:21:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.735+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff8040062a0 tx=0x7ff80400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.735+0000 7ff803fff700 1 -- 172.21.15.31:0/2479698343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400e040 con 0x7ff80c19d9e0 2023-10-06T21:21:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.735+0000 7ff803fff700 1 -- 172.21.15.31:0/2479698343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff804004c50 con 0x7ff80c19d9e0 2023-10-06T21:21:52.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.735+0000 7ff803fff700 1 -- 172.21.15.31:0/2479698343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040052a0 con 0x7ff80c19d9e0 2023-10-06T21:21:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.735+0000 7f8b09d9b700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8afc016040 tx=0x7f8afc003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e80f1c8cd1a069 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.736+0000 7f8af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.736+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c1a5c90 con 0x7ff80c19d9e0 2023-10-06T21:21:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.736+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c1a60c0 con 0x7ff80c19d9e0 2023-10-06T21:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.736+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc03f020 con 0x55d7b82a9410 2023-10-06T21:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.736+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8afc009260 con 0x55d7b82a9410 2023-10-06T21:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.736+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8afc0043b0 con 0x55d7b82a9410 2023-10-06T21:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.737+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8afc005450 con 0x55d7b82a9410 2023-10-06T21:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.737+0000 7f8b244a6b80 10 client.25597 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.737+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8b0000d9e0 con 0x55d7b82a9410 2023-10-06T21:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.738+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8afc0043b0 con 0x55d7b82a9410 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.738+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8afc004730 con 0x55d7b82a9410 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 20 client.25597 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b8319330 0x55d7b831b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 4 client.25597 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.739+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7b81edad0 con 0x55d7b8319330 2023-10-06T21:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.742+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b8319330 0x55d7b831b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.742+0000 7ff803fff700 1 -- 172.21.15.31:0/2479698343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7ff80401d470 con 0x7ff80c19d9e0 2023-10-06T21:21:52.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.743+0000 7ff824e1ab80 1 Processor -- start 2023-10-06T21:21:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.743+0000 7f8b0a59c700 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b8319330 0x55d7b831b750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2558d77cc1f3a918 server_cookie=493a4fe181bccd65 in_seq=0 out_seq=0 2023-10-06T21:21:52.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.743+0000 7ff824e1ab80 1 -- start start 2023-10-06T21:21:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.744+0000 7f8af1ffb700 10 client.25597 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 0x559a3465dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a34668b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a346690a0 0x559a3466b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a345be460 con 0x559a3465d8d0 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a345afc20 con 0x559a3465e1f0 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a345ad540 con 0x559a346690a0 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.744+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 0x559a3465dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 0x559a3465dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44676/0 (socket says 172.21.15.31:44676) 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80a7fc700 1 -- 172.21.15.31:0/1057721492 learned_addr learned my addr 172.21.15.31:0/1057721492 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a34668b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a346690a0 0x559a3466b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 -- 172.21.15.31:0/1057721492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a346690a0 msgr2=0x559a3466b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a346690a0 0x559a3466b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 -- 172.21.15.31:0/1057721492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 msgr2=0x559a3465dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 0x559a3465dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff80affd700 1 -- 172.21.15.31:0/1057721492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a343d3de0 con 0x559a3465e1f0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.745+0000 7ff803fff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 0x7ff7f00a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff803fff700 1 -- 172.21.15.31:0/2479698343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff8040cec50 con 0x7ff80c19d9e0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff80affd700 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a34668b60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00a9d0 tx=0x7ff7fc00ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72712600fb293cec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 0x7ff7f00a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff8017fa700 1 -- 172.21.15.31:0/1057721492 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00ca90 con 0x559a3465e1f0 2023-10-06T21:21:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff8017fa700 1 -- 172.21.15.31:0/1057721492 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7fc00ae70 con 0x559a3465e1f0 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.746+0000 7ff8017fa700 1 -- 172.21.15.31:0/1057721492 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc011e50 con 0x559a3465e1f0 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 -- 172.21.15.31:0/1057721492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 msgr2=0x559a34668b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a34668b60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00a9d0 tx=0x7ff7fc00ace0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 -- 172.21.15.31:0/1057721492 shutdown_connections 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a346690a0 0x559a3466b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465d8d0 0x559a3465dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/1057721492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a34668b60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.747+0000 7ff824e1ab80 1 -- 172.21.15.31:0/1057721492 >> 172.21.15.31:0/1057721492 conn(0x559a345ca260 msgr2=0x559a34651450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.748+0000 7f8af1ffb700 1 -- 172.21.15.31:0/2341799282 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7b81edad0 con 0x55d7b8319330 2023-10-06T21:21:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.748+0000 7f8af1ffb700 10 client.25597 handle_command_reply: tid=0 2023-10-06T21:21:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.748+0000 7f8b244a6b80 1 client.25597 shutdown 2023-10-06T21:21:52.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.748+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 0x7ff7f00a3ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8015670 tx=0x7ff7f8017040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.748+0000 7ff824e1ab80 1 -- 172.21.15.31:0/1057721492 shutdown_connections 2023-10-06T21:21:52.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.748+0000 7ff824e1ab80 1 -- 172.21.15.31:0/1057721492 wait complete. 2023-10-06T21:21:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 msgr2=0x55d7b83116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8afc016040 tx=0x7f8afc003890 comp rx=0 tx=0).stop 2023-10-06T21:21:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 shutdown_connections 2023-10-06T21:21:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7b82b4be0 0x55d7b8310430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7b82a9410 0x55d7b83116a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7b8319330 0x55d7b831b750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 --2- 172.21.15.31:0/2341799282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7b82a9d30 0x55d7b8311be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8af1ffb700 0 client.25597 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.749+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 >> 172.21.15.31:0/2341799282 conn(0x55d7b8216200 msgr2=0x55d7b82b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.749+0000 7ff824e1ab80 1 -- start start 2023-10-06T21:21:52.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.749+0000 7ff824e1ab80 1 Processor -- start 2023-10-06T21:21:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.750+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 shutdown_connections 2023-10-06T21:21:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.750+0000 7f8b244a6b80 1 -- 172.21.15.31:0/2341799282 wait complete. 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a346662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 0x559a34666830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a345a8e30 con 0x559a346690a0 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34545710 con 0x559a3465e1f0 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a345453b0 con 0x559a3465d8d0 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff824e1ab80 10 client.0 _ll_register_callbacks cb 0x559a343a2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:52.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.750+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52420/0 (socket says 172.21.15.31:52420) 2023-10-06T21:21:52.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 -- 172.21.15.31:0/2623450491 learned_addr learned my addr 172.21.15.31:0/2623450491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a346662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.751+0000 7f8b244a6b80 20 client.25597 trim_cache size 0 max 16384 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 0x559a34666830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 -- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 msgr2=0x559a346662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a346662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 -- 172.21.15.31:0/2623450491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 msgr2=0x559a34666830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 0x559a34666830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80affd700 1 -- 172.21.15.31:0/2623450491 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc00a600 con 0x559a3465d8d0 2023-10-06T21:21:52.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.751+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 0x559a34666830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:21:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:21:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:21:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:21:52.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.752+0000 7ff80affd700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01a680 tx=0x7ff7fc01a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e88eef4e6157f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.752+0000 7ff7ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 msgr2=0x7f8af40a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 0x7f8af40a3cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cb2f0 tx=0x7f8b040099e0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 msgr2=0x7f8b0c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b780 tx=0x7f8b040078a0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 shutdown_connections 2023-10-06T21:21:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c106f60 0x7f8b0c19ba20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8af40a1810 0x7f8af40a3cd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b0c0682d0 0x7f8b0c196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 --2- 172.21.15.31:0/2460341812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c068c90 0x7f8b0c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 >> 172.21.15.31:0/2460341812 conn(0x7f8b0c076b70 msgr2=0x7f8b0c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.752+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01f070 con 0x559a3465d8d0 2023-10-06T21:21:52.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.752+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff7fc01a800 con 0x559a3465d8d0 2023-10-06T21:21:52.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.752+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a34648f10 con 0x559a3465d8d0 2023-10-06T21:21:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.752+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 shutdown_connections 2023-10-06T21:21:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:21:52.753+0000 7f8b10bff700 1 -- 172.21.15.31:0/2460341812 wait complete. 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.753+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7fc006d00 con 0x559a3465d8d0 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.753+0000 7ff824e1ab80 10 client.25603 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.753+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559a343bcc60 con 0x559a3465d8d0 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559a34648f10 con 0x559a3465d8d0 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7fc02d070 con 0x559a3465d8d0 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 10 client.25603 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:52.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 10 client.25603 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 20 client.25603 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346789a0 0x559a3467adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 4 client.25603 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.754+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a345a1e10 con 0x559a346789a0 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.755+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346789a0 0x559a3467adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.755+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346789a0 0x559a3467adc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f308c55dbd9fb19 server_cookie=6616988992982535 in_seq=0 out_seq=0 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.756+0000 7ff7ee7fc700 10 client.25603 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.756+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/2623450491 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559a345a1e10 con 0x559a346789a0 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.756+0000 7ff7ee7fc700 10 client.25603 handle_command_reply: tid=0 2023-10-06T21:21:52.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 client.25603 shutdown 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 msgr2=0x559a34665db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01a680 tx=0x7ff7fc01a6b0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 shutdown_connections 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346789a0 0x559a3467adc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a346690a0 0x559a34666830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a34665db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/2623450491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a3465e1f0 0x559a346662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 >> 172.21.15.31:0/2623450491 conn(0x559a345ca260 msgr2=0x559a34650fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.757+0000 7ff7ee7fc700 0 client.25603 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.758+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 shutdown_connections 2023-10-06T21:21:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.758+0000 7ff824e1ab80 1 -- 172.21.15.31:0/2623450491 wait complete. 2023-10-06T21:21:52.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.758+0000 7ff824e1ab80 20 client.25603 trim_cache size 0 max 16384 2023-10-06T21:21:52.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.784+0000 7ff824e1ab80 1 Processor -- start 2023-10-06T21:21:52.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.784+0000 7ff824e1ab80 1 -- start start 2023-10-06T21:21:52.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465e1f0 0x559a3466fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a34666660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34429920 con 0x559a3465e1f0 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34664330 con 0x559a346690a0 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34661c40 con 0x559a3465d8d0 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52434/0 (socket says 172.21.15.31:52434) 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff80affd700 1 -- 172.21.15.31:0/4158169605 learned_addr learned my addr 172.21.15.31:0/4158169605 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465e1f0 0x559a3466fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.785+0000 7ff80b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a34666660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 -- 172.21.15.31:0/4158169605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 msgr2=0x559a34666660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a34666660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 -- 172.21.15.31:0/4158169605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465e1f0 msgr2=0x559a3466fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465e1f0 0x559a3466fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 -- 172.21.15.31:0/4158169605 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a3464d140 con 0x559a3465d8d0 2023-10-06T21:21:52.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.786+0000 7ff80affd700 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011a80 tx=0x7ff7fc00c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed76acf45f0d59c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.787+0000 7ff801ffb700 1 -- 172.21.15.31:0/4158169605 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01f070 con 0x559a3465d8d0 2023-10-06T21:21:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.787+0000 7ff801ffb700 1 -- 172.21.15.31:0/4158169605 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7fc00cd70 con 0x559a3465d8d0 2023-10-06T21:21:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.787+0000 7ff801ffb700 1 -- 172.21.15.31:0/4158169605 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01ac00 con 0x559a3465d8d0 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.787+0000 7ff824e1ab80 1 -- 172.21.15.31:0/4158169605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 msgr2=0x559a346608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.787+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011a80 tx=0x7ff7fc00c4b0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 -- 172.21.15.31:0/4158169605 shutdown_connections 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a34666660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465d8d0 0x559a346608c0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011a80 tx=0x7ff7fc00c4b0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/4158169605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3465e1f0 0x559a3466fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 -- 172.21.15.31:0/4158169605 >> 172.21.15.31:0/4158169605 conn(0x559a345ca260 msgr2=0x559a3467a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 -- 172.21.15.31:0/4158169605 shutdown_connections 2023-10-06T21:21:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 -- 172.21.15.31:0/4158169605 wait complete. 2023-10-06T21:21:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.788+0000 7ff824e1ab80 1 -- start start 2023-10-06T21:21:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.789+0000 7ff824e1ab80 1 Processor -- start 2023-10-06T21:21:52.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a3466dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3466e110 0x559a3466e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34659d90 con 0x559a3466e110 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a34659f00 con 0x559a346690a0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a3465a0c0 con 0x559a3465e1f0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff824e1ab80 10 client.0 _ll_register_callbacks cb 0x7ff7f8009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52446/0 (socket says 172.21.15.31:52446) 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80affd700 1 -- 172.21.15.31:0/3660333473 learned_addr learned my addr 172.21.15.31:0/3660333473 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a3466dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80b7fe700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3466e110 0x559a3466e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80affd700 1 -- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 msgr2=0x559a3466dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.790+0000 7ff80affd700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a3466dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff80affd700 1 -- 172.21.15.31:0/3660333473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3466e110 msgr2=0x559a3466e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff80affd700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3466e110 0x559a3466e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff80affd700 1 -- 172.21.15.31:0/3660333473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a343bcc60 con 0x559a3465e1f0 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a3466dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff80affd700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011f20 tx=0x7ff7fc011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ec21b9a987aee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff7ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01f070 con 0x559a3465e1f0 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff7fc000e40 con 0x559a3465e1f0 2023-10-06T21:21:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.791+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7fc00ae70 con 0x559a3465e1f0 2023-10-06T21:21:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.792+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7fc00c520 con 0x559a3465e1f0 2023-10-06T21:21:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.792+0000 7ff824e1ab80 10 client.25609 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.792+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559a343fa250 con 0x559a3465e1f0 2023-10-06T21:21:52.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.792+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7fc00ae70 con 0x559a3465e1f0 2023-10-06T21:21:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7fc02d050 con 0x559a3465e1f0 2023-10-06T21:21:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 10 client.25609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:21:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 20 client.25609 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:52.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346d57d0 0x559a346d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 4 client.25609 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a345a1b30 con 0x559a346d57d0 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.793+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346d57d0 0x559a346d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.794+0000 7ff80a7fc700 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346d57d0 0x559a346d7bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f629c9ce0a5c8e2f server_cookie=10dccf3e5de13684 in_seq=0 out_seq=0 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.794+0000 7ff7ee7fc700 10 client.25609 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.794+0000 7ff7ee7fc700 1 -- 172.21.15.31:0/3660333473 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a345a1b30 con 0x559a346d57d0 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.794+0000 7ff7ee7fc700 10 client.25609 handle_command_reply: tid=0 2023-10-06T21:21:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.794+0000 7ff824e1ab80 1 client.25609 shutdown 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 msgr2=0x559a346639a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011f20 tx=0x7ff7fc011c70 comp rx=0 tx=0).stop 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 shutdown_connections 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x559a346d57d0 0x559a346d7bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3466e110 0x559a3466e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a3465e1f0 0x559a346639a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 --2- 172.21.15.31:0/3660333473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559a346690a0 0x559a3466dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 >> 172.21.15.31:0/3660333473 conn(0x559a345ca260 msgr2=0x559a34661100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff7ee7fc700 0 client.25609 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 shutdown_connections 2023-10-06T21:21:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 1 -- 172.21.15.31:0/3660333473 wait complete. 2023-10-06T21:21:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.795+0000 7ff824e1ab80 20 client.25609 trim_cache size 0 max 16384 2023-10-06T21:21:52.802 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 msgr2=0x7ff7f00a3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 0x7ff7f00a3ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8015670 tx=0x7ff7f8017040 comp rx=0 tx=0).stop 2023-10-06T21:21:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 msgr2=0x7ff80c19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff8040062a0 tx=0x7ff80400bb80 comp rx=0 tx=0).stop 2023-10-06T21:21:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 shutdown_connections 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff7f00a17e0 0x7ff7f00a3ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c19d9e0 0x7ff80c19de60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c1010b0 0x7ff80c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 --2- 172.21.15.31:0/2479698343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff80c1061d0 0x7ff80c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.796+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 >> 172.21.15.31:0/2479698343 conn(0x7ff80c0fc950 msgr2=0x7ff80c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.797+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 shutdown_connections 2023-10-06T21:21:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:52.797+0000 7ff8113ff700 1 -- 172.21.15.31:0/2479698343 wait complete. 2023-10-06T21:21:52.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T21:21:53.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 -- 172.21.15.31:0/4167753589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c101a60 msgr2=0x7f511c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 --2- 172.21.15.31:0/4167753589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c101a60 0x7f511c105aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f5110009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 -- 172.21.15.31:0/4167753589 shutdown_connections 2023-10-06T21:21:53.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 --2- 172.21.15.31:0/4167753589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c1061d0 0x7f511c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 --2- 172.21.15.31:0/4167753589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 0x7f511c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 --2- 172.21.15.31:0/4167753589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c101a60 0x7f511c105aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 -- 172.21.15.31:0/4167753589 >> 172.21.15.31:0/4167753589 conn(0x7f511c0fc950 msgr2=0x7f511c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 -- 172.21.15.31:0/4167753589 shutdown_connections 2023-10-06T21:21:53.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.173+0000 7f51223ff700 1 -- 172.21.15.31:0/4167753589 wait complete. 2023-10-06T21:21:53.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.175+0000 7f51223ff700 1 Processor -- start 2023-10-06T21:21:53.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.175+0000 7f51223ff700 1 -- start start 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.175+0000 7f51223ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 0x7f511c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.175+0000 7f51223ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 0x7f511c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f51223ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f51223ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19e480 con 0x7f511c1010b0 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f51223ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19e5f0 con 0x7f511c1061d0 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f51223ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19e7b0 con 0x7f511c19da00 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f511bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 0x7f511c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:32784/0 (socket says 172.21.15.31:32784) 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 -- 172.21.15.31:0/2960493535 learned_addr learned my addr 172.21.15.31:0/2960493535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f511b7fe700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 0x7f511c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 -- 172.21.15.31:0/2960493535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 msgr2=0x7f511c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 0x7f511c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 -- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 msgr2=0x7f511c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.176+0000 7f512099c700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 0x7f511c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f512099c700 1 -- 172.21.15.31:0/2960493535 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009520 con 0x7f511c19da00 2023-10-06T21:21:53.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f511b7fe700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 0x7f511c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:53.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f512099c700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f511400ef30 tx=0x7f511400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f51197fa700 1 -- 172.21.15.31:0/2960493535 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114018070 con 0x7f511c19da00 2023-10-06T21:21:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f51197fa700 1 -- 172.21.15.31:0/2960493535 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f511400f040 con 0x7f511c19da00 2023-10-06T21:21:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f51197fa700 1 -- 172.21.15.31:0/2960493535 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400cc70 con 0x7f511c19da00 2023-10-06T21:21:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.177+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51100095f0 con 0x7f511c19da00 2023-10-06T21:21:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.178+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c1a5fb0 con 0x7f511c19da00 2023-10-06T21:21:53.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.184+0000 7f51197fa700 1 -- 172.21.15.31:0/2960493535 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f5114007500 con 0x7f511c19da00 2023-10-06T21:21:53.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.185+0000 7f5135d57b80 1 Processor -- start 2023-10-06T21:21:53.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.185+0000 7f5135d57b80 1 -- start start 2023-10-06T21:21:53.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93716c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af93672460 con 0x55af937099e0 2023-10-06T21:21:53.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af93663c20 con 0x55af93714890 2023-10-06T21:21:53.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f5135d57b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af93661540 con 0x55af937090c0 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f511bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f511bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36100/0 (socket says 172.21.15.31:36100) 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.186+0000 7f511bfff700 1 -- 172.21.15.31:0/4085414085 learned_addr learned my addr 172.21.15.31:0/4085414085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f51197fa700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 0x7f51040a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f51197fa700 1 -- 172.21.15.31:0/2960493535 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f51140ccec0 con 0x7f511c19da00 2023-10-06T21:21:53.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511b7fe700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93716c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511b7fe700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f512099c700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 0x7f51040a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511bfff700 1 -- 172.21.15.31:0/4085414085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 msgr2=0x55af937094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511bfff700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511bfff700 1 -- 172.21.15.31:0/4085414085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 msgr2=0x55af93716c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511bfff700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93716c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.187+0000 7f511bfff700 1 -- 172.21.15.31:0/4085414085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af93487de0 con 0x55af937099e0 2023-10-06T21:21:53.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.188+0000 7f511b7fe700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:53.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.188+0000 7f511bfff700 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f510c009f10 tx=0x7f510c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1786edc0d8aad693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.188+0000 7f51027fc700 1 -- 172.21.15.31:0/4085414085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c019630 con 0x55af937099e0 2023-10-06T21:21:53.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.188+0000 7f51027fc700 1 -- 172.21.15.31:0/4085414085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f510c019c20 con 0x55af937099e0 2023-10-06T21:21:53.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.188+0000 7f51027fc700 1 -- 172.21.15.31:0/4085414085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00eb50 con 0x55af937099e0 2023-10-06T21:21:53.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.189+0000 7f5135d57b80 1 -- 172.21.15.31:0/4085414085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 msgr2=0x55af93714350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.189+0000 7f512099c700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 0x7f51040a3cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51140ca440 tx=0x7f5114004ad0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.189+0000 7f5135d57b80 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f510c009f10 tx=0x7f510c011720 comp rx=0 tx=0).stop 2023-10-06T21:21:53.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 -- 172.21.15.31:0/4085414085 shutdown_connections 2023-10-06T21:21:53.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93716c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 --2- 172.21.15.31:0/4085414085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93714350 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 -- 172.21.15.31:0/4085414085 >> 172.21.15.31:0/4085414085 conn(0x55af9367e260 msgr2=0x55af93705450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 -- 172.21.15.31:0/4085414085 shutdown_connections 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.190+0000 7f5135d57b80 1 -- 172.21.15.31:0/4085414085 wait complete. 2023-10-06T21:21:53.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.191+0000 7f5135d57b80 1 -- start start 2023-10-06T21:21:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.191+0000 7f5135d57b80 1 Processor -- start 2023-10-06T21:21:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af937114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93711b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9365ce30 con 0x55af937090c0 2023-10-06T21:21:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af935f9710 con 0x55af93714890 2023-10-06T21:21:53.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af935f93b0 con 0x55af937099e0 2023-10-06T21:21:53.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f5135d57b80 10 client.0 _ll_register_callbacks cb 0x55af93456800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36108/0 (socket says 172.21.15.31:36108) 2023-10-06T21:21:53.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.192+0000 7f511b7fe700 1 -- 172.21.15.31:0/1422647867 learned_addr learned my addr 172.21.15.31:0/1422647867 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511bfff700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af937114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93711b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 -- 172.21.15.31:0/1422647867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 msgr2=0x55af937114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af937114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 -- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 msgr2=0x55af93711b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93711b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 -- 172.21.15.31:0/1422647867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c009b70 con 0x55af937090c0 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511bfff700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af937114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.193+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f511002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ca0603a17ad7bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f50fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:53.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100279d0 con 0x55af937090c0 2023-10-06T21:21:53.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f5110027b30 con 0x55af937090c0 2023-10-06T21:21:53.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af936fcf10 con 0x55af937090c0 2023-10-06T21:21:53.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5110027cf0 con 0x55af937090c0 2023-10-06T21:21:53.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f5135d57b80 10 client.15924 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.194+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af93470c60 con 0x55af937090c0 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.195+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af936fcf10 con 0x55af937090c0 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5110004970 con 0x55af937090c0 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 10 client.15924 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:21:53.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 20 client.15924 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:53.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af93724270 0x55af93726690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 4 client.15924 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:53.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af93655e10 con 0x55af93724270 2023-10-06T21:21:53.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.196+0000 7f511bfff700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af93724270 0x55af93726690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.197+0000 7f511bfff700 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af93724270 0x55af93726690 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89904cb39af85f09 server_cookie=2b10f4e2a0376072 in_seq=0 out_seq=0 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.197+0000 7f50fb7fe700 10 client.15924 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.198+0000 7f50fb7fe700 1 -- 172.21.15.31:0/1422647867 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af93655e10 con 0x55af93724270 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.198+0000 7f50fb7fe700 10 client.15924 handle_command_reply: tid=0 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.198+0000 7f5135d57b80 1 client.15924 shutdown 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.198+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 msgr2=0x55af93714270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.198+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f511002b040 comp rx=0 tx=0).stop 2023-10-06T21:21:53.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 shutdown_connections 2023-10-06T21:21:53.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af93724270 0x55af93726690 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93711b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93714270 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1422647867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af937114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 >> 172.21.15.31:0/1422647867 conn(0x55af9367e260 msgr2=0x55af93704fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f50fb7fe700 0 client.15924 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:53.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 shutdown_connections 2023-10-06T21:21:53.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 1 -- 172.21.15.31:0/1422647867 wait complete. 2023-10-06T21:21:53.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.199+0000 7f5135d57b80 20 client.15924 trim_cache size 0 max 16384 2023-10-06T21:21:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.225+0000 7f5135d57b80 1 Processor -- start 2023-10-06T21:21:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.225+0000 7f5135d57b80 1 -- start start 2023-10-06T21:21:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af93710cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93713e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af934dd920 con 0x55af937090c0 2023-10-06T21:21:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9370fa50 con 0x55af93714890 2023-10-06T21:21:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f5135d57b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9370d360 con 0x55af937099e0 2023-10-06T21:21:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36112/0 (socket says 172.21.15.31:36112) 2023-10-06T21:21:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f511b7fe700 1 -- 172.21.15.31:0/1246682866 learned_addr learned my addr 172.21.15.31:0/1246682866 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.226+0000 7f511bfff700 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af93710cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93713e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 -- 172.21.15.31:0/1246682866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 msgr2=0x55af93710cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af93710cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 -- 172.21.15.31:0/1246682866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 msgr2=0x55af93713e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93713e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 -- 172.21.15.31:0/1246682866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af934ae250 con 0x55af937090c0 2023-10-06T21:21:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.227+0000 7f511b7fe700 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55af937129c0 tx=0x7f5110009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acce6cc49556bcbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.228+0000 7f5102ffd700 1 -- 172.21.15.31:0/1246682866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110004870 con 0x55af937090c0 2023-10-06T21:21:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.228+0000 7f5102ffd700 1 -- 172.21.15.31:0/1246682866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f511002f070 con 0x55af937090c0 2023-10-06T21:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.228+0000 7f5102ffd700 1 -- 172.21.15.31:0/1246682866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110027c30 con 0x55af937090c0 2023-10-06T21:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 -- 172.21.15.31:0/1246682866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 msgr2=0x55af93726660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55af937129c0 tx=0x7f5110009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 -- 172.21.15.31:0/1246682866 shutdown_connections 2023-10-06T21:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93713e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937090c0 0x55af93726660 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 --2- 172.21.15.31:0/1246682866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937099e0 0x55af93710cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 -- 172.21.15.31:0/1246682866 >> 172.21.15.31:0/1246682866 conn(0x55af9367e260 msgr2=0x55af9371fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 -- 172.21.15.31:0/1246682866 shutdown_connections 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.229+0000 7f5135d57b80 1 -- 172.21.15.31:0/1246682866 wait complete. 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.230+0000 7f5135d57b80 1 -- start start 2023-10-06T21:21:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.230+0000 7f5135d57b80 1 Processor -- start 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93719e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9371b9c0 con 0x55af937099e0 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af937770a0 con 0x55af93714890 2023-10-06T21:21:53.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af937771e0 con 0x55af937090c0 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f5135d57b80 10 client.0 _ll_register_callbacks cb 0x7f5110029f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f511bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.231+0000 7f511bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36122/0 (socket says 172.21.15.31:36122) 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 -- 172.21.15.31:0/226184805 learned_addr learned my addr 172.21.15.31:0/226184805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511b7fe700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93719e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 -- 172.21.15.31:0/226184805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 msgr2=0x55af937193c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937193c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 -- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 msgr2=0x55af93719e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93719e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511bfff700 1 -- 172.21.15.31:0/226184805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af93470c60 con 0x55af937099e0 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.232+0000 7f511b7fe700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93719e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.233+0000 7f511bfff700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f510c009840 tx=0x7f510c0195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2800fb2c10d0cb0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.233+0000 7f50fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.233+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c029040 con 0x55af937099e0 2023-10-06T21:21:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.233+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f510c0052b0 con 0x55af937099e0 2023-10-06T21:21:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.233+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f510c019c20 con 0x55af937099e0 2023-10-06T21:21:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510c005470 con 0x55af937099e0 2023-10-06T21:21:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 10 client.15930 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af934862b0 con 0x55af937099e0 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f510c019c20 con 0x55af937099e0 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f510c024450 con 0x55af937099e0 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 10 client.15930 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:21:53.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 20 client.15930 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af937788e0 0x55af9377ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 4 client.15930 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.234+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af93655b30 con 0x55af937788e0 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.235+0000 7f511b7fe700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af937788e0 0x55af9377ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f511b7fe700 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af937788e0 0x55af9377ad00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a712279a958d274 server_cookie=651372e52b6c151e in_seq=0 out_seq=0 2023-10-06T21:21:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f50fb7fe700 10 client.15930 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f50fb7fe700 1 -- 172.21.15.31:0/226184805 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af93655b30 con 0x55af937788e0 2023-10-06T21:21:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f50fb7fe700 10 client.15930 handle_command_reply: tid=0 2023-10-06T21:21:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 client.15930 shutdown 2023-10-06T21:21:53.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 msgr2=0x55af93719900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f510c009840 tx=0x7f510c0195b0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 shutdown_connections 2023-10-06T21:21:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x55af937788e0 0x55af9377ad00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af93714890 0x55af93719e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55af937090c0 0x55af937193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 --2- 172.21.15.31:0/226184805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af937099e0 0x55af93719900 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 >> 172.21.15.31:0/226184805 conn(0x55af9367e260 msgr2=0x55af9370c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.236+0000 7f50fb7fe700 0 client.15930 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:21:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.237+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 shutdown_connections 2023-10-06T21:21:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.237+0000 7f5135d57b80 1 -- 172.21.15.31:0/226184805 wait complete. 2023-10-06T21:21:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.237+0000 7f5135d57b80 20 client.15930 trim_cache size 0 max 16384 2023-10-06T21:21:53.254 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 msgr2=0x7f51040a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 0x7f51040a3cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51140ca440 tx=0x7f5114004ad0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 msgr2=0x7f511c19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f511400ef30 tx=0x7f511400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 shutdown_connections 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c19da00 0x7f511c19de80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f51040a1810 0x7f51040a3cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f511c1010b0 0x7f511c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 --2- 172.21.15.31:0/2960493535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f511c1061d0 0x7f511c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 >> 172.21.15.31:0/2960493535 conn(0x7f511c0fc950 msgr2=0x7f511c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 shutdown_connections 2023-10-06T21:21:53.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.238+0000 7f51223ff700 1 -- 172.21.15.31:0/2960493535 wait complete. 2023-10-06T21:21:53.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:52 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1588519583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:53.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:21:53.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:52 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1588519583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:53.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:52 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1588519583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:21:53.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 -- 172.21.15.31:0/3945156449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241010b0 msgr2=0x7fda24101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 --2- 172.21.15.31:0/3945156449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241010b0 0x7fda24101490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda2000faf0 tx=0x7fda2000fe00 comp rx=0 tx=0).stop 2023-10-06T21:21:53.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 -- 172.21.15.31:0/3945156449 shutdown_connections 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 --2- 172.21.15.31:0/3945156449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda241061d0 0x7fda2410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 --2- 172.21.15.31:0/3945156449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241010b0 0x7fda24101490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 --2- 172.21.15.31:0/3945156449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda24101a60 0x7fda24105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 -- 172.21.15.31:0/3945156449 >> 172.21.15.31:0/3945156449 conn(0x7fda240fc950 msgr2=0x7fda240fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 -- 172.21.15.31:0/3945156449 shutdown_connections 2023-10-06T21:21:53.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.614+0000 7fda2bdff700 1 -- 172.21.15.31:0/3945156449 wait complete. 2023-10-06T21:21:53.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.616+0000 7fda2bdff700 1 Processor -- start 2023-10-06T21:21:53.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.616+0000 7fda2bdff700 1 -- start start 2023-10-06T21:21:53.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241061d0 0x7fda24199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda24194ad0 0x7fda24194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24195550 con 0x7fda24194ad0 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda241956c0 con 0x7fda241010b0 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24195880 con 0x7fda241061d0 2023-10-06T21:21:53.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60194/0 (socket says 172.21.15.31:60194) 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda29b9b700 1 -- 172.21.15.31:0/1480033506 learned_addr learned my addr 172.21.15.31:0/1480033506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2939a700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241061d0 0x7fda24199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.617+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda24194ad0 0x7fda24194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 -- 172.21.15.31:0/1480033506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241061d0 msgr2=0x7fda24199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241061d0 0x7fda24199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 -- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda24194ad0 msgr2=0x7fda24194f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda24194ad0 0x7fda24194f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 -- 172.21.15.31:0/1480033506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2000f750 con 0x7fda241010b0 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.618+0000 7fda29b9b700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda20019040 tx=0x7fda20005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.619+0000 7fda1affd700 1 -- 172.21.15.31:0/1480033506 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20003ba0 con 0x7fda241010b0 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.619+0000 7fda1affd700 1 -- 172.21.15.31:0/1480033506 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda20004190 con 0x7fda241010b0 2023-10-06T21:21:53.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.619+0000 7fda1affd700 1 -- 172.21.15.31:0/1480033506 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001fb60 con 0x7fda241010b0 2023-10-06T21:21:53.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.620+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda24198640 con 0x7fda241010b0 2023-10-06T21:21:53.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.620+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda24198a70 con 0x7fda241010b0 2023-10-06T21:21:53.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.625+0000 7fda1affd700 1 -- 172.21.15.31:0/1480033506 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fda20003d00 con 0x7fda241010b0 2023-10-06T21:21:53.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.626+0000 7fda3f650b80 1 Processor -- start 2023-10-06T21:21:53.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 -- start start 2023-10-06T21:21:53.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cc8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c24460 con 0x556571cbb9e0 2023-10-06T21:21:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c15c20 con 0x556571cc6890 2023-10-06T21:21:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.627+0000 7fda3f650b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c13540 con 0x556571cbb0c0 2023-10-06T21:21:53.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda1affd700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 0x7fda100a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda2a39c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda1affd700 1 -- 172.21.15.31:0/1480033506 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fda200ce940 con 0x7fda241010b0 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36154/0 (socket says 172.21.15.31:36154) 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda29b9b700 1 -- 172.21.15.31:0/1543100598 learned_addr learned my addr 172.21.15.31:0/1543100598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.628+0000 7fda2939a700 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cc8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 0x7fda100a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 -- 172.21.15.31:0/1543100598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 msgr2=0x556571cc8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cc8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 -- 172.21.15.31:0/1543100598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 msgr2=0x556571cc6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 -- 172.21.15.31:0/1543100598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556571a39de0 con 0x556571cbb0c0 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.629+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cbb4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009790 tx=0x7fda1c004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda0a8338685c916 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.630+0000 7fda0ffff700 1 -- 172.21.15.31:0/1543100598 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c009940 con 0x556571cbb0c0 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.630+0000 7fda0ffff700 1 -- 172.21.15.31:0/1543100598 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda1c00f040 con 0x556571cbb0c0 2023-10-06T21:21:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.630+0000 7fda0ffff700 1 -- 172.21.15.31:0/1543100598 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01a470 con 0x556571cbb0c0 2023-10-06T21:21:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 -- 172.21.15.31:0/1543100598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 msgr2=0x556571cbb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cbb4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009790 tx=0x7fda1c004320 comp rx=0 tx=0).stop 2023-10-06T21:21:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda2a39c700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 0x7fda100a3c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda1c021000 tx=0x7fda1c01f000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 -- 172.21.15.31:0/1543100598 shutdown_connections 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cc8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cbb4a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 --2- 172.21.15.31:0/1543100598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 -- 172.21.15.31:0/1543100598 >> 172.21.15.31:0/1543100598 conn(0x556571c30260 msgr2=0x556571cb7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 -- 172.21.15.31:0/1543100598 shutdown_connections 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.631+0000 7fda3f650b80 1 -- 172.21.15.31:0/1543100598 wait complete. 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.632+0000 7fda3f650b80 1 -- start start 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.632+0000 7fda3f650b80 1 Processor -- start 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cc6890 0x556571cc3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c0ee30 con 0x556571cc6890 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571bab710 con 0x556571cbb9e0 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571bab3b0 con 0x556571cbb0c0 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.633+0000 7fda3f650b80 10 client.0 _ll_register_callbacks cb 0x556571a08800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.634+0000 7fda2939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.634+0000 7fda2939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60218/0 (socket says 172.21.15.31:60218) 2023-10-06T21:21:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.634+0000 7fda2939a700 1 -- 172.21.15.31:0/2548601330 learned_addr learned my addr 172.21.15.31:0/2548601330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.634+0000 7fda29b9b700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.634+0000 7fda2a39c700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cc6890 0x556571cc3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 -- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 msgr2=0x556571cc6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 -- 172.21.15.31:0/2548601330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cc6890 msgr2=0x556571cc3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cc6890 0x556571cc3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 -- 172.21.15.31:0/2548601330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c011040 con 0x556571cbb9e0 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda29b9b700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda2939a700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fda14011740 tx=0x7fda14011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8485a6a6b2ebb31b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.635+0000 7fda0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14016070 con 0x556571cbb9e0 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556571caef10 con 0x556571cbb9e0 2023-10-06T21:21:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fda1400f7b0 con 0x556571cbb9e0 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1400f9b0 con 0x556571cbb9e0 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda3f650b80 10 client.25679 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.636+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556571a22c60 con 0x556571cbb9e0 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556571caef10 con 0x556571cbb9e0 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fda1401abc0 con 0x556571cbb9e0 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 10 client.25679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:21:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 20 client.25679 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571cd6390 0x556571cd87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 4 client.25679 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:53.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.637+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556571c07e10 con 0x556571cd6390 2023-10-06T21:21:53.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.638+0000 7fda29b9b700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571cd6390 0x556571cd87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.639+0000 7fda29b9b700 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571cd6390 0x556571cd87b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=496b0ba57fe8603f server_cookie=f80c6bb78eb9da1d in_seq=0 out_seq=0 2023-10-06T21:21:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.639+0000 7fda0d7fa700 10 client.25679 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda0d7fa700 1 -- 172.21.15.31:0/2548601330 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556571c07e10 con 0x556571cd6390 2023-10-06T21:21:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda0d7fa700 10 client.25679 handle_command_reply: tid=0 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 client.25679 shutdown 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 msgr2=0x556571cc35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fda14011740 tx=0x7fda14011a50 comp rx=0 tx=0).stop 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 shutdown_connections 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571cd6390 0x556571cd87b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cc6890 0x556571cc3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2548601330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571cc35d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda0d7fa700 0 client.25679 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.640+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 >> 172.21.15.31:0/2548601330 conn(0x556571c30260 msgr2=0x556571cb6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.641+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 shutdown_connections 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.641+0000 7fda3f650b80 1 -- 172.21.15.31:0/2548601330 wait complete. 2023-10-06T21:21:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.641+0000 7fda3f650b80 20 client.25679 trim_cache size 0 max 16384 2023-10-06T21:21:53.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 Processor -- start 2023-10-06T21:21:53.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 -- start start 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cbe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571a8f920 con 0x556571cbb9e0 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571cb8280 con 0x556571cc6890 2023-10-06T21:21:53.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.667+0000 7fda3f650b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571cbf420 con 0x556571cbb0c0 2023-10-06T21:21:53.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.668+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.668+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:32852/0 (socket says 172.21.15.31:32852) 2023-10-06T21:21:53.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.668+0000 7fda29b9b700 1 -- 172.21.15.31:0/2268811383 learned_addr learned my addr 172.21.15.31:0/2268811383 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.668+0000 7fda2a39c700 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cbe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.668+0000 7fda2939a700 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 -- 172.21.15.31:0/2268811383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 msgr2=0x556571cc2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 -- 172.21.15.31:0/2268811383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 msgr2=0x556571cc5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 -- 172.21.15.31:0/2268811383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556571a60250 con 0x556571cc6890 2023-10-06T21:21:53.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.669+0000 7fda2a39c700 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cbe8b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda1c007fd0 tx=0x7fda1c00d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346c52dc15b740aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.670+0000 7fda18bff700 1 -- 172.21.15.31:0/2268811383 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c009940 con 0x556571cc6890 2023-10-06T21:21:53.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.670+0000 7fda18bff700 1 -- 172.21.15.31:0/2268811383 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda1c01a470 con 0x556571cc6890 2023-10-06T21:21:53.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.670+0000 7fda18bff700 1 -- 172.21.15.31:0/2268811383 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01a770 con 0x556571cc6890 2023-10-06T21:21:53.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 -- 172.21.15.31:0/2268811383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 msgr2=0x556571cbe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cbe8b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda1c007fd0 tx=0x7fda1c00d000 comp rx=0 tx=0).stop 2023-10-06T21:21:53.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 -- 172.21.15.31:0/2268811383 shutdown_connections 2023-10-06T21:21:53.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cc6890 0x556571cbe8b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cbb0c0 0x556571cc2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 --2- 172.21.15.31:0/2268811383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb9e0 0x556571cc5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 -- 172.21.15.31:0/2268811383 >> 172.21.15.31:0/2268811383 conn(0x556571c30260 msgr2=0x556571cd2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 -- 172.21.15.31:0/2268811383 shutdown_connections 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.671+0000 7fda3f650b80 1 -- 172.21.15.31:0/2268811383 wait complete. 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.672+0000 7fda3f650b80 1 -- start start 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.672+0000 7fda3f650b80 1 Processor -- start 2023-10-06T21:21:53.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571ccb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cc6890 0x556571ccb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571ccbf10 con 0x556571cbb0c0 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571ccc080 con 0x556571cbb9e0 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571ccc240 con 0x556571cc6890 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda3f650b80 10 client.0 _ll_register_callbacks cb 0x7fda14009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36182/0 (socket says 172.21.15.31:36182) 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.673+0000 7fda29b9b700 1 -- 172.21.15.31:0/3736225109 learned_addr learned my addr 172.21.15.31:0/3736225109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda2939a700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571ccb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda2a39c700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cc6890 0x556571ccb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 -- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cc6890 msgr2=0x556571ccb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cc6890 0x556571ccb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 -- 172.21.15.31:0/3736225109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 msgr2=0x556571ccb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571ccb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 -- 172.21.15.31:0/3736225109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c026040 con 0x556571cbb0c0 2023-10-06T21:21:53.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda29b9b700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda2001f480 tx=0x7fda20086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92874fd218f7ea8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.674+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2009be40 con 0x556571cbb0c0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fda20003fb0 con 0x556571cbb0c0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1c00f040 con 0x556571cbb0c0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2009b740 con 0x556571cbb0c0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda3f650b80 10 client.15948 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556571a382b0 con 0x556571cbb0c0 2023-10-06T21:21:53.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.675+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda1c00f040 con 0x556571cbb0c0 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.676+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fda200b93e0 con 0x556571cbb0c0 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 20 client.15948 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571d2b270 0x556571d2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 4 client.15948 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556571c07b30 con 0x556571d2b270 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.677+0000 7fda2939a700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571d2b270 0x556571d2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda2939a700 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571d2b270 0x556571d2d660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5574e04982ca898 server_cookie=7a7290b8052dc308 in_seq=0 out_seq=0 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda0d7fa700 10 client.15948 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda0d7fa700 1 -- 172.21.15.31:0/3736225109 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556571c07b30 con 0x556571d2b270 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda0d7fa700 10 client.15948 handle_command_reply: tid=0 2023-10-06T21:21:53.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda3f650b80 1 client.15948 shutdown 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 msgr2=0x556571ccd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda2001f480 tx=0x7fda20086040 comp rx=0 tx=0).stop 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.678+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 shutdown_connections 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x556571d2b270 0x556571d2d660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556571cc6890 0x556571ccb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556571cbb0c0 0x556571ccd160 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 --2- 172.21.15.31:0/3736225109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556571cbb9e0 0x556571ccb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 >> 172.21.15.31:0/3736225109 conn(0x556571c30260 msgr2=0x556571cbe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda0d7fa700 0 client.15948 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 shutdown_connections 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 1 -- 172.21.15.31:0/3736225109 wait complete. 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.679+0000 7fda3f650b80 20 client.15948 trim_cache size 0 max 16384 2023-10-06T21:21:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 msgr2=0x7fda100a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 0x7fda100a3c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda1c021000 tx=0x7fda1c01f000 comp rx=0 tx=0).stop 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 msgr2=0x7fda241998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda20019040 tx=0x7fda20005ef0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 shutdown_connections 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fda100a1790 0x7fda100a3c50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda24194ad0 0x7fda24194f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda241010b0 0x7fda241998e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 --2- 172.21.15.31:0/1480033506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda241061d0 0x7fda24199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:53.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 >> 172.21.15.31:0/1480033506 conn(0x7fda240fc950 msgr2=0x7fda24104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:53.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.680+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 shutdown_connections 2023-10-06T21:21:53.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:53.681+0000 7fda2bdff700 1 -- 172.21.15.31:0/1480033506 wait complete. 2023-10-06T21:21:53.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:21:54.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:54 smithi031 ceph-mon[98539]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:54.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 -- 172.21.15.31:0/1468245278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b80101a60 msgr2=0x7f9b80105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 --2- 172.21.15.31:0/1468245278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b80101a60 0x7f9b80105aa0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009990 tx=0x7f9b70009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 -- 172.21.15.31:0/1468245278 shutdown_connections 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 --2- 172.21.15.31:0/1468245278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b8010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 --2- 172.21.15.31:0/1468245278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b801010b0 0x7f9b80101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 --2- 172.21.15.31:0/1468245278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b80101a60 0x7f9b80105aa0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.060+0000 7f9b85fff700 1 -- 172.21.15.31:0/1468245278 >> 172.21.15.31:0/1468245278 conn(0x7f9b800fc950 msgr2=0x7f9b800fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.061+0000 7f9b85fff700 1 -- 172.21.15.31:0/1468245278 shutdown_connections 2023-10-06T21:21:54.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.061+0000 7f9b85fff700 1 -- 172.21.15.31:0/1468245278 wait complete. 2023-10-06T21:21:54.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.062+0000 7f9b85fff700 1 Processor -- start 2023-10-06T21:21:54.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.062+0000 7f9b85fff700 1 -- start start 2023-10-06T21:21:54.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b801010b0 0x7f9b80196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 0x7f9b8019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019c320 con 0x7f9b801010b0 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019c490 con 0x7f9b8019b8a0 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.063+0000 7f9b85fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019c650 con 0x7f9b801061d0 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:32870/0 (socket says 172.21.15.31:32870) 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 -- 172.21.15.31:0/2169916908 learned_addr learned my addr 172.21.15.31:0/2169916908 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b801010b0 0x7f9b80196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 0x7f9b8019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 -- 172.21.15.31:0/2169916908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 msgr2=0x7f9b8019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 0x7f9b8019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 -- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b801010b0 msgr2=0x7f9b80196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b801010b0 0x7f9b80196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7effd700 1 -- 172.21.15.31:0/2169916908 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700095f0 con 0x7f9b801061d0 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.064+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 0x7f9b8019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.065+0000 7f9b7effd700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005910 tx=0x7f9b70005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.065+0000 7f9b7cff9700 1 -- 172.21.15.31:0/2169916908 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7001d070 con 0x7f9b801061d0 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.065+0000 7f9b7cff9700 1 -- 172.21.15.31:0/2169916908 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b70022470 con 0x7f9b801061d0 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.065+0000 7f9b7cff9700 1 -- 172.21.15.31:0/2169916908 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70017650 con 0x7f9b801061d0 2023-10-06T21:21:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.066+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b801a3b50 con 0x7f9b801061d0 2023-10-06T21:21:54.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.066+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b801a3f20 con 0x7f9b801061d0 2023-10-06T21:21:54.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.071+0000 7f9b7cff9700 1 -- 172.21.15.31:0/2169916908 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f9b700225e0 con 0x7f9b801061d0 2023-10-06T21:21:54.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.073+0000 7f9b999ffb80 1 Processor -- start 2023-10-06T21:21:54.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.073+0000 7f9b999ffb80 1 -- start start 2023-10-06T21:21:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c30c0 0x5587d43c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 0x5587d43ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b7cff9700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 0x7f9b6c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d432c460 con 0x5587d43c39e0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d431dc20 con 0x5587d43ce890 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d431b540 con 0x5587d43c30c0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.074+0000 7f9b7cff9700 1 -- 172.21.15.31:0/2169916908 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f9b700ce450 con 0x7f9b801061d0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60256/0 (socket says 172.21.15.31:60256) 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1034974938 learned_addr learned my addr 172.21.15.31:0/1034974938 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c30c0 0x5587d43c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7effd700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 0x7f9b6c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 0x5587d43ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1034974938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c30c0 msgr2=0x5587d43c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c30c0 0x5587d43c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1034974938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 msgr2=0x5587d43ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 0x5587d43ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1034974938 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587d4141de0 con 0x5587d43ce890 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.075+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 0x5587d43ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.076+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b78012ed0 tx=0x7f9b78009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1313d42e294f3484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.076+0000 7f9b7effd700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 0x7f9b6c0a3c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b700cba30 tx=0x7f9b70005fb0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.076+0000 7f9b75ffb700 1 -- 172.21.15.31:0/1034974938 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800e040 con 0x5587d43ce890 2023-10-06T21:21:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.076+0000 7f9b75ffb700 1 -- 172.21.15.31:0/1034974938 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b78009310 con 0x5587d43ce890 2023-10-06T21:21:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.076+0000 7f9b75ffb700 1 -- 172.21.15.31:0/1034974938 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78007c20 con 0x5587d43ce890 2023-10-06T21:21:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1034974938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 msgr2=0x5587d43d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b78012ed0 tx=0x7f9b78009920 comp rx=0 tx=0).stop 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1034974938 shutdown_connections 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d43d0c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c30c0 0x5587d43c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1034974938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c39e0 0x5587d43ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.077+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1034974938 >> 172.21.15.31:0/1034974938 conn(0x5587d4338260 msgr2=0x5587d43bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.078+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1034974938 shutdown_connections 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.078+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1034974938 wait complete. 2023-10-06T21:21:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.078+0000 7f9b999ffb80 1 -- start start 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.079+0000 7f9b999ffb80 1 Processor -- start 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d43cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43ce890 0x5587d43cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4316e30 con 0x5587d43ce890 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d42b3710 con 0x5587d43c30c0 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d42b33b0 con 0x5587d43c39e0 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b999ffb80 10 client.0 _ll_register_callbacks cb 0x5587d4110800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60260/0 (socket says 172.21.15.31:60260) 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.080+0000 7f9b7ffff700 1 -- 172.21.15.31:0/3821920586 learned_addr learned my addr 172.21.15.31:0/3821920586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d43cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43ce890 0x5587d43cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 -- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 msgr2=0x5587d43cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d43cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 -- 172.21.15.31:0/3821920586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43ce890 msgr2=0x5587d43cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43ce890 0x5587d43cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.081+0000 7f9b7ffff700 1 -- 172.21.15.31:0/3821920586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b78012bb0 con 0x5587d43c30c0 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.082+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009fd0 tx=0x7f9b78009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ae1e2685c3cd37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.082+0000 7f9b5effd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.082+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800e070 con 0x5587d43c30c0 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.082+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f9b78017e60 con 0x5587d43c30c0 2023-10-06T21:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.082+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587d43b6f10 con 0x5587d43c30c0 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.083+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b78004cd0 con 0x5587d43c30c0 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.083+0000 7f9b999ffb80 10 client.25697 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.083+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5587d412ac60 con 0x5587d43c30c0 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.083+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587d43b6f10 con 0x5587d43c30c0 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9b78014070 con 0x5587d43c30c0 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 10 client.25697 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 10 client.25697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:21:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 20 client.25697 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d43e2680 0x5587d43e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 4 client.25697 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587d430fe10 con 0x5587d43e2680 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.084+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d43e2680 0x5587d43e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.085+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d43e2680 0x5587d43e4aa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21b232fd6d9ad9fe server_cookie=13ee02020dd3f0ac in_seq=0 out_seq=0 2023-10-06T21:21:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.085+0000 7f9b5effd700 10 client.25697 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:54.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b5effd700 1 -- 172.21.15.31:0/3821920586 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5587d430fe10 con 0x5587d43e2680 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b5effd700 10 client.25697 handle_command_reply: tid=0 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b999ffb80 1 client.25697 shutdown 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 msgr2=0x5587d43cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009fd0 tx=0x7f9b78009990 comp rx=0 tx=0).stop 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 shutdown_connections 2023-10-06T21:21:54.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.086+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d43e2680 0x5587d43e4aa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43ce890 0x5587d43cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c30c0 0x5587d43cb730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/3821920586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d43cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 >> 172.21.15.31:0/3821920586 conn(0x5587d4338260 msgr2=0x5587d43befe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b5effd700 0 client.25697 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 shutdown_connections 2023-10-06T21:21:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 1 -- 172.21.15.31:0/3821920586 wait complete. 2023-10-06T21:21:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.087+0000 7f9b999ffb80 20 client.25697 trim_cache size 0 max 16384 2023-10-06T21:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 Processor -- start 2023-10-06T21:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 -- start start 2023-10-06T21:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 0x5587d43c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43ce890 0x5587d43c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4197920 con 0x5587d43c30c0 2023-10-06T21:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d43c0db0 con 0x5587d43c39e0 2023-10-06T21:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.114+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d43b6de0 con 0x5587d43ce890 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.115+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.115+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36248/0 (socket says 172.21.15.31:36248) 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.115+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1036264487 learned_addr learned my addr 172.21.15.31:0/1036264487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.115+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 0x5587d43c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.115+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43ce890 0x5587d43c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 -- 172.21.15.31:0/1036264487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43ce890 msgr2=0x5587d43c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43ce890 0x5587d43c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 -- 172.21.15.31:0/1036264487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 msgr2=0x5587d43d97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 -- 172.21.15.31:0/1036264487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587d43bb140 con 0x5587d43c39e0 2023-10-06T21:21:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 0x5587d43c6030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5587d43e5350 tx=0x7f9b68004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f8eb89a67bc862 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b767fc700 1 -- 172.21.15.31:0/1036264487 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68017040 con 0x5587d43c39e0 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.116+0000 7f9b767fc700 1 -- 172.21.15.31:0/1036264487 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b68020020 con 0x5587d43c39e0 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.117+0000 7f9b767fc700 1 -- 172.21.15.31:0/1036264487 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68003ea0 con 0x5587d43c39e0 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.117+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1036264487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 msgr2=0x5587d43c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.117+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 0x5587d43c6030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5587d43e5350 tx=0x7f9b68004320 comp rx=0 tx=0).stop 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1036264487 shutdown_connections 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43ce890 0x5587d43c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d43d97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1036264487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43c39e0 0x5587d43c6030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1036264487 >> 172.21.15.31:0/1036264487 conn(0x5587d4338260 msgr2=0x5587d43c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1036264487 shutdown_connections 2023-10-06T21:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1036264487 wait complete. 2023-10-06T21:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.118+0000 7f9b999ffb80 1 -- start start 2023-10-06T21:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.119+0000 7f9b999ffb80 1 Processor -- start 2023-10-06T21:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.119+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d442a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.119+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d442ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b999ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d44354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d442b2c0 con 0x5587d43c30c0 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4435a00 con 0x5587d43ce890 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b999ffb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4435bc0 con 0x5587d43c39e0 2023-10-06T21:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b999ffb80 10 client.0 _ll_register_callbacks cb 0x7f9b78012830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d442a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d442a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36258/0 (socket says 172.21.15.31:36258) 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1623576319 learned_addr learned my addr 172.21.15.31:0/1623576319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d44354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.120+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d442ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 msgr2=0x5587d442ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d442ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1623576319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 msgr2=0x5587d442a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d442a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 -- 172.21.15.31:0/1623576319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587d412ac60 con 0x5587d43ce890 2023-10-06T21:21:54.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b7ffff700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d44354c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b78002230 tx=0x7f9b78009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803ec1f5af67008e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b5effd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.121+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800e040 con 0x5587d43ce890 2023-10-06T21:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b78009310 con 0x5587d43ce890 2023-10-06T21:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f9b78017e60 con 0x5587d43ce890 2023-10-06T21:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b78007c20 con 0x5587d43ce890 2023-10-06T21:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b999ffb80 10 client.25703 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9b68009bc0 con 0x5587d43ce890 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.122+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b78009310 con 0x5587d43ce890 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9b78017630 con 0x5587d43ce890 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 10 client.25703 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 10 client.25703 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 20 client.25703 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d4436f80 0x5587d4439370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 4 client.25703 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587d430fb30 con 0x5587d4436f80 2023-10-06T21:21:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.123+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d4436f80 0x5587d4439370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.124+0000 7f9b5effd700 10 client.25703 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.124+0000 7f9b7f7fe700 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d4436f80 0x5587d4439370 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d42881020dc3659 server_cookie=3cf1fccf7252044b in_seq=0 out_seq=0 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.124+0000 7f9b5effd700 1 -- 172.21.15.31:0/1623576319 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587d430fb30 con 0x5587d4436f80 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.124+0000 7f9b5effd700 10 client.25703 handle_command_reply: tid=0 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.124+0000 7f9b999ffb80 1 client.25703 shutdown 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 msgr2=0x5587d44354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d44354c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b78002230 tx=0x7f9b78009990 comp rx=0 tx=0).stop 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 shutdown_connections 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x5587d4436f80 0x5587d4439370 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43ce890 0x5587d44354c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5587d43c30c0 0x5587d442a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b5effd700 0 client.25703 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 --2- 172.21.15.31:0/1623576319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587d43c39e0 0x5587d442ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 >> 172.21.15.31:0/1623576319 conn(0x5587d4338260 msgr2=0x5587d43be660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 shutdown_connections 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.125+0000 7f9b999ffb80 1 -- 172.21.15.31:0/1623576319 wait complete. 2023-10-06T21:21:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.126+0000 7f9b999ffb80 20 client.25703 trim_cache size 0 max 16384 2023-10-06T21:21:54.131 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 msgr2=0x7f9b6c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 0x7f9b6c0a3c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b700cba30 tx=0x7f9b70005fb0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 msgr2=0x7f9b80197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005910 tx=0x7f9b70005740 comp rx=0 tx=0).stop 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 shutdown_connections 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8019b8a0 0x7f9b8019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9b6c0a1790 0x7f9b6c0a3c50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b801010b0 0x7f9b80196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 --2- 172.21.15.31:0/2169916908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b801061d0 0x7f9b80197440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 >> 172.21.15.31:0/2169916908 conn(0x7f9b800fc950 msgr2=0x7f9b800fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 shutdown_connections 2023-10-06T21:21:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.127+0000 7f9b85fff700 1 -- 172.21.15.31:0/2169916908 wait complete. 2023-10-06T21:21:54.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:21:54.308 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:54 smithi146 ceph-mon[102809]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:54.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:54 smithi142 ceph-mon[102951]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:21:54.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 -- 172.21.15.31:0/3583506855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 msgr2=0x7f808810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3583506855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f808810cfb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f807c009990 tx=0x7f807c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 -- 172.21.15.31:0/3583506855 shutdown_connections 2023-10-06T21:21:54.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3583506855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808810d4f0 0x7f808810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3583506855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80880732e0 0x7f80880736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3583506855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f808810cfb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.504+0000 7f808cd9d700 1 -- 172.21.15.31:0/3583506855 >> 172.21.15.31:0/3583506855 conn(0x7f80880fc760 msgr2=0x7f80880feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.505+0000 7f808cd9d700 1 -- 172.21.15.31:0/3583506855 shutdown_connections 2023-10-06T21:21:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.505+0000 7f808cd9d700 1 -- 172.21.15.31:0/3583506855 wait complete. 2023-10-06T21:21:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.506+0000 7f808cd9d700 1 Processor -- start 2023-10-06T21:21:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.506+0000 7f808cd9d700 1 -- start start 2023-10-06T21:21:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f8088106760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 0x7f8088101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088101a40 con 0x7f80880732e0 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088101bb0 con 0x7f808810d4f0 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f808cd9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088101d70 con 0x7f8088073c00 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f80877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f80877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36274/0 (socket says 172.21.15.31:36274) 2023-10-06T21:21:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f80877fe700 1 -- 172.21.15.31:0/3732249129 learned_addr learned my addr 172.21.15.31:0/3732249129 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f8087fff700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 0x7f8088101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.507+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f8088106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 -- 172.21.15.31:0/3732249129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 msgr2=0x7f8088106760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f8088106760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 -- 172.21.15.31:0/3732249129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 msgr2=0x7f8088101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 0x7f8088101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 -- 172.21.15.31:0/3732249129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0095f0 con 0x7f80880732e0 2023-10-06T21:21:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f8087fff700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 0x7f8088101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.508+0000 7f80877fe700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f807800e9a0 tx=0x7f807800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.509+0000 7f8084ff9700 1 -- 172.21.15.31:0/3732249129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078009790 con 0x7f80880732e0 2023-10-06T21:21:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.509+0000 7f8084ff9700 1 -- 172.21.15.31:0/3732249129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8078010430 con 0x7f80880732e0 2023-10-06T21:21:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.509+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8088102050 con 0x7f80880732e0 2023-10-06T21:21:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.509+0000 7f8084ff9700 1 -- 172.21.15.31:0/3732249129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800cea0 con 0x7f80880732e0 2023-10-06T21:21:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.509+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808810c270 con 0x7f80880732e0 2023-10-06T21:21:54.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.515+0000 7f8084ff9700 1 -- 172.21.15.31:0/3732249129 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f80780105a0 con 0x7f80880732e0 2023-10-06T21:21:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 Processor -- start 2023-10-06T21:21:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 -- start start 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f87692530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87692a70 0x564f87694e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f875f0460 con 0x564f876872a0 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f875e1c20 con 0x564f87687bc0 2023-10-06T21:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.516+0000 7f80a1903b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f875df540 con 0x564f87692a70 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.517+0000 7f8087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.517+0000 7f8087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36284/0 (socket says 172.21.15.31:36284) 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.517+0000 7f8087fff700 1 -- 172.21.15.31:0/573163802 learned_addr learned my addr 172.21.15.31:0/573163802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.517+0000 7f80877fe700 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f87692530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8086ffd700 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87692a70 0x564f87694e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8084ff9700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 0x7f80700a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8084ff9700 1 -- 172.21.15.31:0/3732249129 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8078014070 con 0x7f80880732e0 2023-10-06T21:21:54.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 -- 172.21.15.31:0/573163802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87692a70 msgr2=0x564f87694e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87692a70 0x564f87694e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 -- 172.21.15.31:0/573163802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 msgr2=0x564f87692530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f87692530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 -- 172.21.15.31:0/573163802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f87405de0 con 0x564f876872a0 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.518+0000 7f8087fff700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 0x7f80700a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.519+0000 7f8087fff700 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8080003ef0 tx=0x7f8080004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d748c9ba7c2983 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.519+0000 7f806dffb700 1 -- 172.21.15.31:0/573163802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800090b0 con 0x564f876872a0 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.519+0000 7f806dffb700 1 -- 172.21.15.31:0/573163802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f808000f040 con 0x564f876872a0 2023-10-06T21:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.519+0000 7f806dffb700 1 -- 172.21.15.31:0/573163802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080018800 con 0x564f876872a0 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 -- 172.21.15.31:0/573163802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 msgr2=0x564f87687680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8080003ef0 tx=0x7f8080004320 comp rx=0 tx=0).stop 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f8087fff700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 0x7f80700a3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8080022000 tx=0x7f808001f000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 -- 172.21.15.31:0/573163802 shutdown_connections 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87692a70 0x564f87694e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f876872a0 0x564f87687680 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 --2- 172.21.15.31:0/573163802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f87692530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.520+0000 7f80a1903b80 1 -- 172.21.15.31:0/573163802 >> 172.21.15.31:0/573163802 conn(0x564f875fc260 msgr2=0x564f87683630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.521+0000 7f80a1903b80 1 -- 172.21.15.31:0/573163802 shutdown_connections 2023-10-06T21:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.521+0000 7f80a1903b80 1 -- 172.21.15.31:0/573163802 wait complete. 2023-10-06T21:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.521+0000 7f80a1903b80 1 -- start start 2023-10-06T21:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.522+0000 7f80a1903b80 1 Processor -- start 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 0x564f876902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f875dae30 con 0x564f87687bc0 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f87577710 con 0x564f87692a70 2023-10-06T21:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f875773b0 con 0x564f876872a0 2023-10-06T21:21:54.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80a1903b80 10 client.0 _ll_register_callbacks cb 0x564f873d4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:54.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f8086ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f8086ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36294/0 (socket says 172.21.15.31:36294) 2023-10-06T21:21:54.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f80877fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f8086ffd700 1 -- 172.21.15.31:0/3655587802 learned_addr learned my addr 172.21.15.31:0/3655587802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.523+0000 7f8087fff700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 0x564f876902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 -- 172.21.15.31:0/3655587802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 msgr2=0x564f8768f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 -- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 msgr2=0x564f876902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 0x564f876902c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 -- 172.21.15.31:0/3655587802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8080003bd0 con 0x564f87687bc0 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8087fff700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 0x564f876902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f80877fe700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f807c009990 tx=0x7f807c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36abf3adcbfd7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.524+0000 7f8066ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c004500 con 0x564f87687bc0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f807c004660 con 0x564f87687bc0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f8767b160 con 0x564f87687bc0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807c004820 con 0x564f87687bc0 2023-10-06T21:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f80a1903b80 10 client.15975 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.525+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f873eec60 con 0x564f87687bc0 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f8767b160 con 0x564f87687bc0 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f807c00f790 con 0x564f87687bc0 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 20 client.15975 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f876a2690 0x564f876a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 4 client.15975 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.526+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f875d3e10 con 0x564f876a2690 2023-10-06T21:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.527+0000 7f80877fe700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f876a2690 0x564f876a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.528+0000 7f80877fe700 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f876a2690 0x564f876a4ab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72695ef8fe39e60 server_cookie=e30fe1fc433f01e3 in_seq=0 out_seq=0 2023-10-06T21:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.528+0000 7f8066ffd700 10 client.15975 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.528+0000 7f8066ffd700 1 -- 172.21.15.31:0/3655587802 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f875d3e10 con 0x564f876a2690 2023-10-06T21:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.528+0000 7f8066ffd700 10 client.15975 handle_command_reply: tid=0 2023-10-06T21:21:54.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 client.15975 shutdown 2023-10-06T21:21:54.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 msgr2=0x564f8768fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f807c009990 tx=0x7f807c005750 comp rx=0 tx=0).stop 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 shutdown_connections 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87692a70 0x564f876902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f876a2690 0x564f876a4ab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3655587802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87687bc0 0x564f8768fd80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 >> 172.21.15.31:0/3655587802 conn(0x564f875fc260 msgr2=0x564f87683220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f8066ffd700 0 client.15975 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 shutdown_connections 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.529+0000 7f80a1903b80 1 -- 172.21.15.31:0/3655587802 wait complete. 2023-10-06T21:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.530+0000 7f80a1903b80 20 client.15975 trim_cache size 0 max 16384 2023-10-06T21:21:54.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.556+0000 7f80a1903b80 1 Processor -- start 2023-10-06T21:21:54.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.556+0000 7f80a1903b80 1 -- start start 2023-10-06T21:21:54.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f876899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f8768cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8745b920 con 0x564f87692a70 2023-10-06T21:21:54.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f87688d60 con 0x564f87687bc0 2023-10-06T21:21:54.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80a1903b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8767aca0 con 0x564f876872a0 2023-10-06T21:21:54.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f8087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f8768cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f8087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f8768cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36304/0 (socket says 172.21.15.31:36304) 2023-10-06T21:21:54.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f8087fff700 1 -- 172.21.15.31:0/1402749370 learned_addr learned my addr 172.21.15.31:0/1402749370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.557+0000 7f80877fe700 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f8086ffd700 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f876899b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 -- 172.21.15.31:0/1402749370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 msgr2=0x564f876899b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f876899b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 -- 172.21.15.31:0/1402749370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 msgr2=0x564f8768cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f8768cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 -- 172.21.15.31:0/1402749370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f8742c250 con 0x564f876872a0 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.558+0000 7f80877fe700 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f80780caad0 tx=0x7f80780965d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc24e583c420b8ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.559+0000 7f806e7fc700 1 -- 172.21.15.31:0/1402749370 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078094ea0 con 0x564f876872a0 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.559+0000 7f806e7fc700 1 -- 172.21.15.31:0/1402749370 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8078099bf0 con 0x564f876872a0 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.559+0000 7f806e7fc700 1 -- 172.21.15.31:0/1402749370 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780a2620 con 0x564f876872a0 2023-10-06T21:21:54.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 -- 172.21.15.31:0/1402749370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 msgr2=0x564f8768f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f80780caad0 tx=0x7f80780965d0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 -- 172.21.15.31:0/1402749370 shutdown_connections 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f8768cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f876872a0 0x564f8768f000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 --2- 172.21.15.31:0/1402749370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f87687bc0 0x564f876899b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 -- 172.21.15.31:0/1402749370 >> 172.21.15.31:0/1402749370 conn(0x564f875fc260 msgr2=0x564f876892a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 -- 172.21.15.31:0/1402749370 shutdown_connections 2023-10-06T21:21:54.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.560+0000 7f80a1903b80 1 -- 172.21.15.31:0/1402749370 wait complete. 2023-10-06T21:21:54.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.561+0000 7f80a1903b80 1 -- start start 2023-10-06T21:21:54.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.561+0000 7f80a1903b80 1 Processor -- start 2023-10-06T21:21:54.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f876872a0 0x564f87699050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f876974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f87697bf0 con 0x564f87692a70 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f87697d60 con 0x564f876872a0 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f87697ea0 con 0x564f87687bc0 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f80a1903b80 10 client.0 _ll_register_callbacks cb 0x7f807c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f8086ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f8086ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:32948/0 (socket says 172.21.15.31:32948) 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.562+0000 7f8086ffd700 1 -- 172.21.15.31:0/3092284933 learned_addr learned my addr 172.21.15.31:0/3092284933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8087fff700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f876974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 -- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f876872a0 msgr2=0x564f87699050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f876872a0 0x564f87699050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 -- 172.21.15.31:0/3092284933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 msgr2=0x564f876974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:54.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f876974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 -- 172.21.15.31:0/3092284933 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808810c270 con 0x564f87687bc0 2023-10-06T21:21:54.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8086ffd700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x564f8767d760 tx=0x7f807c00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a42a6b75294848b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:54.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.563+0000 7f8066ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:21:54.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f870 con 0x564f87687bc0 2023-10-06T21:21:54.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f807c023550 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f808000f040 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807c005910 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f80a1903b80 10 client.25678 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.564+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f874042b0 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f807c043070 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f807c004790 con 0x564f87687bc0 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:54.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 20 client.25678 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:54.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f877083b0 0x564f8770a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:54.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 4 client.25678 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:21:54.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.565+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f875d3b30 con 0x564f877083b0 2023-10-06T21:21:54.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.566+0000 7f80877fe700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f877083b0 0x564f8770a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:54.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.567+0000 7f80877fe700 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f877083b0 0x564f8770a7d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694ef0f0561e29a3 server_cookie=9e1f8effdb5112a in_seq=0 out_seq=0 2023-10-06T21:21:54.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.567+0000 7f8066ffd700 10 client.25678 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:54.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.567+0000 7f8066ffd700 1 -- 172.21.15.31:0/3092284933 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f875d3b30 con 0x564f877083b0 2023-10-06T21:21:54.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.567+0000 7f8066ffd700 10 client.25678 handle_command_reply: tid=0 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 client.25678 shutdown 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 msgr2=0x564f87699590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x564f8767d760 tx=0x7f807c00b9f0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 shutdown_connections 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f877083b0 0x564f8770a7d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f87692a70 0x564f876974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f876872a0 0x564f87699050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 --2- 172.21.15.31:0/3092284933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f87687bc0 0x564f87699590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 >> 172.21.15.31:0/3092284933 conn(0x564f875fc260 msgr2=0x564f87689b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f8066ffd700 0 client.25678 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:54.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 shutdown_connections 2023-10-06T21:21:54.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.568+0000 7f80a1903b80 1 -- 172.21.15.31:0/3092284933 wait complete. 2023-10-06T21:21:54.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.569+0000 7f80a1903b80 20 client.25678 trim_cache size 0 max 16384 2023-10-06T21:21:54.574 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:21:54.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 msgr2=0x7f80700a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 0x7f80700a3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8080022000 tx=0x7f808001f000 comp rx=0 tx=0).stop 2023-10-06T21:21:54.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 msgr2=0x7f8088106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:54.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f807800e9a0 tx=0x7f807800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 shutdown_connections 2023-10-06T21:21:54.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808810d4f0 0x7f8088101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f80700a1830 0x7f80700a3cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80880732e0 0x7f8088106220 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 --2- 172.21.15.31:0/3732249129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8088073c00 0x7f8088106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:54.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 >> 172.21.15.31:0/3732249129 conn(0x7f80880fc760 msgr2=0x7f8088198360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:54.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 shutdown_connections 2023-10-06T21:21:54.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:54.570+0000 7f808cd9d700 1 -- 172.21.15.31:0/3732249129 wait complete. 2023-10-06T21:21:55.601 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:21:55.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:21:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.954+0000 7eff6a1ff700 1 -- 172.21.15.31:0/3959583798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 msgr2=0x7eff64101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.954+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/3959583798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff64101490 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff54009990 tx=0x7eff54009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.954+0000 7eff6a1ff700 1 -- 172.21.15.31:0/3959583798 shutdown_connections 2023-10-06T21:21:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.954+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/3959583798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff6410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.955+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/3959583798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff64101490 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.955+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/3959583798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff64101a60 0x7eff64105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.955+0000 7eff6a1ff700 1 -- 172.21.15.31:0/3959583798 >> 172.21.15.31:0/3959583798 conn(0x7eff640fc930 msgr2=0x7eff640fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.955+0000 7eff6a1ff700 1 -- 172.21.15.31:0/3959583798 shutdown_connections 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.955+0000 7eff6a1ff700 1 -- 172.21.15.31:0/3959583798 wait complete. 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.956+0000 7eff6a1ff700 1 Processor -- start 2023-10-06T21:21:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.956+0000 7eff6a1ff700 1 -- start start 2023-10-06T21:21:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff641990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff64199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6419da70 0x7eff6419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419e4f0 con 0x7eff641010b0 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419e660 con 0x7eff6419da70 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff6a1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419e820 con 0x7eff641061d0 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.957+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff641990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff641990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36322/0 (socket says 172.21.15.31:36322) 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff637fe700 1 -- 172.21.15.31:0/1640984566 learned_addr learned my addr 172.21.15.31:0/1640984566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff64199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff63fff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6419da70 0x7eff6419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 -- 172.21.15.31:0/1640984566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6419da70 msgr2=0x7eff6419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:55.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6419da70 0x7eff6419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 -- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 msgr2=0x7eff641990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:55.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff641990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.958+0000 7eff62ffd700 1 -- 172.21.15.31:0/1640984566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540095f0 con 0x7eff641061d0 2023-10-06T21:21:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff64199610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e9c0 tx=0x7eff5800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff60ff9700 1 -- 172.21.15.31:0/1640984566 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800c9c0 con 0x7eff641061d0 2023-10-06T21:21:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff641a5d80 con 0x7eff641061d0 2023-10-06T21:21:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff60ff9700 1 -- 172.21.15.31:0/1640984566 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff58010430 con 0x7eff641061d0 2023-10-06T21:21:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff60ff9700 1 -- 172.21.15.31:0/1640984566 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58018680 con 0x7eff641061d0 2023-10-06T21:21:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.959+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a6250 con 0x7eff641061d0 2023-10-06T21:21:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.965+0000 7eff60ff9700 1 -- 172.21.15.31:0/1640984566 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7eff580105a0 con 0x7eff641061d0 2023-10-06T21:21:55.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.966+0000 7eff7db45b80 1 Processor -- start 2023-10-06T21:21:55.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.966+0000 7eff7db45b80 1 -- start start 2023-10-06T21:21:55.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0da3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0384e0 con 0x5556fe0cf140 2023-10-06T21:21:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe029ca0 con 0x5556fe0cfa60 2023-10-06T21:21:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff7db45b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0275c0 con 0x5556fe0da910 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff60ff9700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 0x7eff4c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff63fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:32970/0 (socket says 172.21.15.31:32970) 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 -- 172.21.15.31:0/3701069002 learned_addr learned my addr 172.21.15.31:0/3701069002 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.967+0000 7eff637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0da3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff60ff9700 1 -- 172.21.15.31:0/1640984566 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7eff58014070 con 0x7eff641061d0 2023-10-06T21:21:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff63fff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 0x7eff4c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 -- 172.21.15.31:0/3701069002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 msgr2=0x5556fe0da3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0da3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 -- 172.21.15.31:0/3701069002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 msgr2=0x5556fe0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.968+0000 7eff62ffd700 1 -- 172.21.15.31:0/3701069002 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556fde4de60 con 0x5556fe0da910 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.969+0000 7eff62ffd700 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff580caa80 tx=0x7eff58095ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c0f5b3af8591e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.969+0000 7eff49ffb700 1 -- 172.21.15.31:0/3701069002 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5809e0d0 con 0x5556fe0da910 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.969+0000 7eff63fff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 0x7eff4c0a3c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009ec0 tx=0x7eff5c004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.969+0000 7eff49ffb700 1 -- 172.21.15.31:0/3701069002 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff5809a8b0 con 0x5556fe0da910 2023-10-06T21:21:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.969+0000 7eff49ffb700 1 -- 172.21.15.31:0/3701069002 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5809a410 con 0x5556fe0da910 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.970+0000 7eff7db45b80 1 -- 172.21.15.31:0/3701069002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 msgr2=0x5556fe0dcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.970+0000 7eff7db45b80 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff580caa80 tx=0x7eff58095ef0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 -- 172.21.15.31:0/3701069002 shutdown_connections 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0dcd00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 --2- 172.21.15.31:0/3701069002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0da3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 -- 172.21.15.31:0/3701069002 >> 172.21.15.31:0/3701069002 conn(0x5556fe0442e0 msgr2=0x5556fe0cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 -- 172.21.15.31:0/3701069002 shutdown_connections 2023-10-06T21:21:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 -- 172.21.15.31:0/3701069002 wait complete. 2023-10-06T21:21:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.971+0000 7eff7db45b80 1 -- start start 2023-10-06T21:21:55.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.972+0000 7eff7db45b80 1 Processor -- start 2023-10-06T21:21:55.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe022eb0 con 0x5556fe0cf140 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fdfbf790 con 0x5556fe0cfa60 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fdfbf430 con 0x5556fe0da910 2023-10-06T21:21:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff7db45b80 10 client.0 _ll_register_callbacks cb 0x5556fde1c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36350/0 (socket says 172.21.15.31:36350) 2023-10-06T21:21:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.973+0000 7eff637fe700 1 -- 172.21.15.31:0/1502892356 learned_addr learned my addr 172.21.15.31:0/1502892356 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff63fff700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 -- 172.21.15.31:0/1502892356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 msgr2=0x5556fe0d81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0d81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 -- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 msgr2=0x5556fe0d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 -- 172.21.15.31:0/1502892356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a6250 con 0x5556fe0cf140 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff63fff700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff637fe700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7eff54005830 tx=0x7eff5400b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519bd30fd40d8cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.974+0000 7eff42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5401d070 con 0x5556fe0cf140 2023-10-06T21:21:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556fe0c2f90 con 0x5556fe0cf140 2023-10-06T21:21:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7eff5400be80 con 0x5556fe0cf140 2023-10-06T21:21:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff54004700 con 0x5556fe0cf140 2023-10-06T21:21:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff7db45b80 10 client.15993 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.975+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556fde36ce0 con 0x5556fe0cf140 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.976+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5556fe0c2f90 con 0x5556fe0cf140 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff54004860 con 0x5556fe0cf140 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 20 client.15993 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe0f2b80 0x5556fe0f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 4 client.15993 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556fe01be90 con 0x5556fe0f2b80 2023-10-06T21:21:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.977+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe0f2b80 0x5556fe0f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:55.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.978+0000 7eff62ffd700 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe0f2b80 0x5556fe0f4fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2570c60787efbe5f server_cookie=9f30c2f677deed96 in_seq=0 out_seq=0 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.978+0000 7eff42ffd700 10 client.15993 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff42ffd700 1 -- 172.21.15.31:0/1502892356 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556fe01be90 con 0x5556fe0f2b80 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff42ffd700 10 client.15993 handle_command_reply: tid=0 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 client.15993 shutdown 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 msgr2=0x5556fe0d7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7eff54005830 tx=0x7eff5400b6b0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 shutdown_connections 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe0f2b80 0x5556fe0f4fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe0d7730 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 --2- 172.21.15.31:0/1502892356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe0d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 >> 172.21.15.31:0/1502892356 conn(0x5556fe0442e0 msgr2=0x5556fe0cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.979+0000 7eff42ffd700 0 client.15993 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.980+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 shutdown_connections 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.980+0000 7eff7db45b80 1 -- 172.21.15.31:0/1502892356 wait complete. 2023-10-06T21:21:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:55.980+0000 7eff7db45b80 20 client.15993 trim_cache size 0 max 16384 2023-10-06T21:21:56.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:56 smithi031 ceph-mon[98539]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T21:21:56.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.014+0000 7eff7db45b80 1 Processor -- start 2023-10-06T21:21:56.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 -- start start 2023-10-06T21:21:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0cf140 0x5556fe0d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cfa60 0x5556fe0d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fdea39a0 con 0x5556fe0cfa60 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0c2ec0 con 0x5556fe0da910 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.015+0000 7eff7db45b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0f29e0 con 0x5556fe0cf140 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.016+0000 7eff637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0cf140 0x5556fe0d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.016+0000 7eff63fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.016+0000 7eff63fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60382/0 (socket says 172.21.15.31:60382) 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.016+0000 7eff63fff700 1 -- 172.21.15.31:0/2890943877 learned_addr learned my addr 172.21.15.31:0/2890943877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.016+0000 7eff62ffd700 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cfa60 0x5556fe0d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 -- 172.21.15.31:0/2890943877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0cf140 msgr2=0x5556fe0d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0cf140 0x5556fe0d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 -- 172.21.15.31:0/2890943877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cfa60 msgr2=0x5556fe0d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cfa60 0x5556fe0d49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 -- 172.21.15.31:0/2890943877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556fe0f9bf0 con 0x5556fe0da910 2023-10-06T21:21:56.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.017+0000 7eff63fff700 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0093e0 tx=0x7eff5c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9fbe8558ad2d38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.018+0000 7eff4a7fc700 1 -- 172.21.15.31:0/2890943877 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00e040 con 0x5556fe0da910 2023-10-06T21:21:56.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.018+0000 7eff4a7fc700 1 -- 172.21.15.31:0/2890943877 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff5c00f040 con 0x5556fe0da910 2023-10-06T21:21:56.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.018+0000 7eff4a7fc700 1 -- 172.21.15.31:0/2890943877 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01a9a0 con 0x5556fe0da910 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- 172.21.15.31:0/2890943877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 msgr2=0x5556fe0d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0093e0 tx=0x7eff5c009920 comp rx=0 tx=0).stop 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- 172.21.15.31:0/2890943877 shutdown_connections 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0da910 0x5556fe0d4ee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0cf140 0x5556fe0d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2890943877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cfa60 0x5556fe0d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- 172.21.15.31:0/2890943877 >> 172.21.15.31:0/2890943877 conn(0x5556fe0442e0 msgr2=0x5556fe0d6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- 172.21.15.31:0/2890943877 shutdown_connections 2023-10-06T21:21:56.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- 172.21.15.31:0/2890943877 wait complete. 2023-10-06T21:21:56.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.019+0000 7eff7db45b80 1 -- start start 2023-10-06T21:21:56.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.020+0000 7eff7db45b80 1 Processor -- start 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0e9290 con 0x5556fe0cf140 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0e9400 con 0x5556fe0cfa60 2023-10-06T21:21:56.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fe0e95c0 con 0x5556fe0da910 2023-10-06T21:21:56.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff7db45b80 10 client.0 _ll_register_callbacks cb 0x7eff54005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:56.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36366/0 (socket says 172.21.15.31:36366) 2023-10-06T21:21:56.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff637fe700 1 -- 172.21.15.31:0/2350875208 learned_addr learned my addr 172.21.15.31:0/2350875208 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff63fff700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.021+0000 7eff62ffd700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 -- 172.21.15.31:0/2350875208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 msgr2=0x5556fe0e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 -- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 msgr2=0x5556fe1402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe1402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 -- 172.21.15.31:0/2350875208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c009120 con 0x5556fe0cf140 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff63fff700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff637fe700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7eff54005800 tx=0x7eff5400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb0118a17e17044 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.022+0000 7eff42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5401d070 con 0x5556fe0cf140 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7eff5403d020 con 0x5556fe0cf140 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5809a8b0 con 0x5556fe0cf140 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff54004eb0 con 0x5556fe0cf140 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff7db45b80 10 client.15999 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:56.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.023+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556fde742d0 con 0x5556fe0cf140 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.024+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff5809a8b0 con 0x5556fe0cf140 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.024+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff540244e0 con 0x5556fe0cf140 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 20 client.15999 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe148a60 0x5556fe14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 4 client.15999 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:21:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5556fe01bbb0 con 0x5556fe148a60 2023-10-06T21:21:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.025+0000 7eff62ffd700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe148a60 0x5556fe14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.026+0000 7eff62ffd700 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe148a60 0x5556fe14ae80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a323a038e34bee5c server_cookie=8c543ed02c80f39c in_seq=0 out_seq=0 2023-10-06T21:21:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.026+0000 7eff42ffd700 10 client.15999 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.027+0000 7eff42ffd700 1 -- 172.21.15.31:0/2350875208 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5556fe01bbb0 con 0x5556fe148a60 2023-10-06T21:21:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.027+0000 7eff42ffd700 10 client.15999 handle_command_reply: tid=0 2023-10-06T21:21:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.027+0000 7eff7db45b80 1 client.15999 shutdown 2023-10-06T21:21:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 msgr2=0x5556fe13fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7eff54005800 tx=0x7eff5400be80 comp rx=0 tx=0).stop 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 shutdown_connections 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556fe0da910 0x5556fe0e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5556fe148a60 0x5556fe14ae80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556fe0cf140 0x5556fe13fdb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 --2- 172.21.15.31:0/2350875208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556fe0cfa60 0x5556fe1402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 >> 172.21.15.31:0/2350875208 conn(0x5556fe0442e0 msgr2=0x5556fe0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff42ffd700 0 client.15999 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.028+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 shutdown_connections 2023-10-06T21:21:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.029+0000 7eff7db45b80 1 -- 172.21.15.31:0/2350875208 wait complete. 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.029+0000 7eff7db45b80 20 client.15999 trim_cache size 0 max 16384 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "50222a11-4e6f-4db7-8464-1f34d0a2fb5a", 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-10-06T21:21:56.033 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 msgr2=0x7eff4c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 0x7eff4c0a3c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009ec0 tx=0x7eff5c004b40 comp rx=0 tx=0).stop 2023-10-06T21:21:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 msgr2=0x7eff64199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff64199610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e9c0 tx=0x7eff5800ecd0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 shutdown_connections 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6419da70 0x7eff6419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff4c0a1790 0x7eff4c0a3c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff641010b0 0x7eff641990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 --2- 172.21.15.31:0/1640984566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff641061d0 0x7eff64199610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 >> 172.21.15.31:0/1640984566 conn(0x7eff640fc930 msgr2=0x7eff640fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 shutdown_connections 2023-10-06T21:21:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.030+0000 7eff6a1ff700 1 -- 172.21.15.31:0/1640984566 wait complete. 2023-10-06T21:21:56.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:56 smithi146 ceph-mon[102809]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T21:21:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:56 smithi142 ceph-mon[102951]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T21:21:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 -- 172.21.15.31:0/2228452913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 msgr2=0x7f25101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 --2- 172.21.15.31:0/2228452913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25101023d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 -- 172.21.15.31:0/2228452913 shutdown_connections 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 --2- 172.21.15.31:0/2228452913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 0x7f251010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 --2- 172.21.15.31:0/2228452913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25101023d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 --2- 172.21.15.31:0/2228452913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 0x7f251010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.408+0000 7f2515bff700 1 -- 172.21.15.31:0/2228452913 >> 172.21.15.31:0/2228452913 conn(0x7f25100fb830 msgr2=0x7f25100fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.409+0000 7f2515bff700 1 -- 172.21.15.31:0/2228452913 shutdown_connections 2023-10-06T21:21:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.409+0000 7f2515bff700 1 -- 172.21.15.31:0/2228452913 wait complete. 2023-10-06T21:21:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.410+0000 7f2515bff700 1 Processor -- start 2023-10-06T21:21:56.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 -- start start 2023-10-06T21:21:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 0x7f25101001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 0x7f2510071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101009f0 con 0x7f2510101ff0 2023-10-06T21:21:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100718a0 con 0x7f2510102910 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.411+0000 7f2515bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2510071a60 con 0x7f251010b410 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36384/0 (socket says 172.21.15.31:36384) 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 -- 172.21.15.31:0/3358255057 learned_addr learned my addr 172.21.15.31:0/3358255057 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250ffff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 0x7f2510071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250effd700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 0x7f25101001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 -- 172.21.15.31:0/3358255057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 msgr2=0x7f2510071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 0x7f2510071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 -- 172.21.15.31:0/3358255057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 msgr2=0x7f25101001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 0x7f25101001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 -- 172.21.15.31:0/3358255057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000095f0 con 0x7f2510101ff0 2023-10-06T21:21:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.412+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2500005740 tx=0x7f2500005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.413+0000 7f250cff9700 1 -- 172.21.15.31:0/3358255057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250001d070 con 0x7f2510101ff0 2023-10-06T21:21:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.413+0000 7f250cff9700 1 -- 172.21.15.31:0/3358255057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2500022470 con 0x7f2510101ff0 2023-10-06T21:21:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.413+0000 7f250cff9700 1 -- 172.21.15.31:0/3358255057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500017650 con 0x7f2510101ff0 2023-10-06T21:21:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.413+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2510071ce0 con 0x7f2510101ff0 2023-10-06T21:21:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.414+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2510072140 con 0x7f2510101ff0 2023-10-06T21:21:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.419+0000 7f250cff9700 1 -- 172.21.15.31:0/3358255057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f25000225e0 con 0x7f2510101ff0 2023-10-06T21:21:56.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.420+0000 7f2529619b80 1 Processor -- start 2023-10-06T21:21:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 -- start start 2023-10-06T21:21:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e68ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff9e0 0x5619e690a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e6868460 con 0x5619e68ff0c0 2023-10-06T21:21:56.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e6859c20 con 0x5619e68ff9e0 2023-10-06T21:21:56.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.421+0000 7f2529619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e6857540 con 0x5619e690a890 2023-10-06T21:21:56.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:33036/0 (socket says 172.21.15.31:33036) 2023-10-06T21:21:56.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 -- 172.21.15.31:0/3887535240 learned_addr learned my addr 172.21.15.31:0/3887535240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 -- 172.21.15.31:0/3887535240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff9e0 msgr2=0x5619e690a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff9e0 0x5619e690a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 -- 172.21.15.31:0/3887535240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 msgr2=0x5619e68ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250cff9700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 0x7f24f80a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e68ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.422+0000 7f250effd700 1 -- 172.21.15.31:0/3887535240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e667dde0 con 0x5619e690a890 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f250cff9700 1 -- 172.21.15.31:0/3358255057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f25000ce900 con 0x7f2510101ff0 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f250effd700 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2504009d10 tx=0x7f2504011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8490469c260a0ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f24f5ffb700 1 -- 172.21.15.31:0/3887535240 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504004a30 con 0x5619e690a890 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f24f5ffb700 1 -- 172.21.15.31:0/3887535240 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25040054b0 con 0x5619e690a890 2023-10-06T21:21:56.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f24f5ffb700 1 -- 172.21.15.31:0/3887535240 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504019690 con 0x5619e690a890 2023-10-06T21:21:56.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.423+0000 7f250ffff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 0x7f24f80a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 -- 172.21.15.31:0/3887535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 msgr2=0x5619e690cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2504009d10 tx=0x7f2504011720 comp rx=0 tx=0).stop 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 -- 172.21.15.31:0/3887535240 shutdown_connections 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e690cc80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e68ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 --2- 172.21.15.31:0/3887535240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff9e0 0x5619e690a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.424+0000 7f2529619b80 1 -- 172.21.15.31:0/3887535240 >> 172.21.15.31:0/3887535240 conn(0x5619e6874260 msgr2=0x5619e68fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.425+0000 7f250ffff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 0x7f24f80a3cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2508009550 tx=0x7f25080093b0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.425+0000 7f2529619b80 1 -- 172.21.15.31:0/3887535240 shutdown_connections 2023-10-06T21:21:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.425+0000 7f2529619b80 1 -- 172.21.15.31:0/3887535240 wait complete. 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.425+0000 7f2529619b80 1 -- start start 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.426+0000 7f2529619b80 1 Processor -- start 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.426+0000 7f2529619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e69080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e6852e30 con 0x5619e68ff9e0 2023-10-06T21:21:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e67ef710 con 0x5619e68ff0c0 2023-10-06T21:21:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e67ef3b0 con 0x5619e690a890 2023-10-06T21:21:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f2529619b80 10 client.0 _ll_register_callbacks cb 0x5619e664c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60420/0 (socket says 172.21.15.31:60420) 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 -- 172.21.15.31:0/2642835096 learned_addr learned my addr 172.21.15.31:0/2642835096 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250ffff700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e69080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 -- 172.21.15.31:0/2642835096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 msgr2=0x5619e69080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e69080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.427+0000 7f250f7fe700 1 -- 172.21.15.31:0/2642835096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 msgr2=0x5619e6907b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.428+0000 7f250f7fe700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.428+0000 7f250f7fe700 1 -- 172.21.15.31:0/2642835096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2504009970 con 0x5619e68ff0c0 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.428+0000 7f250f7fe700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f250009b720 tx=0x7f250009bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789acc2d133bcff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.428+0000 7f24eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:21:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000a3800 con 0x5619e68ff0c0 2023-10-06T21:21:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619e68f2f10 con 0x5619e68ff0c0 2023-10-06T21:21:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f250009be90 con 0x5619e68ff0c0 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250009b0f0 con 0x5619e68ff0c0 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f2529619b80 10 client.25745 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619e6666c60 con 0x5619e68ff0c0 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.429+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5619e68f2f10 con 0x5619e68ff0c0 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25000b7070 con 0x5619e68ff0c0 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 10 client.25745 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 20 client.25745 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e691a020 0x5619e691c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 4 client.25745 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619e684be10 con 0x5619e691a020 2023-10-06T21:21:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.430+0000 7f250effd700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e691a020 0x5619e691c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.431+0000 7f250effd700 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e691a020 0x5619e691c440 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4732a9d6018ee97d server_cookie=30d868565bf321e8 in_seq=0 out_seq=0 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.431+0000 7f24eeffd700 10 client.25745 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f24eeffd700 1 -- 172.21.15.31:0/2642835096 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619e684be10 con 0x5619e691a020 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f24eeffd700 10 client.25745 handle_command_reply: tid=0 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 client.25745 shutdown 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 msgr2=0x5619e6907630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f250009b720 tx=0x7f250009bcd0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 shutdown_connections 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e690a890 0x5619e69080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e68ff0c0 0x5619e6907630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.432+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e691a020 0x5619e691c440 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f2529619b80 1 --2- 172.21.15.31:0/2642835096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 >> 172.21.15.31:0/2642835096 conn(0x5619e6874260 msgr2=0x5619e68fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f24eeffd700 0 client.25745 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 shutdown_connections 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f2529619b80 1 -- 172.21.15.31:0/2642835096 wait complete. 2023-10-06T21:21:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.433+0000 7f2529619b80 20 client.25745 trim_cache size 0 max 16384 2023-10-06T21:21:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.467+0000 7f2529619b80 1 Processor -- start 2023-10-06T21:21:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.467+0000 7f2529619b80 1 -- start start 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e69064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e66d3920 con 0x5619e68ff9e0 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e68f2de0 con 0x5619e690a890 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f2529619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e690a710 con 0x5619e68ff0c0 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36418/0 (socket says 172.21.15.31:36418) 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:33054/0 (socket says 172.21.15.31:33054) 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250effd700 1 -- 172.21.15.31:0/3624500652 learned_addr learned my addr 172.21.15.31:0/3624500652 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.468+0000 7f250ffff700 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e69064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 -- 172.21.15.31:0/3624500652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 msgr2=0x5619e69064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e69064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 -- 172.21.15.31:0/3624500652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 msgr2=0x5619e6907ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 -- 172.21.15.31:0/3624500652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e68f7140 con 0x5619e68ff0c0 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250effd700 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f250f7fe700 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f250009b490 tx=0x7f2500022b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0554a277d9c15ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f24f67fc700 1 -- 172.21.15.31:0/3624500652 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250009b0f0 con 0x5619e68ff0c0 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f24f67fc700 1 -- 172.21.15.31:0/3624500652 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250009be90 con 0x5619e68ff0c0 2023-10-06T21:21:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.469+0000 7f24f67fc700 1 -- 172.21.15.31:0/3624500652 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250009b790 con 0x5619e68ff0c0 2023-10-06T21:21:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 -- 172.21.15.31:0/3624500652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 msgr2=0x5619e6901f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f250009b490 tx=0x7f2500022b50 comp rx=0 tx=0).stop 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 -- 172.21.15.31:0/3624500652 shutdown_connections 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e69064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff0c0 0x5619e6901f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 --2- 172.21.15.31:0/3624500652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff9e0 0x5619e6907ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.470+0000 7f2529619b80 1 -- 172.21.15.31:0/3624500652 >> 172.21.15.31:0/3624500652 conn(0x5619e6874260 msgr2=0x5619e6905de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.471+0000 7f2529619b80 1 -- 172.21.15.31:0/3624500652 shutdown_connections 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.471+0000 7f2529619b80 1 -- 172.21.15.31:0/3624500652 wait complete. 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.471+0000 7f2529619b80 1 -- start start 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 Processor -- start 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e6910d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 0x5619e69112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e690f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e690f860 con 0x5619e68ff0c0 2023-10-06T21:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e690f9d0 con 0x5619e690a890 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e690fb90 con 0x5619e68ff9e0 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.472+0000 7f2529619b80 10 client.0 _ll_register_callbacks cb 0x5619e664c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.473+0000 7f250effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 0x5619e69112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.473+0000 7f250effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 0x5619e69112d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:33066/0 (socket says 172.21.15.31:33066) 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.473+0000 7f250effd700 1 -- 172.21.15.31:0/905564202 learned_addr learned my addr 172.21.15.31:0/905564202 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.473+0000 7f250ffff700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e690f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.473+0000 7f250f7fe700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e6910d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 -- 172.21.15.31:0/905564202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 msgr2=0x5619e69112d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 0x5619e69112d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 -- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 msgr2=0x5619e690f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:56.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e690f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 -- 172.21.15.31:0/905564202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e667dde0 con 0x5619e68ff0c0 2023-10-06T21:21:56.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f250f7fe700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e6910d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5619e68fc2d0 tx=0x7f250009bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e7cdd94759909c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:56.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.474+0000 7f24eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:21:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250009b0f0 con 0x5619e68ff0c0 2023-10-06T21:21:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f250009b790 con 0x5619e68ff0c0 2023-10-06T21:21:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25040054b0 con 0x5619e68ff0c0 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250009f8e0 con 0x5619e68ff0c0 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f2529619b80 10 client.16011 fetch_fsmap learned FSMap version 27 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619e6666c60 con 0x5619e68ff0c0 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.475+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25040054b0 con 0x5619e68ff0c0 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25000b7070 con 0x5619e68ff0c0 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 20 client.16011 populate_metadata read hostname 'smithi031' 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e696eb20 0x5619e6970f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 4 client.16011 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5619e684bb30 con 0x5619e696eb20 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.476+0000 7f250effd700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e696eb20 0x5619e6970f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.477+0000 7f250effd700 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e696eb20 0x5619e6970f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d66a18bc47edee server_cookie=5140f939bc5212b6 in_seq=0 out_seq=0 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.477+0000 7f24eeffd700 10 client.16011 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.477+0000 7f24eeffd700 1 -- 172.21.15.31:0/905564202 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5619e684bb30 con 0x5619e696eb20 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.477+0000 7f24eeffd700 10 client.16011 handle_command_reply: tid=0 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.477+0000 7f2529619b80 1 client.16011 shutdown 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 msgr2=0x5619e6910d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e6910d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5619e68fc2d0 tx=0x7f250009bcd0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 shutdown_connections 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619e690a890 0x5619e690f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5619e696eb20 0x5619e6970f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619e68ff0c0 0x5619e6910d90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 --2- 172.21.15.31:0/905564202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619e68ff9e0 0x5619e69112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 >> 172.21.15.31:0/905564202 conn(0x5619e6874260 msgr2=0x5619e69097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f24eeffd700 0 client.16011 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 shutdown_connections 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 1 -- 172.21.15.31:0/905564202 wait complete. 2023-10-06T21:21:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.478+0000 7f2529619b80 20 client.16011 trim_cache size 0 max 16384 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "50222a11-4e6f-4db7-8464-1f34d0a2fb5a": { 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "50222a11-4e6f-4db7-8464-1f34d0a2fb5a", 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-10-06T21:21:56.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:21:56.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:21:56.488 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:21:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 msgr2=0x7f24f80a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 0x7f24f80a3cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2508009550 tx=0x7f25080093b0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 msgr2=0x7f25100ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2500005740 tx=0x7f2500005770 comp rx=0 tx=0).stop 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 shutdown_connections 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f251010b410 0x7f2510071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f24f80a1810 0x7f24f80a3cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2510101ff0 0x7f25100ffc60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 --2- 172.21.15.31:0/3358255057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2510102910 0x7f25101001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.479+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 >> 172.21.15.31:0/3358255057 conn(0x7f25100fb830 msgr2=0x7f25100fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.480+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 shutdown_connections 2023-10-06T21:21:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:21:56.480+0000 7f2515bff700 1 -- 172.21.15.31:0/3358255057 wait complete. 2023-10-06T21:21:56.510 INFO:tasks.cephfs.filesystem:scrub status for tag:50222a11-4e6f-4db7-8464-1f34d0a2fb5a - {'path': '/', 'tag': '50222a11-4e6f-4db7-8464-1f34d0a2fb5a', 'options': 'recursive,force'} 2023-10-06T21:21:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:57 smithi146 ceph-mon[102809]: scrub queued for path: / 2023-10-06T21:21:57.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:57 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:57 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:21:57.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:57 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:57.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:57 smithi142 ceph-mon[102951]: scrub queued for path: / 2023-10-06T21:21:57.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:57 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:57.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:57 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:21:57.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:57 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:57 smithi031 ceph-mon[98539]: scrub queued for path: / 2023-10-06T21:21:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:57 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:57 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:21:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:57 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:21:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:58 smithi146 ceph-mon[102809]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:58 smithi142 ceph-mon[102951]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:58 smithi031 ceph-mon[98539]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T21:21:59.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:21:59 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:21:59.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:21:59 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:21:59.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:21:59 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:22:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:00 smithi146 ceph-mon[102809]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:22:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:00 smithi142 ceph-mon[102951]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:22:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:00 smithi031 ceph-mon[98539]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T21:22:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:02 smithi146 ceph-mon[102809]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:22:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:02 smithi142 ceph-mon[102951]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:22:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:02 smithi031 ceph-mon[98539]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T21:22:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:04 smithi146 ceph-mon[102809]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2023-10-06T21:22:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:04 smithi142 ceph-mon[102951]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2023-10-06T21:22:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:04 smithi031 ceph-mon[98539]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2023-10-06T21:22:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 -- 172.21.15.31:0/2467596033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c106f60 msgr2=0x7f336c107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 --2- 172.21.15.31:0/2467596033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c106f60 0x7f336c107340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f335c009990 tx=0x7f335c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 -- 172.21.15.31:0/2467596033 shutdown_connections 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 --2- 172.21.15.31:0/2467596033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c068c90 0x7f336c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 --2- 172.21.15.31:0/2467596033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c106f60 0x7f336c107340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 --2- 172.21.15.31:0/2467596033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c0682d0 0x7f336c068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 -- 172.21.15.31:0/2467596033 >> 172.21.15.31:0/2467596033 conn(0x7f336c076b70 msgr2=0x7f336c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.922+0000 7f3370fff700 1 -- 172.21.15.31:0/2467596033 shutdown_connections 2023-10-06T21:22:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.923+0000 7f3370fff700 1 -- 172.21.15.31:0/2467596033 wait complete. 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.923+0000 7f3370fff700 1 Processor -- start 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 -- start start 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 0x7f336c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19e2e0 con 0x7f336c106f60 2023-10-06T21:22:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19e450 con 0x7f336c0682d0 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.924+0000 7f3370fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19e610 con 0x7f336c068c90 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f3363fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39324/0 (socket says 172.21.15.31:39324) 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f3363fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41528/0 (socket says 172.21.15.31:41528) 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 -- 172.21.15.31:0/489778775 learned_addr learned my addr 172.21.15.31:0/489778775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336ad9d700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 0x7f336c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 -- 172.21.15.31:0/489778775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 msgr2=0x7f336c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 -- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 msgr2=0x7f336c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 0x7f336c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 -- 172.21.15.31:0/489778775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354009c40 con 0x7f336c0682d0 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f3363fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336ad9d700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 0x7f336c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.925+0000 7f336a59c700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f335c000c00 tx=0x7f335c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.926+0000 7f33637fe700 1 -- 172.21.15.31:0/489778775 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c01d070 con 0x7f336c0682d0 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.926+0000 7f33637fe700 1 -- 172.21.15.31:0/489778775 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f335c022470 con 0x7f336c0682d0 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.926+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c0095f0 con 0x7f336c0682d0 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.926+0000 7f33637fe700 1 -- 172.21.15.31:0/489778775 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c017650 con 0x7f336c0682d0 2023-10-06T21:22:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.926+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c1a5f50 con 0x7f336c0682d0 2023-10-06T21:22:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 Processor -- start 2023-10-06T21:22:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33637fe700 1 -- 172.21.15.31:0/489778775 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f335c0225e0 con 0x7f336c0682d0 2023-10-06T21:22:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 -- start start 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58d24400 con 0x563c58dc63d0 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58d15bc0 con 0x563c58dbac00 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.931+0000 7f33848c9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58d134e0 con 0x563c58dbb520 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f3363fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39334/0 (socket says 172.21.15.31:39334) 2023-10-06T21:22:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 -- 172.21.15.31:0/3562672506 learned_addr learned my addr 172.21.15.31:0/3562672506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336a59c700 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 -- 172.21.15.31:0/3562672506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 msgr2=0x563c58dc5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 -- 172.21.15.31:0/3562672506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 msgr2=0x563c58dc87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336ad9d700 1 -- 172.21.15.31:0/3562672506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c58b39d80 con 0x563c58dbac00 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.932+0000 7f336a59c700 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3364011540 tx=0x7f3364011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e847ac6fd21b041 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f3360ff9700 1 -- 172.21.15.31:0/3562672506 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364019630 con 0x563c58dbac00 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f3360ff9700 1 -- 172.21.15.31:0/3562672506 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3364019c20 con 0x563c58dbac00 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f33637fe700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 0x7f33580a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f3360ff9700 1 -- 172.21.15.31:0/3562672506 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400ebd0 con 0x563c58dbac00 2023-10-06T21:22:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f33637fe700 1 -- 172.21.15.31:0/489778775 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f335c0ce740 con 0x7f336c0682d0 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.933+0000 7f336a59c700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 0x7f33580a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 -- 172.21.15.31:0/3562672506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 msgr2=0x563c58dbafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f336a59c700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 0x7f33580a3c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f335c0cbd20 tx=0x7f335c009ca0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3364011540 tx=0x7f3364011850 comp rx=0 tx=0).stop 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 -- 172.21.15.31:0/3562672506 shutdown_connections 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbafe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3562672506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 -- 172.21.15.31:0/3562672506 >> 172.21.15.31:0/3562672506 conn(0x563c58d30200 msgr2=0x563c58db7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 -- 172.21.15.31:0/3562672506 shutdown_connections 2023-10-06T21:22:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.934+0000 7f33848c9b80 1 -- 172.21.15.31:0/3562672506 wait complete. 2023-10-06T21:22:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.935+0000 7f33848c9b80 1 -- start start 2023-10-06T21:22:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.935+0000 7f33848c9b80 1 Processor -- start 2023-10-06T21:22:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dc5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58d0edd0 con 0x563c58dc63d0 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58cab6b0 con 0x563c58dbac00 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58cab350 con 0x563c58dbb520 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f33848c9b80 10 client.0 _ll_register_callbacks cb 0x563c58b087a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dc5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f3363fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dc5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39348/0 (socket says 172.21.15.31:39348) 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f336ad9d700 1 -- 172.21.15.31:0/1210691102 learned_addr learned my addr 172.21.15.31:0/1210691102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f3363fff700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41550/0 (socket says 172.21.15.31:41550) 2023-10-06T21:22:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.936+0000 7f336ad9d700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 -- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 msgr2=0x563c58dc5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dc5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 -- 172.21.15.31:0/1210691102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 msgr2=0x563c58dc36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 -- 172.21.15.31:0/1210691102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3364009d40 con 0x563c58dbb520 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f336ad9d700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3363fff700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3354007860 tx=0x7f3354003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2afe7f50aca21a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354004db0 con 0x563c58dbb520 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c58daed50 con 0x563c58dbb520 2023-10-06T21:22:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.937+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3354004450 con 0x563c58dbb520 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3354004610 con 0x563c58dbb520 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 10 client.25723 fetch_fsmap learned FSMap version 27 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c58b22c00 con 0x563c58dbb520 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c58daed50 con 0x563c58dbb520 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f335401c030 con 0x563c58dbb520 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 20 client.25723 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.938+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58dd5ff0 0x563c58dd8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.939+0000 7f33848c9b80 4 client.25723 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.939+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c58d07db0 con 0x563c58dd5ff0 2023-10-06T21:22:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.939+0000 7f336ad9d700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58dd5ff0 0x563c58dd8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.939+0000 7f3351ffb700 10 client.25723 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.939+0000 7f336ad9d700 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58dd5ff0 0x563c58dd8410 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce03fbd81709b83f server_cookie=6ed0e7636ecc872d in_seq=0 out_seq=0 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.940+0000 7f3351ffb700 1 -- 172.21.15.31:0/1210691102 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c58d07db0 con 0x563c58dd5ff0 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.940+0000 7f3351ffb700 10 client.25723 handle_command_reply: tid=0 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.940+0000 7f33848c9b80 1 client.25723 shutdown 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 msgr2=0x563c58dc3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3354007860 tx=0x7f3354003f90 comp rx=0 tx=0).stop 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 shutdown_connections 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58dd5ff0 0x563c58dd8410 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f3351ffb700 0 client.25723 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dc5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 --2- 172.21.15.31:0/1210691102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbb520 0x563c58dc3020 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 >> 172.21.15.31:0/1210691102 conn(0x563c58d30200 msgr2=0x563c58db6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 shutdown_connections 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 1 -- 172.21.15.31:0/1210691102 wait complete. 2023-10-06T21:22:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.941+0000 7f33848c9b80 20 client.25723 trim_cache size 0 max 16384 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 Processor -- start 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 -- start start 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 0x563c58dd16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dc63d0 0x563c58dbe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58b8f8c0 con 0x563c58dbb520 2023-10-06T21:22:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58dd8710 con 0x563c58dbac00 2023-10-06T21:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f33848c9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58dc3680 con 0x563c58dc63d0 2023-10-06T21:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f3363fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 0x563c58dd16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39350/0 (socket says 172.21.15.31:39350) 2023-10-06T21:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.969+0000 7f336ad9d700 1 -- 172.21.15.31:0/3541880474 learned_addr learned my addr 172.21.15.31:0/3541880474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dc63d0 0x563c58dbe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 -- 172.21.15.31:0/3541880474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dc63d0 msgr2=0x563c58dbe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dc63d0 0x563c58dbe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 -- 172.21.15.31:0/3541880474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 msgr2=0x563c58dd16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 0x563c58dd16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 -- 172.21.15.31:0/3541880474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c58b601f0 con 0x563c58dbac00 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f3363fff700 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 0x563c58dd16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.970+0000 7f336ad9d700 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33640046e0 tx=0x7f3364000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830192adac858093 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.971+0000 7f33617fa700 1 -- 172.21.15.31:0/3541880474 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402d3b0 con 0x563c58dbac00 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.971+0000 7f33617fa700 1 -- 172.21.15.31:0/3541880474 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3364034070 con 0x563c58dbac00 2023-10-06T21:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.971+0000 7f33617fa700 1 -- 172.21.15.31:0/3541880474 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364005520 con 0x563c58dbac00 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.971+0000 7f33848c9b80 1 -- 172.21.15.31:0/3541880474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 msgr2=0x563c58dbdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.971+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33640046e0 tx=0x7f3364000bc0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 -- 172.21.15.31:0/3541880474 shutdown_connections 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dc63d0 0x563c58dbe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbac00 0x563c58dbdaa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 --2- 172.21.15.31:0/3541880474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dbb520 0x563c58dd16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 -- 172.21.15.31:0/3541880474 >> 172.21.15.31:0/3541880474 conn(0x563c58d30200 msgr2=0x563c58dd8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 -- 172.21.15.31:0/3541880474 shutdown_connections 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.972+0000 7f33848c9b80 1 -- 172.21.15.31:0/3541880474 wait complete. 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 -- start start 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 Processor -- start 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 0x563c58dcaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 0x563c58dcb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dcb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58e28e20 con 0x563c58dc63d0 2023-10-06T21:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58e28f90 con 0x563c58dbb520 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c58e29150 con 0x563c58dbac00 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.973+0000 7f33848c9b80 10 client.0 _ll_register_callbacks cb 0x7f3354003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.974+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 0x563c58dcaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.974+0000 7f336ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 0x563c58dcaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41574/0 (socket says 172.21.15.31:41574) 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.974+0000 7f336ad9d700 1 -- 172.21.15.31:0/856818447 learned_addr learned my addr 172.21.15.31:0/856818447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.974+0000 7f336ad9d700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dcb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.974+0000 7f3363fff700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 0x563c58dcb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 -- 172.21.15.31:0/856818447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 msgr2=0x563c58dcaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 0x563c58dcaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 -- 172.21.15.31:0/856818447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 msgr2=0x563c58dcb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dcb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 -- 172.21.15.31:0/856818447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c58b22c00 con 0x563c58dbb520 2023-10-06T21:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3363fff700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 0x563c58dcb4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f335400b010 tx=0x7f33540057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e8f2d4359bb4a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400e4f0 con 0x563c58dbb520 2023-10-06T21:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f335400e650 con 0x563c58dbb520 2023-10-06T21:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.975+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3364019c20 con 0x563c58dbb520 2023-10-06T21:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335400e830 con 0x563c58dbb520 2023-10-06T21:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 10 client.25769 fetch_fsmap learned FSMap version 27 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c58b39d80 con 0x563c58dbb520 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3364019c20 con 0x563c58dbb520 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3354037060 con 0x563c58dbb520 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 20 client.25769 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58e2a850 0x563c58e2cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 4 client.25769 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.976+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c58d07ad0 con 0x563c58e2a850 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.977+0000 7f336ad9d700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58e2a850 0x563c58e2cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.978+0000 7f3351ffb700 10 client.25769 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.977+0000 7f336ad9d700 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58e2a850 0x563c58e2cc70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6058d73b4fd31320 server_cookie=8325f522556cdc3 in_seq=0 out_seq=0 2023-10-06T21:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f3351ffb700 1 -- 172.21.15.31:0/856818447 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c58d07ad0 con 0x563c58e2a850 2023-10-06T21:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f3351ffb700 10 client.25769 handle_command_reply: tid=0 2023-10-06T21:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 client.25769 shutdown 2023-10-06T21:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 msgr2=0x563c58dcb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 0x563c58dcb4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f335400b010 tx=0x7f33540057a0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 shutdown_connections 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563c58e2a850 0x563c58e2cc70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c58dc63d0 0x563c58dcb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f3351ffb700 0 client.25769 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c58dbac00 0x563c58dcaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 --2- 172.21.15.31:0/856818447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c58dbb520 0x563c58dcb4a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.985+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 >> 172.21.15.31:0/856818447 conn(0x563c58d30200 msgr2=0x563c58db6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 shutdown_connections 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f33848c9b80 1 -- 172.21.15.31:0/856818447 wait complete. 2023-10-06T21:22:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f33848c9b80 20 client.25769 trim_cache size 0 max 16384 2023-10-06T21:22:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:22:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:22:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:22:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 msgr2=0x7f33580a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 0x7f33580a3c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f335c0cbd20 tx=0x7f335c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 msgr2=0x7f336c1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.986+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f335c000c00 tx=0x7f335c004730 comp rx=0 tx=0).stop 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 shutdown_connections 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f33580a1790 0x7f33580a3c50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c106f60 0x7f336c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c0682d0 0x7f336c1990f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 --2- 172.21.15.31:0/489778775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f336c068c90 0x7f336c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 >> 172.21.15.31:0/489778775 conn(0x7f336c076b70 msgr2=0x7f336c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 shutdown_connections 2023-10-06T21:22:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:05.987+0000 7f3370fff700 1 -- 172.21.15.31:0/489778775 wait complete. 2023-10-06T21:22:06.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:06 smithi146 ceph-mon[102809]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2023-10-06T21:22:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:06 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:06 smithi142 ceph-mon[102951]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2023-10-06T21:22:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:06 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:06 smithi031 ceph-mon[98539]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2023-10-06T21:22:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:06 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T21:22:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:07.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:07 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T21:22:07.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:07.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:07.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:07 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:07.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:07 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:22:09.025 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 61% (7406/12065) Updating files: 62% (7481/12065) Updating files: 63% (7601/12065) Updating files: 64% (7722/12065) Updating files: 65% (7843/12065) Updating files: 66% (7963/12065) Updating files: 67% (8084/12065) Updating files: 68% (8205/12065) Updating files: 69% (8325/12065) Updating files: 70% (8446/12065) Updating files: 71% (8567/12065) Updating files: 72% (8687/12065) Updating files: 73% (8808/12065) Updating files: 74% (8929/12065) Updating files: 75% (9049/12065) Updating files: 76% (9170/12065) Updating files: 77% (9291/12065) Updating files: 78% (9411/12065) Updating files: 79% (9532/12065) Updating files: 80% (9652/12065) Updating files: 81% (9773/12065) Updating files: 82% (9894/12065) Updating files: 83% (10014/12065) Updating files: 84% (10135/12065) Updating files: 85% (10256/12065) Updating files: 86% (10376/12065) Updating files: 87% (10497/12065) Updating files: 88% (10618/12065) Updating files: 89% (10738/12065) Updating files: 90% (10859/12065) Updating files: 91% (10980/12065) Updating files: 92% (11100/12065) Updating files: 93% (11221/12065) Updating files: 94% (11342/12065) Updating files: 95% (11462/12065) Updating files: 96% (11583/12065) Updating files: 97% (11704/12065) Updating files: 97% (11759/12065) Updating files: 98% (11824/12065) Updating files: 99% (11945/12065) Updating files: 100% (12065/12065) Updating files: 100% (12065/12065), done. 2023-10-06T21:22:09.135 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to '14d8d032507701c76ca224e769ee65a96b660a8c'. 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.136 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-10-06T21:22:09.137 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at 14d8d032507 Merge remote-tracking branch 'rishabh/ceph-auth-caps-val-caps' into rishabh-cephfs-mon 2023-10-06T21:22:09.140 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-06T21:22:09.243 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-06T21:22:09.248 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T21:22:09.248 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-06T21:22:09.837 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-06T21:22:09.920 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-06T21:22:09.984 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T21:22:09.988 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T21:22:09.988 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-06T21:22:10.051 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T21:22:10.058 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:22:10.059 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-06T21:22:10.102 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-10-06T21:22:10.103 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-10-06T21:22:10.104 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=14d8d032507701c76ca224e769ee65a96b660a8c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-10-06T21:22:10.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:09 smithi142 ceph-mon[102951]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-10-06T21:22:10.209 INFO:tasks.workunit.client.0.smithi031.stderr:+ dbench 1 2023-10-06T21:22:10.219 INFO:tasks.workunit.client.0.smithi031.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-10-06T21:22:10.220 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:22:10.220 INFO:tasks.workunit.client.0.smithi031.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-10-06T21:22:10.226 INFO:tasks.workunit.client.0.smithi031.stdout:0 of 1 processes prepared for launch 0 sec 2023-10-06T21:22:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:09 smithi031 ceph-mon[98539]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-10-06T21:22:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:09 smithi146 ceph-mon[102809]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-10-06T21:22:10.326 INFO:tasks.workunit.client.0.smithi031.stdout:1 of 1 processes prepared for launch 0 sec 2023-10-06T21:22:10.327 INFO:tasks.workunit.client.0.smithi031.stdout:releasing clients 2023-10-06T21:22:11.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 671 21.16 MB/sec warmup 1 sec latency 15.690 ms 2023-10-06T21:22:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:11 smithi031 ceph-mon[98539]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-10-06T21:22:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:11 smithi146 ceph-mon[102809]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-10-06T21:22:12.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 1099 11.47 MB/sec warmup 2 sec latency 22.660 ms 2023-10-06T21:22:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:11 smithi142 ceph-mon[102951]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-10-06T21:22:13.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 1551 8.70 MB/sec warmup 3 sec latency 16.687 ms 2023-10-06T21:22:14.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:13 smithi031 ceph-mon[98539]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 926 KiB/s wr, 27 op/s 2023-10-06T21:22:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:13 smithi146 ceph-mon[102809]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 926 KiB/s wr, 27 op/s 2023-10-06T21:22:14.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 1973 6.61 MB/sec warmup 4 sec latency 21.472 ms 2023-10-06T21:22:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:13 smithi142 ceph-mon[102951]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 926 KiB/s wr, 27 op/s 2023-10-06T21:22:15.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 2396 5.59 MB/sec warmup 5 sec latency 18.487 ms 2023-10-06T21:22:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:15 smithi031 ceph-mon[98539]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-10-06T21:22:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:15 smithi146 ceph-mon[102809]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-10-06T21:22:16.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 2966 5.58 MB/sec warmup 6 sec latency 17.909 ms 2023-10-06T21:22:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:15 smithi142 ceph-mon[102951]: pgmap v42: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-10-06T21:22:17.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 3332 5.05 MB/sec warmup 7 sec latency 18.856 ms 2023-10-06T21:22:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:18 smithi031 ceph-mon[98539]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-10-06T21:22:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:18 smithi146 ceph-mon[102809]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-10-06T21:22:18.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 3790 4.97 MB/sec warmup 8 sec latency 24.264 ms 2023-10-06T21:22:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:18 smithi142 ceph-mon[102951]: pgmap v43: 97 pgs: 97 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 -- 172.21.15.31:0/1500226260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 msgr2=0x7f72941062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 --2- 172.21.15.31:0/1500226260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f72941062d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7284009990 tx=0x7f7284009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 -- 172.21.15.31:0/1500226260 shutdown_connections 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 --2- 172.21.15.31:0/1500226260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294100930 0x7f729410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 --2- 172.21.15.31:0/1500226260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f72941062d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 --2- 172.21.15.31:0/1500226260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72940fff70 0x7f72941003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.154+0000 7f7298fff700 1 -- 172.21.15.31:0/1500226260 >> 172.21.15.31:0/1500226260 conn(0x7f7294076b70 msgr2=0x7f7294076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.155+0000 7f7298fff700 1 -- 172.21.15.31:0/1500226260 shutdown_connections 2023-10-06T21:22:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.155+0000 7f7298fff700 1 -- 172.21.15.31:0/1500226260 wait complete. 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 Processor -- start 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 -- start start 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7294100930 0x7f72941996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f729419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e390 con 0x7f7294105ef0 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e500 con 0x7f72940fff70 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.156+0000 7f7298fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419e6c0 con 0x7f7294100930 2023-10-06T21:22:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f729259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f729259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:37298/0 (socket says 172.21.15.31:37298) 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f729259c700 1 -- 172.21.15.31:0/1796987015 learned_addr learned my addr 172.21.15.31:0/1796987015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f729419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f728bfff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7294100930 0x7f72941996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 -- 172.21.15.31:0/1796987015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7294100930 msgr2=0x7f72941996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7294100930 0x7f72941996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 -- 172.21.15.31:0/1796987015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 msgr2=0x7f72941991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 -- 172.21.15.31:0/1796987015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840095f0 con 0x7f7294105ef0 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f729259c700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.157+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f729419dd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f728c00efc0 tx=0x7f728c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.158+0000 7f728b7fe700 1 -- 172.21.15.31:0/1796987015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c018070 con 0x7f7294105ef0 2023-10-06T21:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.158+0000 7f728b7fe700 1 -- 172.21.15.31:0/1796987015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f728c00f040 con 0x7f7294105ef0 2023-10-06T21:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.158+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729419e940 con 0x7f7294105ef0 2023-10-06T21:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.158+0000 7f728b7fe700 1 -- 172.21.15.31:0/1796987015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00cb20 con 0x7f7294105ef0 2023-10-06T21:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.158+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72941a61e0 con 0x7f7294105ef0 2023-10-06T21:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.162+0000 7f728b7fe700 1 -- 172.21.15.31:0/1796987015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f728c00cce0 con 0x7f7294105ef0 2023-10-06T21:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.163+0000 7f72ac8f0b80 1 Processor -- start 2023-10-06T21:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.163+0000 7f72ac8f0b80 1 -- start start 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.163+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b238400 con 0x557c4b2da3d0 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b229bc0 con 0x557c4b2cec00 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b2274e0 con 0x557c4b2cf520 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49354/0 (socket says 172.21.15.31:49354) 2023-10-06T21:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 -- 172.21.15.31:0/1780301449 learned_addr learned my addr 172.21.15.31:0/1780301449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728b7fe700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 0x7f72800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 -- 172.21.15.31:0/1780301449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 msgr2=0x557c4b2d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 -- 172.21.15.31:0/1780301449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 msgr2=0x557c4b2cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.164+0000 7f728bfff700 1 -- 172.21.15.31:0/1780301449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c4b04dd80 con 0x557c4b2da3d0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f728b7fe700 1 -- 172.21.15.31:0/1796987015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f728c0cce60 con 0x7f7294105ef0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 0x7f72800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f728bfff700 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097c0 tx=0x7f727c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478f73f517d4d82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f7288ff9700 1 -- 172.21.15.31:0/1780301449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c01f600 con 0x557c4b2da3d0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f7288ff9700 1 -- 172.21.15.31:0/1780301449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f727c00f850 con 0x557c4b2da3d0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.165+0000 7f7288ff9700 1 -- 172.21.15.31:0/1780301449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c01e6d0 con 0x557c4b2da3d0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/1780301449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 msgr2=0x557c4b2dc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097c0 tx=0x7f727c009260 comp rx=0 tx=0).stop 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f7292d9d700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 0x7f72800a3bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f728c0ca3e0 tx=0x7f728c0b9040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/1780301449 shutdown_connections 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2dc7c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/1780301449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/1780301449 >> 172.21.15.31:0/1780301449 conn(0x557c4b244200 msgr2=0x557c4b2cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/1780301449 shutdown_connections 2023-10-06T21:22:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.166+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/1780301449 wait complete. 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.167+0000 7f72ac8f0b80 1 -- start start 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.167+0000 7f72ac8f0b80 1 Processor -- start 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cec00 0x557c4b2d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 0x557c4b2d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b222dd0 con 0x557c4b2da3d0 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b1bf6b0 con 0x557c4b2cf520 2023-10-06T21:22:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b1bf350 con 0x557c4b2cec00 2023-10-06T21:22:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f72ac8f0b80 10 client.0 _ll_register_callbacks cb 0x557c4b01c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49364/0 (socket says 172.21.15.31:49364) 2023-10-06T21:22:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f728bfff700 1 -- 172.21.15.31:0/3097310474 learned_addr learned my addr 172.21.15.31:0/3097310474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f729259c700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 0x557c4b2d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.168+0000 7f728bfff700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cec00 0x557c4b2d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 -- 172.21.15.31:0/3097310474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cec00 msgr2=0x557c4b2d9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cec00 0x557c4b2d9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 -- 172.21.15.31:0/3097310474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 msgr2=0x557c4b2d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 -- 172.21.15.31:0/3097310474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c0106e0 con 0x557c4b2cf520 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f728bfff700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f729259c700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 0x557c4b2d7120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f72840095c0 tx=0x7f728400bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12f065f87de1d2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.169+0000 7f7279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284027070 con 0x557c4b2cf520 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f728400f840 con 0x557c4b2cf520 2023-10-06T21:22:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c4b2c2d50 con 0x557c4b2cf520 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f728401d050 con 0x557c4b2cf520 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f72ac8f0b80 10 client.25781 fetch_fsmap learned FSMap version 28 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c4b036c00 con 0x557c4b2cf520 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c4b2c2d50 con 0x557c4b2cf520 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f728402a020 con 0x557c4b2cf520 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f72ac8f0b80 10 client.25781 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.170+0000 7f72ac8f0b80 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f72ac8f0b80 20 client.25781 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b2e9f40 0x557c4b2ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f72ac8f0b80 4 client.25781 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c4b21bdb0 con 0x557c4b2e9f40 2023-10-06T21:22:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f728bfff700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b2e9f40 0x557c4b2ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f728bfff700 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b2e9f40 0x557c4b2ec360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52e128652cea35b server_cookie=58e0d30ebcff111a in_seq=0 out_seq=0 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.171+0000 7f7279ffb700 10 client.25781 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f7279ffb700 1 -- 172.21.15.31:0/3097310474 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c4b21bdb0 con 0x557c4b2e9f40 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f7279ffb700 10 client.25781 handle_command_reply: tid=0 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f72ac8f0b80 1 client.25781 shutdown 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 msgr2=0x557c4b2d7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 0x557c4b2d7120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f72840095c0 tx=0x7f728400bd90 comp rx=0 tx=0).stop 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 shutdown_connections 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.172+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b2e9f40 0x557c4b2ec360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f7279ffb700 0 client.25781 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cec00 0x557c4b2d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3097310474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cf520 0x557c4b2d7120 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 >> 172.21.15.31:0/3097310474 conn(0x557c4b244200 msgr2=0x557c4b2cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 shutdown_connections 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3097310474 wait complete. 2023-10-06T21:22:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.173+0000 7f72ac8f0b80 20 client.25781 trim_cache size 0 max 16384 2023-10-06T21:22:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.191+0000 7f72ac8f0b80 1 Processor -- start 2023-10-06T21:22:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.191+0000 7f72ac8f0b80 1 -- start start 2023-10-06T21:22:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b0a38c0 con 0x557c4b2da3d0 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b2c7860 con 0x557c4b2cec00 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b2c2890 con 0x557c4b2cf520 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49374/0 (socket says 172.21.15.31:49374) 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f728bfff700 1 -- 172.21.15.31:0/2959243461 learned_addr learned my addr 172.21.15.31:0/2959243461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f729259c700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.192+0000 7f728bfff700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 -- 172.21.15.31:0/2959243461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 msgr2=0x557c4b2e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 -- 172.21.15.31:0/2959243461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 msgr2=0x557c4b2d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2d7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 -- 172.21.15.31:0/2959243461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c4b0741f0 con 0x557c4b2da3d0 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f729259c700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f728bfff700 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f727c013040 tx=0x7f727c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a8f70339409692 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f72897fa700 1 -- 172.21.15.31:0/2959243461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c00f6c0 con 0x557c4b2da3d0 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f72897fa700 1 -- 172.21.15.31:0/2959243461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f727c032560 con 0x557c4b2da3d0 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.193+0000 7f72897fa700 1 -- 172.21.15.31:0/2959243461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c003680 con 0x557c4b2da3d0 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/2959243461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 msgr2=0x557c4b2d68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f727c013040 tx=0x7f727c009c90 comp rx=0 tx=0).stop 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/2959243461 shutdown_connections 2023-10-06T21:22:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2da3d0 0x557c4b2d68e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2cec00 0x557c4b2d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/2959243461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/2959243461 >> 172.21.15.31:0/2959243461 conn(0x557c4b244200 msgr2=0x557c4b2e61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/2959243461 shutdown_connections 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.194+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/2959243461 wait complete. 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 -- start start 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 Processor -- start 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2da3d0 0x557c4b33cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b2df540 con 0x557c4b2cec00 2023-10-06T21:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b33d270 con 0x557c4b2da3d0 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4b33d3b0 con 0x557c4b2cf520 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.195+0000 7f72ac8f0b80 10 client.0 _ll_register_callbacks cb 0x7f7284009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f729259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49390/0 (socket says 172.21.15.31:49390) 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 -- 172.21.15.31:0/3297749509 learned_addr learned my addr 172.21.15.31:0/3297749509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2da3d0 0x557c4b33cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 -- 172.21.15.31:0/3297749509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 msgr2=0x557c4b2dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 -- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2da3d0 msgr2=0x557c4b33cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2da3d0 0x557c4b33cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 -- 172.21.15.31:0/3297749509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c4b036c00 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f729259c700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f728bfff700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f727c0105c0 tx=0x7f727c00f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730a9e33b00f01d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.196+0000 7f7279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c032c40 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f727c000e40 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f727c00f850 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f727c00f6b0 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f72ac8f0b80 10 client.16047 fetch_fsmap learned FSMap version 28 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f727c028040 con 0x557c4b2cec00 2023-10-06T21:22:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f727c043020 con 0x557c4b2cec00 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f727c01d030 con 0x557c4b2cec00 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f72ac8f0b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.197+0000 7f72ac8f0b80 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f72ac8f0b80 20 client.16047 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b33e6f0 0x557c4b340b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f72ac8f0b80 4 client.16047 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c4b21bad0 con 0x557c4b33e6f0 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f729259c700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b33e6f0 0x557c4b340b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f7279ffb700 10 client.16047 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.198+0000 7f729259c700 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b33e6f0 0x557c4b340b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1801bb61d398bfe server_cookie=e8b17d21965a1101 in_seq=0 out_seq=0 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f7279ffb700 1 -- 172.21.15.31:0/3297749509 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c4b21bad0 con 0x557c4b33e6f0 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f7279ffb700 10 client.16047 handle_command_reply: tid=0 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 client.16047 shutdown 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 msgr2=0x557c4b2de740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f727c0105c0 tx=0x7f727c00f4d0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 shutdown_connections 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c4b2da3d0 0x557c4b33cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c4b33e6f0 0x557c4b340b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c4b2cec00 0x557c4b2de740 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 --2- 172.21.15.31:0/3297749509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c4b2cf520 0x557c4b2dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f7279ffb700 0 client.16047 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 >> 172.21.15.31:0/3297749509 conn(0x557c4b244200 msgr2=0x557c4b2ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 shutdown_connections 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 1 -- 172.21.15.31:0/3297749509 wait complete. 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.250+0000 7f72ac8f0b80 20 client.16047 trim_cache size 0 max 16384 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:22:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:22:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 msgr2=0x7f72800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 0x7f72800a3bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f728c0ca3e0 tx=0x7f728c0b9040 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 msgr2=0x7f729419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f729419dd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f728c00efc0 tx=0x7f728c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 shutdown_connections 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f72800a1710 0x7f72800a3bd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7294105ef0 0x7f729419dd50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72940fff70 0x7f72941991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 --2- 172.21.15.31:0/1796987015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7294100930 0x7f72941996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 >> 172.21.15.31:0/1796987015 conn(0x7f7294076b70 msgr2=0x7f729410b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.251+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 shutdown_connections 2023-10-06T21:22:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:19.252+0000 7f7298fff700 1 -- 172.21.15.31:0/1796987015 wait complete. 2023-10-06T21:22:19.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 4005 4.45 MB/sec warmup 9 sec latency 22.811 ms 2023-10-06T21:22:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:20 smithi031 ceph-mon[98539]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 52 op/s 2023-10-06T21:22:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:20 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:20 smithi146 ceph-mon[102809]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 52 op/s 2023-10-06T21:22:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:20 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:20.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 4179 4.01 MB/sec warmup 10 sec latency 325.976 ms 2023-10-06T21:22:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:20 smithi142 ceph-mon[102951]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 52 op/s 2023-10-06T21:22:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:20 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:21.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 4539 3.72 MB/sec warmup 11 sec latency 16.471 ms 2023-10-06T21:22:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:22 smithi146 ceph-mon[102809]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.3 MiB/s wr, 47 op/s 2023-10-06T21:22:22.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 4965 3.76 MB/sec warmup 12 sec latency 20.000 ms 2023-10-06T21:22:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:22 smithi142 ceph-mon[102951]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.3 MiB/s wr, 47 op/s 2023-10-06T21:22:22.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:22 smithi031 ceph-mon[98539]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.3 MiB/s wr, 47 op/s 2023-10-06T21:22:23.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 5337 3.49 MB/sec warmup 13 sec latency 19.257 ms 2023-10-06T21:22:24.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:24 smithi146 ceph-mon[102809]: pgmap v46: 97 pgs: 97 active+clean; 24 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.4 MiB/s wr, 67 op/s 2023-10-06T21:22:24.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 5743 3.28 MB/sec warmup 14 sec latency 26.223 ms 2023-10-06T21:22:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:24 smithi142 ceph-mon[102951]: pgmap v46: 97 pgs: 97 active+clean; 24 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.4 MiB/s wr, 67 op/s 2023-10-06T21:22:24.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:24 smithi031 ceph-mon[98539]: pgmap v46: 97 pgs: 97 active+clean; 24 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.4 MiB/s wr, 67 op/s 2023-10-06T21:22:25.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:25 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:25.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 6062 3.19 MB/sec warmup 15 sec latency 17.411 ms 2023-10-06T21:22:25.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:25 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:25.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:25 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:26.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:26 smithi146 ceph-mon[102809]: pgmap v47: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2023-10-06T21:22:26.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 6549 3.29 MB/sec warmup 16 sec latency 20.091 ms 2023-10-06T21:22:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:26 smithi142 ceph-mon[102951]: pgmap v47: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2023-10-06T21:22:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:26 smithi031 ceph-mon[98539]: pgmap v47: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2023-10-06T21:22:26.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.734+0000 7f895c9ff700 1 -- 172.21.15.31:0/2794067286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958101a60 msgr2=0x7f8958105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.734+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2794067286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958101a60 0x7f8958105aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f894c0098f0 tx=0x7f894c009c00 comp rx=0 tx=0).stop 2023-10-06T21:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 -- 172.21.15.31:0/2794067286 shutdown_connections 2023-10-06T21:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2794067286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89581061d0 0x7f895810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2794067286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581010b0 0x7f8958101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2794067286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958101a60 0x7f8958105aa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 -- 172.21.15.31:0/2794067286 >> 172.21.15.31:0/2794067286 conn(0x7f89580fc950 msgr2=0x7f89580fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 -- 172.21.15.31:0/2794067286 shutdown_connections 2023-10-06T21:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.735+0000 7f895c9ff700 1 -- 172.21.15.31:0/2794067286 wait complete. 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 Processor -- start 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 -- start start 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581061d0 0x7f89581996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f895819db00 0x7f895819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e580 con 0x7f89581010b0 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e6f0 con 0x7f89581061d0 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.736+0000 7f895c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819e8b0 con 0x7f895819db00 2023-10-06T21:22:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51054/0 (socket says 172.21.15.31:51054) 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 -- 172.21.15.31:0/2827331513 learned_addr learned my addr 172.21.15.31:0/2827331513 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f8956d9d700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f895819db00 0x7f895819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581061d0 0x7f89581996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 -- 172.21.15.31:0/2827331513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f895819db00 msgr2=0x7f895819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f895819db00 0x7f895819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 -- 172.21.15.31:0/2827331513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581061d0 msgr2=0x7f89581996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581061d0 0x7f89581996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.737+0000 7f895659c700 1 -- 172.21.15.31:0/2827331513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009520 con 0x7f89581010b0 2023-10-06T21:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f895659c700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9c0 tx=0x7f894800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f89477fe700 1 -- 172.21.15.31:0/2827331513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948004a30 con 0x7f89581010b0 2023-10-06T21:22:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89581a5e10 con 0x7f89581010b0 2023-10-06T21:22:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f89477fe700 1 -- 172.21.15.31:0/2827331513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f894800cc50 con 0x7f89581010b0 2023-10-06T21:22:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f89477fe700 1 -- 172.21.15.31:0/2827331513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948005410 con 0x7f89581010b0 2023-10-06T21:22:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.738+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581a6240 con 0x7f89581010b0 2023-10-06T21:22:26.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.742+0000 7f89477fe700 1 -- 172.21.15.31:0/2827331513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8948005570 con 0x7f89581010b0 2023-10-06T21:22:26.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.743+0000 7f89477fe700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 0x7f89400a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.743+0000 7f89702d0b80 1 Processor -- start 2023-10-06T21:22:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.743+0000 7f89477fe700 1 -- 172.21.15.31:0/2827331513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8948014070 con 0x7f89581010b0 2023-10-06T21:22:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 0x7f89400a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 -- start start 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 0x55fe32a9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe329f6460 con 0x55fe32a98890 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe329e7c20 con 0x55fe32a8d9e0 2023-10-06T21:22:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f89702d0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe329e5540 con 0x55fe32a8d0c0 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 0x55fe32a9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f895659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 0x7f89400a3c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f894c0098c0 tx=0x7f894c019040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f895659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35700/0 (socket says 172.21.15.31:35700) 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.744+0000 7f895659c700 1 -- 172.21.15.31:0/2230992792 learned_addr learned my addr 172.21.15.31:0/2230992792 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 -- 172.21.15.31:0/2230992792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 msgr2=0x55fe32a98350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a98350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 -- 172.21.15.31:0/2230992792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 msgr2=0x55fe32a9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 0x55fe32a9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 -- 172.21.15.31:0/2230992792 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3280bde0 con 0x55fe32a8d0c0 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f8956d9d700 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 0x55fe32a9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f895659c700 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8948096630 tx=0x7f8948099b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb9121135c0ac53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f8944ff9700 1 -- 172.21.15.31:0/2230992792 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894809a130 con 0x55fe32a8d0c0 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f8944ff9700 1 -- 172.21.15.31:0/2230992792 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f894809e280 con 0x55fe32a8d0c0 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.745+0000 7f8944ff9700 1 -- 172.21.15.31:0/2230992792 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894809a5f0 con 0x55fe32a8d0c0 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 -- 172.21.15.31:0/2230992792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 msgr2=0x55fe32a8d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8948096630 tx=0x7f8948099b60 comp rx=0 tx=0).stop 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 -- 172.21.15.31:0/2230992792 shutdown_connections 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a98890 0x55fe32a9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d0c0 0x55fe32a8d4a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2230992792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 -- 172.21.15.31:0/2230992792 >> 172.21.15.31:0/2230992792 conn(0x55fe32a02260 msgr2=0x55fe32a89450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.746+0000 7f89702d0b80 1 -- 172.21.15.31:0/2230992792 shutdown_connections 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.747+0000 7f89702d0b80 1 -- 172.21.15.31:0/2230992792 wait complete. 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.747+0000 7f89702d0b80 1 -- start start 2023-10-06T21:22:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.747+0000 7f89702d0b80 1 Processor -- start 2023-10-06T21:22:26.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a95ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a98890 0x55fe32a96020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe329e0e30 con 0x55fe32a8d0c0 2023-10-06T21:22:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3297d710 con 0x55fe32a8d9e0 2023-10-06T21:22:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3297d3b0 con 0x55fe32a98890 2023-10-06T21:22:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f89702d0b80 10 client.0 _ll_register_callbacks cb 0x55fe327da800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:26.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51072/0 (socket says 172.21.15.31:51072) 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f895659c700 1 -- 172.21.15.31:0/2193785420 learned_addr learned my addr 172.21.15.31:0/2193785420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a95ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.748+0000 7f8956d9d700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a98890 0x55fe32a96020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 -- 172.21.15.31:0/2193785420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a98890 msgr2=0x55fe32a96020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a98890 0x55fe32a96020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 -- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 msgr2=0x55fe32a95ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a95ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 -- 172.21.15.31:0/2193785420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8950003ae0 con 0x55fe32a8d0c0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f895659c700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8948005950 tx=0x7f8948099850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbd9a686b0eee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f893dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89480a24c0 con 0x55fe32a8d0c0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe32a80f10 con 0x55fe32a8d0c0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.749+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f89480b83f0 con 0x55fe32a8d0c0 2023-10-06T21:22:26.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894809a130 con 0x55fe32a8d0c0 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 10 client.16059 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f89581a6240 con 0x55fe32a8d0c0 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe32a80f10 con 0x55fe32a8d0c0 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f89480cd070 con 0x55fe32a8d0c0 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 20 client.16059 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32aa8030 0x55fe32aaa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 4 client.16059 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:26.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe329d9e10 con 0x55fe32aa8030 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.750+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32aa8030 0x55fe32aaa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.751+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32aa8030 0x55fe32aaa450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a177aff4ee0ff4 server_cookie=c53d4fe1639c94d8 in_seq=0 out_seq=0 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.751+0000 7f893dffb700 10 client.16059 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f893dffb700 1 -- 172.21.15.31:0/2193785420 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe329d9e10 con 0x55fe32aa8030 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f893dffb700 10 client.16059 handle_command_reply: tid=0 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 client.16059 shutdown 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 msgr2=0x55fe32a955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8948005950 tx=0x7f8948099850 comp rx=0 tx=0).stop 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 shutdown_connections 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a98890 0x55fe32a96020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32aa8030 0x55fe32aaa450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d0c0 0x55fe32a955a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f893dffb700 0 client.16059 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2193785420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a8d9e0 0x55fe32a95ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 >> 172.21.15.31:0/2193785420 conn(0x55fe32a02260 msgr2=0x55fe32a88fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 shutdown_connections 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.752+0000 7f89702d0b80 1 -- 172.21.15.31:0/2193785420 wait complete. 2023-10-06T21:22:26.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.753+0000 7f89702d0b80 20 client.16059 trim_cache size 0 max 16384 2023-10-06T21:22:26.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 Processor -- start 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 -- start start 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 0x55fe32a8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32861920 con 0x55fe32a97ec0 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32a8e2e0 con 0x55fe32a98890 2023-10-06T21:22:26.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f89702d0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32a83520 con 0x55fe32a8d9e0 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51074/0 (socket says 172.21.15.31:51074) 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.775+0000 7f8956d9d700 1 -- 172.21.15.31:0/1056329905 learned_addr learned my addr 172.21.15.31:0/1056329905 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f895659c700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 0x55fe32a8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 -- 172.21.15.31:0/1056329905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 msgr2=0x55fe32a8ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 0x55fe32a8ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 -- 172.21.15.31:0/1056329905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 msgr2=0x55fe32a95e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a95e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 -- 172.21.15.31:0/1056329905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe32a85140 con 0x55fe32a97ec0 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f895659c700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 0x55fe32a8ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8955d9b700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f8956d9d700 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f8950009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47d887ce9243176 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.776+0000 7f89457fa700 1 -- 172.21.15.31:0/1056329905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950007e40 con 0x55fe32a97ec0 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89457fa700 1 -- 172.21.15.31:0/1056329905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f895000f040 con 0x55fe32a97ec0 2023-10-06T21:22:26.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89457fa700 1 -- 172.21.15.31:0/1056329905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950005020 con 0x55fe32a97ec0 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 -- 172.21.15.31:0/1056329905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 msgr2=0x55fe32a94230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f8950009e80 comp rx=0 tx=0).stop 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 -- 172.21.15.31:0/1056329905 shutdown_connections 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a97ec0 0x55fe32a94230 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a8d9e0 0x55fe32a8ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 --2- 172.21.15.31:0/1056329905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 -- 172.21.15.31:0/1056329905 >> 172.21.15.31:0/1056329905 conn(0x55fe32a02260 msgr2=0x55fe32a93b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 -- 172.21.15.31:0/1056329905 shutdown_connections 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.777+0000 7f89702d0b80 1 -- 172.21.15.31:0/1056329905 wait complete. 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.778+0000 7f89702d0b80 1 -- start start 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.778+0000 7f89702d0b80 1 Processor -- start 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.778+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.778+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 0x55fe32a9da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.778+0000 7f89702d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f89702d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32a9e850 con 0x55fe32a8d9e0 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f89702d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32afbe30 con 0x55fe32a98890 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f89702d0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe32afbf70 con 0x55fe32a97ec0 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f89702d0b80 10 client.0 _ll_register_callbacks cb 0x7f8950009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51076/0 (socket says 172.21.15.31:51076) 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 -- 172.21.15.31:0/2029281595 learned_addr learned my addr 172.21.15.31:0/2029281595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f8956d9d700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 0x55fe32a9da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 -- 172.21.15.31:0/2029281595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 msgr2=0x55fe32a9da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 0x55fe32a9da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 -- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 msgr2=0x55fe32a9e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a9e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f895659c700 1 -- 172.21.15.31:0/2029281595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895810e5c0 con 0x55fe32a8d9e0 2023-10-06T21:22:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f8956d9d700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.779+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 0x55fe32a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f895659c700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8948098480 tx=0x7f8948004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc04e6297b56be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948099a90 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f894809e280 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f89480b83f0 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89480a24c0 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f89702d0b80 10 client.16065 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8950009ad0 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8948084e90 con 0x55fe32a8d9e0 2023-10-06T21:22:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.780+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f89480a9070 con 0x55fe32a8d9e0 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 20 client.16065 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32afd2e0 0x55fe32aff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 4 client.16065 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe329d9b30 con 0x55fe32afd2e0 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.781+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32afd2e0 0x55fe32aff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f8955d9b700 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32afd2e0 0x55fe32aff700 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e79f59667ed30ada server_cookie=551a45e20b5c3738 in_seq=0 out_seq=0 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f893dffb700 10 client.16065 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f893dffb700 1 -- 172.21.15.31:0/2029281595 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe329d9b30 con 0x55fe32afd2e0 2023-10-06T21:22:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f893dffb700 10 client.16065 handle_command_reply: tid=0 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 client.16065 shutdown 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 msgr2=0x55fe32a9f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8948098480 tx=0x7f8948004750 comp rx=0 tx=0).stop 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 shutdown_connections 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe32a98890 0x55fe32a9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fe32afd2e0 0x55fe32aff700 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fe32a8d9e0 0x55fe32a9f560 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 --2- 172.21.15.31:0/2029281595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe32a97ec0 0x55fe32a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 >> 172.21.15.31:0/2029281595 conn(0x55fe32a02260 msgr2=0x55fe32a97740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f893dffb700 0 client.16065 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.782+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 shutdown_connections 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f89702d0b80 1 -- 172.21.15.31:0/2029281595 wait complete. 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f89702d0b80 20 client.16065 trim_cache size 0 max 16384 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-10-06T21:22:26.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 msgr2=0x7f89400a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 0x7f89400a3c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f894c0098c0 tx=0x7f894c019040 comp rx=0 tx=0).stop 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 msgr2=0x7f8958199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9c0 tx=0x7f894800ecd0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 shutdown_connections 2023-10-06T21:22:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f895819db00 0x7f895819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f89400a1790 0x7f89400a3c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.783+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89581010b0 0x7f8958199160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.784+0000 7f895c9ff700 1 --2- 172.21.15.31:0/2827331513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89581061d0 0x7f89581996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.784+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 >> 172.21.15.31:0/2827331513 conn(0x7f89580fc950 msgr2=0x7f89580fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.784+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 shutdown_connections 2023-10-06T21:22:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:26.784+0000 7f895c9ff700 1 -- 172.21.15.31:0/2827331513 wait complete. 2023-10-06T21:22:26.806 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:22:26.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 -- 172.21.15.31:0/713171862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e410b410 msgr2=0x7f69e410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/713171862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e410b410 0x7f69e410d8d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b0a0 tx=0x7f69dc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 -- 172.21.15.31:0/713171862 shutdown_connections 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/713171862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e410b410 0x7f69e410d8d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/713171862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 0x7f69e41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/713171862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e4102910 0x7f69e410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 -- 172.21.15.31:0/713171862 >> 172.21.15.31:0/713171862 conn(0x7f69e40fb7f0 msgr2=0x7f69e40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.027+0000 7f69eb7ff700 1 -- 172.21.15.31:0/713171862 shutdown_connections 2023-10-06T21:22:27.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.028+0000 7f69eb7ff700 1 -- 172.21.15.31:0/713171862 wait complete. 2023-10-06T21:22:27.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 Processor -- start 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 -- start start 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 0x7f69e40ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e410b410 0x7f69e41a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41009b0 con 0x7f69e4102910 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41a56d0 con 0x7f69e410b410 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69eb7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41a5810 con 0x7f69e4101ff0 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.029+0000 7f69e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51100/0 (socket says 172.21.15.31:51100) 2023-10-06T21:22:27.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 -- 172.21.15.31:0/1400799966 learned_addr learned my addr 172.21.15.31:0/1400799966 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e9d9c700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e410b410 0x7f69e41a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e959b700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 0x7f69e40ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 -- 172.21.15.31:0/1400799966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 msgr2=0x7f69e40ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 0x7f69e40ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 -- 172.21.15.31:0/1400799966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e410b410 msgr2=0x7f69e41a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e410b410 0x7f69e41a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 -- 172.21.15.31:0/1400799966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc009d00 con 0x7f69e4102910 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.030+0000 7f69e8d9a700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e9f0 tx=0x7f69d400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.031+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d400c9c0 con 0x7f69e4102910 2023-10-06T21:22:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.031+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69d4010430 con 0x7f69e4102910 2023-10-06T21:22:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.031+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e41a5a70 con 0x7f69e4102910 2023-10-06T21:22:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.031+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4018680 con 0x7f69e4102910 2023-10-06T21:22:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.031+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e41a5f50 con 0x7f69e4102910 2023-10-06T21:22:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.032+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e4067cd0 con 0x7f69e4102910 2023-10-06T21:22:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.034+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f69d40105a0 con 0x7f69e4102910 2023-10-06T21:22:27.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.036+0000 7f69da7fc700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 0x7f69d00a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.036+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f69d4014070 con 0x7f69e4102910 2023-10-06T21:22:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.036+0000 7f69e959b700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 0x7f69d00a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.037+0000 7f69e959b700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 0x7f69d00a3c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009c60 tx=0x7f69e0009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.041+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69d4052100 con 0x7f69e4102910 2023-10-06T21:22:27.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 6886 3.21 MB/sec warmup 17 sec latency 19.469 ms 2023-10-06T21:22:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.336+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e4110e70 con 0x7f69e4102910 2023-10-06T21:22:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.337+0000 7f69da7fc700 1 -- 172.21.15.31:0/1400799966 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f69d4096750 con 0x7f69e4102910 2023-10-06T21:22:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:22:27.339 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:22:23.142233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:22:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 msgr2=0x7f69d00a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 0x7f69d00a3c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009c60 tx=0x7f69e0009260 comp rx=0 tx=0).stop 2023-10-06T21:22:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 msgr2=0x7f69e4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e9f0 tx=0x7f69d400ed00 comp rx=0 tx=0).stop 2023-10-06T21:22:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 shutdown_connections 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69e410b410 0x7f69e41a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e4101ff0 0x7f69e40ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f69d00a1760 0x7f69d00a3c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 --2- 172.21.15.31:0/1400799966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4102910 0x7f69e4100180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 >> 172.21.15.31:0/1400799966 conn(0x7f69e40fb7f0 msgr2=0x7f69e410d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 shutdown_connections 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.339+0000 7f69eb7ff700 1 -- 172.21.15.31:0/1400799966 wait complete. 2023-10-06T21:22:27.342 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 29 2023-10-06T21:22:27.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T21:22:27.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 -- 172.21.15.31:0/187858501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 msgr2=0x7f8c0c10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 --2- 172.21.15.31:0/187858501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c10d1a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009990 tx=0x7f8c00009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 -- 172.21.15.31:0/187858501 shutdown_connections 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 --2- 172.21.15.31:0/187858501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c10d6e0 0x7f8c0c10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 --2- 172.21.15.31:0/187858501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c073280 0x7f8c0c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 --2- 172.21.15.31:0/187858501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c10d1a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 -- 172.21.15.31:0/187858501 >> 172.21.15.31:0/187858501 conn(0x7f8c0c0fc970 msgr2=0x7f8c0c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 -- 172.21.15.31:0/187858501 shutdown_connections 2023-10-06T21:22:27.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.583+0000 7f8c129ff700 1 -- 172.21.15.31:0/187858501 wait complete. 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.584+0000 7f8c129ff700 1 Processor -- start 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.584+0000 7f8c129ff700 1 -- start start 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c073280 0x7f8c0c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c10d6e0 0x7f8c0c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19e0e0 con 0x7f8c0c073ba0 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19e250 con 0x7f8c0c073280 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c129ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19e410 con 0x7f8c0c10d6e0 2023-10-06T21:22:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51122/0 (socket says 172.21.15.31:51122) 2023-10-06T21:22:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/3525199731 learned_addr learned my addr 172.21.15.31:0/3525199731 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c073280 0x7f8c0c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c10d6e0 0x7f8c0c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/3525199731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c10d6e0 msgr2=0x7f8c0c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c10d6e0 0x7f8c0c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/3525199731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c073280 msgr2=0x7f8c0c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c073280 0x7f8c0c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.585+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/3525199731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000095f0 con 0x7f8c0c073ba0 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8c000036c0 tx=0x7f8c00005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c097fa700 1 -- 172.21.15.31:0/3525199731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0001d070 con 0x7f8c0c073ba0 2023-10-06T21:22:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c097fa700 1 -- 172.21.15.31:0/3525199731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c00003ec0 con 0x7f8c0c073ba0 2023-10-06T21:22:27.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c19e690 con 0x7f8c0c073ba0 2023-10-06T21:22:27.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c097fa700 1 -- 172.21.15.31:0/3525199731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00022b90 con 0x7f8c0c073ba0 2023-10-06T21:22:27.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.586+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c1a5f30 con 0x7f8c0c073ba0 2023-10-06T21:22:27.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.589+0000 7f8c097fa700 1 -- 172.21.15.31:0/3525199731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8c00004030 con 0x7f8c0c073ba0 2023-10-06T21:22:27.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.591+0000 7f8c26395b80 1 Processor -- start 2023-10-06T21:22:27.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.591+0000 7f8c26395b80 1 -- start start 2023-10-06T21:22:27.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe0c0 0x561e6e1fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e20bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e167460 con 0x561e6e1fe9e0 2023-10-06T21:22:27.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e158c20 con 0x561e6e209890 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c26395b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e156540 con 0x561e6e1fe0c0 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c10f9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe0c0 0x561e6e1fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e20bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c097fa700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 0x7f8bf40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51138/0 (socket says 172.21.15.31:51138) 2023-10-06T21:22:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.592+0000 7f8c0bfff700 1 -- 172.21.15.31:0/3919202990 learned_addr learned my addr 172.21.15.31:0/3919202990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 0x7f8bf40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 -- 172.21.15.31:0/3919202990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe0c0 msgr2=0x561e6e1fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe0c0 0x561e6e1fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c097fa700 1 -- 172.21.15.31:0/3525199731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8c000ce9b0 con 0x7f8c0c073ba0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 -- 172.21.15.31:0/3919202990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 msgr2=0x561e6e20bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e20bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 -- 172.21.15.31:0/3919202990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6df7cde0 con 0x561e6e1fe9e0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00bd70 tx=0x7f8bfc0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe7f2dcaa561cec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 0x7f8bf40a3bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c04003fd0 tx=0x7f8c04004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8bf27fc700 1 -- 172.21.15.31:0/3919202990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc010e10 con 0x561e6e1fe9e0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.593+0000 7f8bf27fc700 1 -- 172.21.15.31:0/3919202990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bfc015070 con 0x561e6e1fe9e0 2023-10-06T21:22:27.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8bf27fc700 1 -- 172.21.15.31:0/3919202990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0116a0 con 0x561e6e1fe9e0 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 -- 172.21.15.31:0/3919202990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 msgr2=0x561e6e209350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00bd70 tx=0x7f8bfc0106b0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 -- 172.21.15.31:0/3919202990 shutdown_connections 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e20bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe0c0 0x561e6e1fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 --2- 172.21.15.31:0/3919202990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe9e0 0x561e6e209350 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.594+0000 7f8c26395b80 1 -- 172.21.15.31:0/3919202990 >> 172.21.15.31:0/3919202990 conn(0x561e6e173260 msgr2=0x561e6e1fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.595+0000 7f8c26395b80 1 -- 172.21.15.31:0/3919202990 shutdown_connections 2023-10-06T21:22:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.595+0000 7f8c26395b80 1 -- 172.21.15.31:0/3919202990 wait complete. 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.595+0000 7f8c26395b80 1 -- start start 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 Processor -- start 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e207020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e151e30 con 0x561e6e1fe0c0 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.596+0000 7f8c26395b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e0ee710 con 0x561e6e209890 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c26395b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e0ee3b0 con 0x561e6e1fe9e0 2023-10-06T21:22:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c26395b80 10 client.0 _ll_register_callbacks cb 0x561e6df4b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51148/0 (socket says 172.21.15.31:51148) 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 -- 172.21.15.31:0/1492563519 learned_addr learned my addr 172.21.15.31:0/1492563519 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e207020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 -- 172.21.15.31:0/1492563519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 msgr2=0x561e6e206ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 -- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 msgr2=0x561e6e207020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e207020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 -- 172.21.15.31:0/1492563519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc00b9a0 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00b6c0 tx=0x7f8bfc010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9545dc49e6269c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.597+0000 7f8beb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc01c890 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e6e1f1f10 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc004030 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8c26395b80 10 client.16083 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561e6df65c60 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.598+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bfc032020 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8bfc003ae0 con 0x561e6e1fe0c0 2023-10-06T21:22:27.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 10 client.16083 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 20 client.16083 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e2192b0 0x561e6e21b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 4 client.16083 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e6e14ae10 con 0x561e6e2192b0 2023-10-06T21:22:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.599+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e2192b0 0x561e6e21b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.600+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e2192b0 0x561e6e21b6d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a003c4ca2ed3195 server_cookie=7930ce4881ac7f03 in_seq=0 out_seq=0 2023-10-06T21:22:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.600+0000 7f8beb7fe700 10 client.16083 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:22:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.600+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1492563519 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561e6e14ae10 con 0x561e6e2192b0 2023-10-06T21:22:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.600+0000 7f8beb7fe700 10 client.16083 handle_command_reply: tid=0 2023-10-06T21:22:27.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 client.16083 shutdown 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 msgr2=0x561e6e2065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00b6c0 tx=0x7f8bfc010ec0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 shutdown_connections 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e2192b0 0x561e6e21b6d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e207020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e2065a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 --2- 172.21.15.31:0/1492563519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 >> 172.21.15.31:0/1492563519 conn(0x561e6e173260 msgr2=0x561e6e1f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8beb7fe700 0 client.16083 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 shutdown_connections 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 1 -- 172.21.15.31:0/1492563519 wait complete. 2023-10-06T21:22:27.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.601+0000 7f8c26395b80 20 client.16083 trim_cache size 0 max 16384 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 Processor -- start 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 -- start start 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e205460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6dfd2920 con 0x561e6e1fe0c0 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e204b60 con 0x561e6e209890 2023-10-06T21:22:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c26395b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e202420 con 0x561e6e1fe9e0 2023-10-06T21:22:27.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51160/0 (socket says 172.21.15.31:51160) 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.617+0000 7f8c0bfff700 1 -- 172.21.15.31:0/2870042682 learned_addr learned my addr 172.21.15.31:0/2870042682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e205460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/2870042682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 msgr2=0x561e6e205460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e205460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/2870042682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 msgr2=0x561e6e200f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/2870042682 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6e1f8e90 con 0x561e6e1fe9e0 2023-10-06T21:22:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c000cbed0 tx=0x7f8c000d0dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d0b8d2d6e96cfc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8bf2ffd700 1 -- 172.21.15.31:0/2870042682 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009b260 con 0x561e6e1fe9e0 2023-10-06T21:22:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8bf2ffd700 1 -- 172.21.15.31:0/2870042682 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0009b080 con 0x561e6e1fe9e0 2023-10-06T21:22:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.618+0000 7f8bf2ffd700 1 -- 172.21.15.31:0/2870042682 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00099ac0 con 0x561e6e1fe9e0 2023-10-06T21:22:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 -- 172.21.15.31:0/2870042682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 msgr2=0x561e6e206e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c000cbed0 tx=0x7f8c000d0dc0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 -- 172.21.15.31:0/2870042682 shutdown_connections 2023-10-06T21:22:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e209890 0x561e6e205460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e1fe0c0 0x561e6e200f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2870042682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e206e50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 -- 172.21.15.31:0/2870042682 >> 172.21.15.31:0/2870042682 conn(0x561e6e173260 msgr2=0x561e6e204d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 -- 172.21.15.31:0/2870042682 shutdown_connections 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.619+0000 7f8c26395b80 1 -- 172.21.15.31:0/2870042682 wait complete. 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 -- start start 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 Processor -- start 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e20e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 0x561e6e26c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e26c660 con 0x561e6e209890 2023-10-06T21:22:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e26c7d0 con 0x561e6e1fe0c0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.620+0000 7f8c26395b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6e26c990 con 0x561e6e1fe9e0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c26395b80 10 client.0 _ll_register_callbacks cb 0x7f8c0009b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42438/0 (socket says 172.21.15.31:42438) 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 -- 172.21.15.31:0/2961043712 learned_addr learned my addr 172.21.15.31:0/2961043712 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e20e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 0x561e6e26c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 -- 172.21.15.31:0/2961043712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 msgr2=0x561e6e20e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e20e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 -- 172.21.15.31:0/2961043712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 msgr2=0x561e6e26c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 0x561e6e26c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 -- 172.21.15.31:0/2961043712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e6df65c60 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c10f9c700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 0x561e6e26c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8c0bfff700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc010fe0 tx=0x7f8bfc011e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8a9d316c831cff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.621+0000 7f8beb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc005ee0 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bfc015070 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f8bfc000e40 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc00bec0 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8c26395b80 10 client.25829 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8c0c1a5f30 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.622+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bfc015070 con 0x561e6e1fe0c0 2023-10-06T21:22:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8bfc005410 con 0x561e6e1fe0c0 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 10 client.25829 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 20 client.25829 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e26ddf0 0x561e6e2701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 4 client.25829 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e6e14ab30 con 0x561e6e26ddf0 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.623+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e26ddf0 0x561e6e2701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c0b7fe700 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e26ddf0 0x561e6e2701e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae418b49e93be153 server_cookie=7828c3b697f0b9c0 in_seq=0 out_seq=0 2023-10-06T21:22:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8beb7fe700 10 client.25829 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:22:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8beb7fe700 1 -- 172.21.15.31:0/2961043712 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e6e14ab30 con 0x561e6e26ddf0 2023-10-06T21:22:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8beb7fe700 10 client.25829 handle_command_reply: tid=0 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 client.25829 shutdown 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 msgr2=0x561e6e20e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc010fe0 tx=0x7f8bfc011e70 comp rx=0 tx=0).stop 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 shutdown_connections 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x561e6e26ddf0 0x561e6e2701e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e6e209890 0x561e6e26c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8beb7fe700 0 client.25829 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e6e1fe0c0 0x561e6e20e270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 --2- 172.21.15.31:0/2961043712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e6e1fe9e0 0x561e6e20e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 >> 172.21.15.31:0/2961043712 conn(0x561e6e173260 msgr2=0x561e6e2055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 shutdown_connections 2023-10-06T21:22:27.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 1 -- 172.21.15.31:0/2961043712 wait complete. 2023-10-06T21:22:27.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.624+0000 7f8c26395b80 20 client.25829 trim_cache size 0 max 16384 2023-10-06T21:22:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:22:27.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 msgr2=0x7f8bf40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 0x7f8bf40a3bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c04003fd0 tx=0x7f8c04004b40 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 msgr2=0x7f8c0c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8c000036c0 tx=0x7f8c00005740 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 shutdown_connections 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c10d6e0 0x7f8c0c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c073280 0x7f8c0c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8bf40a1710 0x7f8bf40a3bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 --2- 172.21.15.31:0/3525199731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c0c073ba0 0x7f8c0c199640 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 >> 172.21.15.31:0/3525199731 conn(0x7f8c0c0fc970 msgr2=0x7f8c0c10f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 shutdown_connections 2023-10-06T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.625+0000 7f8c129ff700 1 -- 172.21.15.31:0/3525199731 wait complete. 2023-10-06T21:22:27.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T21:22:27.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 -- 172.21.15.31:0/963287653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073260 msgr2=0x7f2820073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 --2- 172.21.15.31:0/963287653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073260 0x7f2820073640 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2814009990 tx=0x7f2814009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 -- 172.21.15.31:0/963287653 shutdown_connections 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 --2- 172.21.15.31:0/963287653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282010d6c0 0x7f282010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 --2- 172.21.15.31:0/963287653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073260 0x7f2820073640 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 --2- 172.21.15.31:0/963287653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2820073b80 0x7f282010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 -- 172.21.15.31:0/963287653 >> 172.21.15.31:0/963287653 conn(0x7f28200fc910 msgr2=0x7f28200fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 -- 172.21.15.31:0/963287653 shutdown_connections 2023-10-06T21:22:27.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.861+0000 7f2826bff700 1 -- 172.21.15.31:0/963287653 wait complete. 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.862+0000 7f2826bff700 1 Processor -- start 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 -- start start 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2820073260 0x7f2820199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282010d6c0 0x7f282019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282019e010 con 0x7f2820073b80 2023-10-06T21:22:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282019e180 con 0x7f282010d6c0 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f2826bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282019e340 con 0x7f2820073260 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51190/0 (socket says 172.21.15.31:51190) 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.863+0000 7f281ffff700 1 -- 172.21.15.31:0/1360507675 learned_addr learned my addr 172.21.15.31:0/1360507675 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f282519c700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282010d6c0 0x7f282019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f282499b700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2820073260 0x7f2820199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 -- 172.21.15.31:0/1360507675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2820073260 msgr2=0x7f2820199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2820073260 0x7f2820199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 -- 172.21.15.31:0/1360507675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282010d6c0 msgr2=0x7f282019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282010d6c0 0x7f282019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 -- 172.21.15.31:0/1360507675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140095f0 con 0x7f2820073b80 2023-10-06T21:22:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281ffff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9a0 tx=0x7f280c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.864+0000 7f281dffb700 1 -- 172.21.15.31:0/1360507675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c009790 con 0x7f2820073b80 2023-10-06T21:22:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.865+0000 7f281dffb700 1 -- 172.21.15.31:0/1360507675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f280c010430 con 0x7f2820073b80 2023-10-06T21:22:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.865+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282019e620 con 0x7f2820073b80 2023-10-06T21:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.865+0000 7f281dffb700 1 -- 172.21.15.31:0/1360507675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c00cea0 con 0x7f2820073b80 2023-10-06T21:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.865+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201a5e60 con 0x7f2820073b80 2023-10-06T21:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.869+0000 7f281dffb700 1 -- 172.21.15.31:0/1360507675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f280c0105a0 con 0x7f2820073b80 2023-10-06T21:22:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 Processor -- start 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 -- start start 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309aa64460 con 0x56309aafb9e0 2023-10-06T21:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309aa55c20 con 0x56309ab06890 2023-10-06T21:22:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.870+0000 7f283a556b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309aa53540 con 0x56309aafb0c0 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f281dffb700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 0x7f28100a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51200/0 (socket says 172.21.15.31:51200) 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 -- 172.21.15.31:0/286084920 learned_addr learned my addr 172.21.15.31:0/286084920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f281dffb700 1 -- 172.21.15.31:0/1360507675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f280c014070 con 0x7f2820073b80 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282519c700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f281ffff700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282519c700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 0x7f28100a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 -- 172.21.15.31:0/286084920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 msgr2=0x56309aafb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 -- 172.21.15.31:0/286084920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 msgr2=0x56309ab08c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab08c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282499b700 1 -- 172.21.15.31:0/286084920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309a879de0 con 0x56309aafb9e0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f282519c700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.871+0000 7f281ffff700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.872+0000 7f282499b700 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2814009990 tx=0x7f2814005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d34d7003b4d3052 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.872+0000 7f282519c700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 0x7f28100a3bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28180075c0 tx=0x7f2818009100 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.872+0000 7f280affd700 1 -- 172.21.15.31:0/286084920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281401d070 con 0x56309aafb9e0 2023-10-06T21:22:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.872+0000 7f280affd700 1 -- 172.21.15.31:0/286084920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f281401e040 con 0x56309aafb9e0 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.872+0000 7f280affd700 1 -- 172.21.15.31:0/286084920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140049c0 con 0x56309aafb9e0 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 -- 172.21.15.31:0/286084920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 msgr2=0x56309ab06350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2814009990 tx=0x7f2814005040 comp rx=0 tx=0).stop 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 -- 172.21.15.31:0/286084920 shutdown_connections 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 --2- 172.21.15.31:0/286084920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab06350 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 -- 172.21.15.31:0/286084920 >> 172.21.15.31:0/286084920 conn(0x56309aa70260 msgr2=0x56309aaf7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 -- 172.21.15.31:0/286084920 shutdown_connections 2023-10-06T21:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.873+0000 7f283a556b80 1 -- 172.21.15.31:0/286084920 wait complete. 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.874+0000 7f283a556b80 1 -- start start 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.874+0000 7f283a556b80 1 Processor -- start 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb9e0 0x56309ab034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab03c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309aa4ee30 con 0x56309aafb0c0 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309a9eb710 con 0x56309ab06890 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309a9eb3b0 con 0x56309aafb9e0 2023-10-06T21:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f283a556b80 10 client.0 _ll_register_callbacks cb 0x56309a848800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f282499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f282499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51212/0 (socket says 172.21.15.31:51212) 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f282499b700 1 -- 172.21.15.31:0/3378562581 learned_addr learned my addr 172.21.15.31:0/3378562581 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f282519c700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab03c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.875+0000 7f281ffff700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb9e0 0x56309ab034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 -- 172.21.15.31:0/3378562581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb9e0 msgr2=0x56309ab034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb9e0 0x56309ab034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 -- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 msgr2=0x56309ab03c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab03c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 -- 172.21.15.31:0/3378562581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2820110770 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282519c700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab03c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f282499b700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2814009ce0 tx=0x7f281400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dccbda5624cd60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f27fffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281401d030 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56309aaeef10 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.876+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2814004840 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.877+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2814004a00 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.877+0000 7f283a556b80 10 client.16101 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.877+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56309a862c60 con 0x56309aafb0c0 2023-10-06T21:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.877+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56309aaeef10 con 0x56309aafb0c0 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.877+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f281400fab0 con 0x56309aafb0c0 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 10 client.16101 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 20 client.16101 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab15d90 0x56309ab18180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 4 client.16101 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56309aa47e10 con 0x56309ab15d90 2023-10-06T21:22:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.878+0000 7f281ffff700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab15d90 0x56309ab18180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.879+0000 7f281ffff700 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab15d90 0x56309ab18180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5fba14fde4c5f94 server_cookie=d021a999e3883a7f in_seq=0 out_seq=0 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.879+0000 7f27fffff700 10 client.16101 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.879+0000 7f27fffff700 1 -- 172.21.15.31:0/3378562581 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56309aa47e10 con 0x56309ab15d90 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.879+0000 7f27fffff700 10 client.16101 handle_command_reply: tid=0 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.879+0000 7f283a556b80 1 client.16101 shutdown 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 msgr2=0x56309ab062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2814009ce0 tx=0x7f281400be20 comp rx=0 tx=0).stop 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 shutdown_connections 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab15d90 0x56309ab18180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab03c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb0c0 0x56309ab062e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 --2- 172.21.15.31:0/3378562581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb9e0 0x56309ab034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 >> 172.21.15.31:0/3378562581 conn(0x56309aa70260 msgr2=0x56309aaf6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f27fffff700 0 client.16101 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 shutdown_connections 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 1 -- 172.21.15.31:0/3378562581 wait complete. 2023-10-06T21:22:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.880+0000 7f283a556b80 20 client.16101 trim_cache size 0 max 16384 2023-10-06T21:22:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.893+0000 7f283a556b80 1 Processor -- start 2023-10-06T21:22:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 -- start start 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab02d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309a8cf920 con 0x56309aafb9e0 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309aaf1520 con 0x56309ab06890 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f283a556b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ab04840 con 0x56309aafb0c0 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f282499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51218/0 (socket says 172.21.15.31:51218) 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f281ffff700 1 -- 172.21.15.31:0/512192487 learned_addr learned my addr 172.21.15.31:0/512192487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.894+0000 7f282519c700 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab02d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 -- 172.21.15.31:0/512192487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 msgr2=0x56309aafdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 -- 172.21.15.31:0/512192487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 msgr2=0x56309ab02d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab02d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 -- 172.21.15.31:0/512192487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309a8a0250 con 0x56309aafb9e0 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f282519c700 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab02d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f281ffff700 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f280c096d80 tx=0x7f280c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd6114ef6bec8d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f280b7fe700 1 -- 172.21.15.31:0/512192487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c09aa10 con 0x56309aafb9e0 2023-10-06T21:22:27.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f280b7fe700 1 -- 172.21.15.31:0/512192487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f280c09a310 con 0x56309aafb9e0 2023-10-06T21:22:27.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.895+0000 7f280b7fe700 1 -- 172.21.15.31:0/512192487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c099800 con 0x56309aafb9e0 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- 172.21.15.31:0/512192487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 msgr2=0x56309ab05e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f280c096d80 tx=0x7f280c0cf040 comp rx=0 tx=0).stop 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- 172.21.15.31:0/512192487 shutdown_connections 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab02d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309aafdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 --2- 172.21.15.31:0/512192487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab05e10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- 172.21.15.31:0/512192487 >> 172.21.15.31:0/512192487 conn(0x56309aa70260 msgr2=0x56309ab12610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- 172.21.15.31:0/512192487 shutdown_connections 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- 172.21.15.31:0/512192487 wait complete. 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.896+0000 7f283a556b80 1 -- start start 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 Processor -- start 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309ab0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ab63080 con 0x56309aafb9e0 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ab631f0 con 0x56309ab06890 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ab633b0 con 0x56309aafb0c0 2023-10-06T21:22:27.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.897+0000 7f283a556b80 10 client.0 _ll_register_callbacks cb 0x7f2814005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309ab0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282519c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f281ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51224/0 (socket says 172.21.15.31:51224) 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f281ffff700 1 -- 172.21.15.31:0/3027748040 learned_addr learned my addr 172.21.15.31:0/3027748040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 -- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 msgr2=0x56309ab62b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab62b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 -- 172.21.15.31:0/3027748040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 msgr2=0x56309ab0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 -- 172.21.15.31:0/3027748040 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201a5e60 con 0x56309aafb0c0 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f281ffff700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f282499b700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309ab0b550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2814006010 tx=0x7f2814022a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315f3aaa008ca915 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.898+0000 7f27fffff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281401d070 con 0x56309aafb0c0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f281401e040 con 0x56309aafb0c0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2814005910 con 0x56309aafb0c0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2814022400 con 0x56309aafb0c0 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f283a556b80 10 client.25804 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.899+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56309a8782b0 con 0x56309aafb0c0 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f281401e040 con 0x56309aafb0c0 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2814049760 con 0x56309aafb0c0 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 10 client.25804 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 20 client.25804 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab6a680 0x56309ab6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 4 client.25804 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:22:27.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56309aa47b30 con 0x56309ab6a680 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.900+0000 7f281ffff700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab6a680 0x56309ab6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f281ffff700 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab6a680 0x56309ab6ca70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=568b1a4732a98e8a server_cookie=c86c84277701c7fe in_seq=0 out_seq=0 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f27fffff700 10 client.25804 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f27fffff700 1 -- 172.21.15.31:0/3027748040 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56309aa47b30 con 0x56309ab6a680 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f27fffff700 10 client.25804 handle_command_reply: tid=0 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 client.25804 shutdown 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 msgr2=0x56309ab0b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309ab0b550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2814006010 tx=0x7f2814022a60 comp rx=0 tx=0).stop 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 shutdown_connections 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x56309ab6a680 0x56309ab6ca70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56309ab06890 0x56309ab62b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f27fffff700 0 client.25804 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309aafb0c0 0x56309ab0b550 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 --2- 172.21.15.31:0/3027748040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56309aafb9e0 0x56309ab0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 >> 172.21.15.31:0/3027748040 conn(0x56309aa70260 msgr2=0x56309aaf6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 shutdown_connections 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 1 -- 172.21.15.31:0/3027748040 wait complete. 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.901+0000 7f283a556b80 20 client.25804 trim_cache size 0 max 16384 2023-10-06T21:22:27.908 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 msgr2=0x7f28100a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 0x7f28100a3bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28180075c0 tx=0x7f2818009100 comp rx=0 tx=0).stop 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 msgr2=0x7f2820199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9a0 tx=0x7f280c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 shutdown_connections 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282010d6c0 0x7f282019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2820073260 0x7f2820199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f28100a1710 0x7f28100a3bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 --2- 172.21.15.31:0/1360507675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2820073b80 0x7f2820199570 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 >> 172.21.15.31:0/1360507675 conn(0x7f28200fc910 msgr2=0x7f282010f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 shutdown_connections 2023-10-06T21:22:27.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:27.902+0000 7f2826bff700 1 -- 172.21.15.31:0/1360507675 wait complete. 2023-10-06T21:22:27.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:22:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4154507623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf8010b2c0 msgr2=0x7fcf8010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4154507623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf8010b2c0 0x7fcf8010d780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b0a0 tx=0x7fcf7800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4154507623 shutdown_connections 2023-10-06T21:22:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4154507623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf8010b2c0 0x7fcf8010d780 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4154507623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf80101ea0 0x7fcf80102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4154507623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf801027c0 0x7fcf8010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4154507623 >> 172.21.15.31:0/4154507623 conn(0x7fcf800fb6e0 msgr2=0x7fcf800fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4154507623 shutdown_connections 2023-10-06T21:22:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.151+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4154507623 wait complete. 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 Processor -- start 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 -- start start 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801027c0 0x7fcf80199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf8010b2c0 0x7fcf8019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019df30 con 0x7fcf80101ea0 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019e0a0 con 0x7fcf801027c0 2023-10-06T21:22:28.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.152+0000 7fcf7ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019e260 con 0x7fcf8010b2c0 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801027c0 0x7fcf80199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51240/0 (socket says 172.21.15.31:51240) 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 -- 172.21.15.31:0/4233490799 learned_addr learned my addr 172.21.15.31:0/4233490799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7f7fe700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf8010b2c0 0x7fcf8019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 -- 172.21.15.31:0/4233490799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf8010b2c0 msgr2=0x7fcf8019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf8010b2c0 0x7fcf8019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 -- 172.21.15.31:0/4233490799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801027c0 msgr2=0x7fcf80199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801027c0 0x7fcf80199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 -- 172.21.15.31:0/4233490799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78009d00 con 0x7fcf80101ea0 2023-10-06T21:22:28.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.153+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000e9a0 tx=0x7fcf7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.154+0000 7fcf67fff700 1 -- 172.21.15.31:0/4233490799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70009790 con 0x7fcf80101ea0 2023-10-06T21:22:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.154+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8019e540 con 0x7fcf80101ea0 2023-10-06T21:22:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.154+0000 7fcf67fff700 1 -- 172.21.15.31:0/4233490799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf70010430 con 0x7fcf80101ea0 2023-10-06T21:22:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.154+0000 7fcf67fff700 1 -- 172.21.15.31:0/4233490799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000cea0 con 0x7fcf80101ea0 2023-10-06T21:22:28.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.154+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801a5d80 con 0x7fcf80101ea0 2023-10-06T21:22:28.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.158+0000 7fcf67fff700 1 -- 172.21.15.31:0/4233490799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fcf700105a0 con 0x7fcf80101ea0 2023-10-06T21:22:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.159+0000 7fcf98eadb80 1 Processor -- start 2023-10-06T21:22:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.159+0000 7fcf67fff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 0x7fcf680a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.159+0000 7fcf98eadb80 1 -- start start 2023-10-06T21:22:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.159+0000 7fcf67fff700 1 -- 172.21.15.31:0/4233490799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcf70014070 con 0x7fcf80101ea0 2023-10-06T21:22:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.159+0000 7fcf7f7fe700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 0x7fcf680a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c360ea70 0x55e1c3610e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c356c460 con 0x55e1c3603bc0 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c355dc20 con 0x55e1c36032a0 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c355b540 con 0x55e1c360ea70 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42494/0 (socket says 172.21.15.31:42494) 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 -- 172.21.15.31:0/4028030077 learned_addr learned my addr 172.21.15.31:0/4028030077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7f7fe700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 0x7fcf680a3c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009fd0 tx=0x7fcf7800b720 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 -- 172.21.15.31:0/4028030077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c360ea70 msgr2=0x55e1c3610e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c360ea70 0x55e1c3610e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 -- 172.21.15.31:0/4028030077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 msgr2=0x55e1c360e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.160+0000 7fcf7effd700 1 -- 172.21.15.31:0/4028030077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1c3381de0 con 0x55e1c36032a0 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.161+0000 7fcf7effd700 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf7009a2a0 tx=0x7fcf70099d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7643a0cb4ad65ee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.161+0000 7fcf657fa700 1 -- 172.21.15.31:0/4028030077 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7009aa10 con 0x55e1c36032a0 2023-10-06T21:22:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.161+0000 7fcf657fa700 1 -- 172.21.15.31:0/4028030077 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf7009a310 con 0x55e1c36032a0 2023-10-06T21:22:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.161+0000 7fcf657fa700 1 -- 172.21.15.31:0/4028030077 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70099ad0 con 0x55e1c36032a0 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- 172.21.15.31:0/4028030077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 msgr2=0x55e1c3603680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf7009a2a0 tx=0x7fcf70099d70 comp rx=0 tx=0).stop 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- 172.21.15.31:0/4028030077 shutdown_connections 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c360ea70 0x55e1c3610e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3603680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/4028030077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- 172.21.15.31:0/4028030077 >> 172.21.15.31:0/4028030077 conn(0x55e1c3578260 msgr2=0x55e1c35ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- 172.21.15.31:0/4028030077 shutdown_connections 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- 172.21.15.31:0/4028030077 wait complete. 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.162+0000 7fcf98eadb80 1 -- start start 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 Processor -- start 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c360b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c360ea70 0x55e1c360c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c3556e30 con 0x55e1c3603bc0 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c34f3710 con 0x55e1c360ea70 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c34f33b0 con 0x55e1c36032a0 2023-10-06T21:22:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf98eadb80 10 client.0 _ll_register_callbacks cb 0x55e1c3350800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.163+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c360b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c360ea70 0x55e1c360c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51254/0 (socket says 172.21.15.31:51254) 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2025528958 learned_addr learned my addr 172.21.15.31:0/2025528958 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 msgr2=0x55e1c360b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c360b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2025528958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c360ea70 msgr2=0x55e1c360c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c360ea70 0x55e1c360c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2025528958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801a5d80 con 0x55e1c3603bc0 2023-10-06T21:22:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcf74012680 tx=0x7fcf74012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c731d390144aa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.164+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017070 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1c35f7160 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcf74010430 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf740122e0 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf98eadb80 10 client.16119 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e1c336ac60 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1c35f7160 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.165+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf7401c780 con 0x55e1c3603bc0 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 10 client.16119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:22:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 20 client.16119 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c361e620 0x55e1c3620a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 4 client.16119 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1c354fe10 con 0x55e1c361e620 2023-10-06T21:22:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.166+0000 7fcf7effd700 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c361e620 0x55e1c3620a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.167+0000 7fcf62ff5700 10 client.16119 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.167+0000 7fcf7effd700 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c361e620 0x55e1c3620a40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a27e2d6789b7ca49 server_cookie=86da6498bc9f80b in_seq=0 out_seq=0 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.167+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2025528958 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1c354fe10 con 0x55e1c361e620 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.167+0000 7fcf62ff5700 10 client.16119 handle_command_reply: tid=0 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.167+0000 7fcf98eadb80 1 client.16119 shutdown 2023-10-06T21:22:28.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 msgr2=0x55e1c360bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcf74012680 tx=0x7fcf74012990 comp rx=0 tx=0).stop 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 shutdown_connections 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c361e620 0x55e1c3620a40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c360ea70 0x55e1c360c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c360b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2025528958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c3603bc0 0x55e1c360bcf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf62ff5700 0 client.16119 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 >> 172.21.15.31:0/2025528958 conn(0x55e1c3578260 msgr2=0x55e1c35ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 shutdown_connections 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2025528958 wait complete. 2023-10-06T21:22:28.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.168+0000 7fcf98eadb80 20 client.16119 trim_cache size 0 max 16384 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 Processor -- start 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 -- start start 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 0x55e1c360c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c360a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c33d7920 con 0x55e1c360ea70 2023-10-06T21:22:28.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c35f9720 con 0x55e1c3603bc0 2023-10-06T21:22:28.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.183+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c3607640 con 0x55e1c36032a0 2023-10-06T21:22:28.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35902/0 (socket says 172.21.15.31:35902) 2023-10-06T21:22:28.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 -- 172.21.15.31:0/1317054702 learned_addr learned my addr 172.21.15.31:0/1317054702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7f7fe700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c360a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 0x55e1c360c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 -- 172.21.15.31:0/1317054702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 msgr2=0x55e1c360c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 0x55e1c360c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 -- 172.21.15.31:0/1317054702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 msgr2=0x55e1c360a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c360a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 -- 172.21.15.31:0/1317054702 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1c35fe0b0 con 0x55e1c36032a0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 0x55e1c360c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.184+0000 7fcf7effd700 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf7009a2a0 tx=0x7fcf70099cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5a5491a16385ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf65ffb700 1 -- 172.21.15.31:0/1317054702 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700b8070 con 0x55e1c36032a0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf65ffb700 1 -- 172.21.15.31:0/1317054702 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf7009e460 con 0x55e1c36032a0 2023-10-06T21:22:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf65ffb700 1 -- 172.21.15.31:0/1317054702 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700a2210 con 0x55e1c36032a0 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 -- 172.21.15.31:0/1317054702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 msgr2=0x55e1c36061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf7009a2a0 tx=0x7fcf70099cf0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 -- 172.21.15.31:0/1317054702 shutdown_connections 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c360a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c36032a0 0x55e1c36061a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/1317054702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c3603bc0 0x55e1c360c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 -- 172.21.15.31:0/1317054702 >> 172.21.15.31:0/1317054702 conn(0x55e1c3578260 msgr2=0x55e1c3609d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.185+0000 7fcf98eadb80 1 -- 172.21.15.31:0/1317054702 shutdown_connections 2023-10-06T21:22:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.186+0000 7fcf98eadb80 1 -- 172.21.15.31:0/1317054702 wait complete. 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.186+0000 7fcf98eadb80 1 -- start start 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.186+0000 7fcf98eadb80 1 Processor -- start 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3613790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c3671450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c3671990 con 0x55e1c360ea70 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c3671ad0 con 0x55e1c36032a0 2023-10-06T21:22:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c3671c40 con 0x55e1c3603bc0 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf98eadb80 10 client.0 _ll_register_callbacks cb 0x7fcf74009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf7e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35914/0 (socket says 172.21.15.31:35914) 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2403278251 learned_addr learned my addr 172.21.15.31:0/2403278251 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf7effd700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3613790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.187+0000 7fcf7f7fe700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c3671450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2403278251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 msgr2=0x55e1c3613790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3613790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2403278251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 msgr2=0x55e1c3671450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c3671450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 -- 172.21.15.31:0/2403278251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1c3381de0 con 0x55e1c3603bc0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf7e7fc700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf7401cbe0 tx=0x7fcf74005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d9c18914fdc2e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017040 con 0x55e1c3603bc0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf7009a310 con 0x55e1c3603bc0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.188+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcf74010840 con 0x55e1c3603bc0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf740122e0 con 0x55e1c3603bc0 2023-10-06T21:22:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 10 client.25819 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e1c33a8250 con 0x55e1c3603bc0 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf7009a310 con 0x55e1c3603bc0 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf7402d470 con 0x55e1c3603bc0 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 10 client.25819 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 10 client.25819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 20 client.25819 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c36732c0 0x55e1c36756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 4 client.25819 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.189+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1c354fb30 con 0x55e1c36732c0 2023-10-06T21:22:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf7effd700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c36732c0 0x55e1c36756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf7effd700 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c36732c0 0x55e1c36756e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92030cebc480e189 server_cookie=f5f0018f6830b063 in_seq=0 out_seq=0 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf62ff5700 10 client.25819 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf62ff5700 1 -- 172.21.15.31:0/2403278251 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1c354fb30 con 0x55e1c36732c0 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf62ff5700 10 client.25819 handle_command_reply: tid=0 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.190+0000 7fcf98eadb80 1 client.25819 shutdown 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 msgr2=0x55e1c3613cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf7401cbe0 tx=0x7fcf74005dc0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 shutdown_connections 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x55e1c36732c0 0x55e1c36756e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e1c360ea70 0x55e1c3671450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1c36032a0 0x55e1c3613790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 --2- 172.21.15.31:0/2403278251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1c3603bc0 0x55e1c3613cd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf62ff5700 0 client.25819 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 >> 172.21.15.31:0/2403278251 conn(0x55e1c3578260 msgr2=0x55e1c360a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 shutdown_connections 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 1 -- 172.21.15.31:0/2403278251 wait complete. 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf98eadb80 20 client.25819 trim_cache size 0 max 16384 2023-10-06T21:22:28.198 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:22:28.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.191+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 msgr2=0x7fcf680a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 0x7fcf680a3c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009fd0 tx=0x7fcf7800b720 comp rx=0 tx=0).stop 2023-10-06T21:22:28.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 msgr2=0x7fcf80198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000e9a0 tx=0x7fcf7000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 shutdown_connections 2023-10-06T21:22:28.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf8010b2c0 0x7fcf8019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcf680a17b0 0x7fcf680a3c70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf80101ea0 0x7fcf80198f50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 --2- 172.21.15.31:0/4233490799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf801027c0 0x7fcf80199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 >> 172.21.15.31:0/4233490799 conn(0x7fcf800fb6e0 msgr2=0x7fcf800ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 shutdown_connections 2023-10-06T21:22:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.192+0000 7fcf7ffff700 1 -- 172.21.15.31:0/4233490799 wait complete. 2023-10-06T21:22:28.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:28 smithi031 ceph-mon[98539]: pgmap v48: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 811 KiB/s wr, 82 op/s 2023-10-06T21:22:28.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:28 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/1400799966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:22:28.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:22:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:28 smithi146 ceph-mon[102809]: pgmap v48: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 811 KiB/s wr, 82 op/s 2023-10-06T21:22:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:28 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/1400799966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:22:28.327 INFO:tasks.workunit.client.0.smithi031.stdout: 1 7241 3.21 MB/sec warmup 18 sec latency 19.765 ms 2023-10-06T21:22:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:28 smithi142 ceph-mon[102951]: pgmap v48: 97 pgs: 97 active+clean; 24 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 811 KiB/s wr, 82 op/s 2023-10-06T21:22:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:28 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/1400799966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:22:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.439+0000 7f3ed7bff700 1 -- 172.21.15.31:0/2313030452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 msgr2=0x7f3ed010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.439+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/2313030452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed010dd60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800b0a0 tx=0x7f3ec800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 -- 172.21.15.31:0/2313030452 shutdown_connections 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/2313030452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed010dd60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/2313030452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 0x7f3ed01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/2313030452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0101a70 0x7f3ed0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 -- 172.21.15.31:0/2313030452 >> 172.21.15.31:0/2313030452 conn(0x7f3ed00fc960 msgr2=0x7f3ed00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 -- 172.21.15.31:0/2313030452 shutdown_connections 2023-10-06T21:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.440+0000 7f3ed7bff700 1 -- 172.21.15.31:0/2313030452 wait complete. 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.441+0000 7f3ed7bff700 1 Processor -- start 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 -- start start 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 0x7f3ed01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e5f0 con 0x7f3ed01061e0 2023-10-06T21:22:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e760 con 0x7f3ed019db70 2023-10-06T21:22:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.442+0000 7f3ed7bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed019e920 con 0x7f3ed01010c0 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 0x7f3ed01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42522/0 (socket says 172.21.15.31:42522) 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 -- 172.21.15.31:0/4001018612 learned_addr learned my addr 172.21.15.31:0/4001018612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed519a700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 -- 172.21.15.31:0/4001018612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 msgr2=0x7f3ed01991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 0x7f3ed01991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 -- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 msgr2=0x7f3ed0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 -- 172.21.15.31:0/4001018612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8009d00 con 0x7f3ed019db70 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.443+0000 7f3ed619c700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8008f20 tx=0x7f3ec8008f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.444+0000 7f3ec6ffd700 1 -- 172.21.15.31:0/4001018612 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800e040 con 0x7f3ed019db70 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.444+0000 7f3ec6ffd700 1 -- 172.21.15.31:0/4001018612 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec8003c00 con 0x7f3ed019db70 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.444+0000 7f3ec6ffd700 1 -- 172.21.15.31:0/4001018612 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8005040 con 0x7f3ed019db70 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.444+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed01a5e20 con 0x7f3ed019db70 2023-10-06T21:22:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.444+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed01a62a0 con 0x7f3ed019db70 2023-10-06T21:22:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.448+0000 7f3ec6ffd700 1 -- 172.21.15.31:0/4001018612 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f3ec801d470 con 0x7f3ed019db70 2023-10-06T21:22:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.449+0000 7f3eeb5d2b80 1 Processor -- start 2023-10-06T21:22:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.449+0000 7f3ec6ffd700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 0x7f3ebc0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 -- start start 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3ec6ffd700 1 -- 172.21.15.31:0/4001018612 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ec80ce600 con 0x7f3ed019db70 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3ed519a700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 0x7f3ebc0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d742349e0 0x559d7423f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d7423f890 0x559d74241c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7419d460 con 0x559d742349e0 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7418ec20 con 0x559d7423f890 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7418c540 con 0x559d742340c0 2023-10-06T21:22:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.450+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35938/0 (socket says 172.21.15.31:35938) 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 -- 172.21.15.31:0/3902363663 learned_addr learned my addr 172.21.15.31:0/3902363663 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d742349e0 0x559d7423f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed619c700 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d7423f890 0x559d74241c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 -- 172.21.15.31:0/3902363663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d7423f890 msgr2=0x559d74241c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d7423f890 0x559d74241c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 -- 172.21.15.31:0/3902363663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d742349e0 msgr2=0x559d7423f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d742349e0 0x559d7423f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 -- 172.21.15.31:0/3902363663 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d73fb2de0 con 0x559d742340c0 2023-10-06T21:22:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed519a700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 0x7f3ebc0a3c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ed019a7e0 tx=0x7f3ec0011440 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.451+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009d80 tx=0x7f3ecc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06e30dde2df82ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.452+0000 7f3ebbfff700 1 -- 172.21.15.31:0/3902363663 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004230 con 0x559d742340c0 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.452+0000 7f3ebbfff700 1 -- 172.21.15.31:0/3902363663 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ecc004390 con 0x559d742340c0 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.452+0000 7f3ebbfff700 1 -- 172.21.15.31:0/3902363663 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc019800 con 0x559d742340c0 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.452+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3902363663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 msgr2=0x559d742344a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.452+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009d80 tx=0x7f3ecc011720 comp rx=0 tx=0).stop 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3902363663 shutdown_connections 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d7423f890 0x559d74241c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742344a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3902363663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d742349e0 0x559d7423f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3902363663 >> 172.21.15.31:0/3902363663 conn(0x559d741a9260 msgr2=0x559d74230450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3902363663 shutdown_connections 2023-10-06T21:22:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3902363663 wait complete. 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.453+0000 7f3eeb5d2b80 1 -- start start 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 Processor -- start 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d74187e30 con 0x559d7423f890 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d74124710 con 0x559d742349e0 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d741243b0 con 0x559d742340c0 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.454+0000 7f3eeb5d2b80 10 client.0 _ll_register_callbacks cb 0x559d73f81800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42534/0 (socket says 172.21.15.31:42534) 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 -- 172.21.15.31:0/3466184228 learned_addr learned my addr 172.21.15.31:0/3466184228 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed619c700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 -- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 msgr2=0x559d7423f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 -- 172.21.15.31:0/3466184228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 msgr2=0x559d7423cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 -- 172.21.15.31:0/3466184228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc0099e0 con 0x559d742349e0 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed619c700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ec001e9a0 tx=0x7f3ec00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b89885ad6885934 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.455+0000 7f3eb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:22:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000f9d0 con 0x559d742349e0 2023-10-06T21:22:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d74227f10 con 0x559d742349e0 2023-10-06T21:22:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ec000fb30 con 0x559d742349e0 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ec000fd30 con 0x559d742349e0 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eeb5d2b80 10 client.25874 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559d73f9bc60 con 0x559d742349e0 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d74227f10 con 0x559d742349e0 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.456+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3ec00263f0 con 0x559d742349e0 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 10 client.25874 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 10 client.25874 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:22:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 20 client.25874 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d7424f3f0 0x559d74251810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 4 client.25874 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d74180e10 con 0x559d7424f3f0 2023-10-06T21:22:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.457+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d7424f3f0 0x559d74251810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.458+0000 7f3eb97fa700 10 client.25874 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.458+0000 7f3ed599b700 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d7424f3f0 0x559d74251810 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8c71b892ca9968 server_cookie=5613d49a8787e30d in_seq=0 out_seq=0 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.458+0000 7f3eb97fa700 1 -- 172.21.15.31:0/3466184228 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d74180e10 con 0x559d7424f3f0 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.458+0000 7f3eb97fa700 10 client.25874 handle_command_reply: tid=0 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.458+0000 7f3eeb5d2b80 1 client.25874 shutdown 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 msgr2=0x559d7423c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ec001e9a0 tx=0x7f3ec00047c0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 shutdown_connections 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d7424f3f0 0x559d74251810 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eb97fa700 0 client.25874 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3466184228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423c560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 >> 172.21.15.31:0/3466184228 conn(0x559d741a9260 msgr2=0x559d7422ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 shutdown_connections 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3466184228 wait complete. 2023-10-06T21:22:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.459+0000 7f3eeb5d2b80 20 client.25874 trim_cache size 0 max 16384 2023-10-06T21:22:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.473+0000 7f3eeb5d2b80 1 Processor -- start 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 -- start start 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d74008920 con 0x559d7423f890 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d74235be0 con 0x559d742349e0 2023-10-06T21:22:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d742312f0 con 0x559d742340c0 2023-10-06T21:22:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35954/0 (socket says 172.21.15.31:35954) 2023-10-06T21:22:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.474+0000 7f3ed599b700 1 -- 172.21.15.31:0/3108821920 learned_addr learned my addr 172.21.15.31:0/3108821920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed619c700 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 -- 172.21.15.31:0/3108821920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 msgr2=0x559d7423ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 -- 172.21.15.31:0/3108821920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 msgr2=0x559d7423c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.475+0000 7f3ed519a700 1 -- 172.21.15.31:0/3108821920 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d73fd9250 con 0x559d742349e0 2023-10-06T21:22:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.476+0000 7f3ed519a700 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423bd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0026750 tx=0x7f3ec0011ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbd01d2bc91de19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.476+0000 7f3ec4bff700 1 -- 172.21.15.31:0/3108821920 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00269c0 con 0x559d742349e0 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.476+0000 7f3ec4bff700 1 -- 172.21.15.31:0/3108821920 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec0026b20 con 0x559d742349e0 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.476+0000 7f3ec4bff700 1 -- 172.21.15.31:0/3108821920 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec002e7e0 con 0x559d742349e0 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3108821920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 msgr2=0x559d7423bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423bd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0026750 tx=0x7f3ec0011ad0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3108821920 shutdown_connections 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d7423c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d7423ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/3108821920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d7423bd40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3108821920 >> 172.21.15.31:0/3108821920 conn(0x559d741a9260 msgr2=0x559d742378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3108821920 shutdown_connections 2023-10-06T21:22:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.477+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/3108821920 wait complete. 2023-10-06T21:22:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 -- start start 2023-10-06T21:22:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 Processor -- start 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d74244af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d742a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d742451d0 con 0x559d7423f890 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d742a2760 con 0x559d742349e0 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d742a28a0 con 0x559d742340c0 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.478+0000 7f3eeb5d2b80 10 client.0 _ll_register_callbacks cb 0x7f3ec001e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed619c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d742a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35966/0 (socket says 172.21.15.31:35966) 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 -- 172.21.15.31:0/422889502 learned_addr learned my addr 172.21.15.31:0/422889502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed519a700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d74244af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 -- 172.21.15.31:0/422889502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 msgr2=0x559d74244af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d74244af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 -- 172.21.15.31:0/422889502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 msgr2=0x559d742a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d742a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed599b700 1 -- 172.21.15.31:0/422889502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d73f9bc60 con 0x559d742340c0 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.479+0000 7f3ed619c700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d742a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3ed599b700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009f70 tx=0x7f3ecc009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd26b5cb7c3fe13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc02a040 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3ecc02b240 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ecc004390 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ecc004500 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eeb5d2b80 10 client.25837 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559d73fb12b0 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.480+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ecc004390 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3ecc0051d0 con 0x559d742340c0 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 10 client.25837 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 10 client.25837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:22:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 20 client.25837 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d742a3f60 0x559d742a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 4 client.25837 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d74180b30 con 0x559d742a3f60 2023-10-06T21:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.481+0000 7f3ed519a700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d742a3f60 0x559d742a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eb97fa700 10 client.25837 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3ed519a700 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d742a3f60 0x559d742a6380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=123f5ce732db6424 server_cookie=769242b19e69ddad in_seq=0 out_seq=0 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eb97fa700 1 -- 172.21.15.31:0/422889502 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d74180b30 con 0x559d742a3f60 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eb97fa700 10 client.25837 handle_command_reply: tid=0 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 client.25837 shutdown 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 msgr2=0x559d742445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009f70 tx=0x7f3ecc009fa0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 shutdown_connections 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x559d742a3f60 0x559d742a6380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d7423f890 0x559d742a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d742340c0 0x559d742445b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eb97fa700 0 client.25837 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 --2- 172.21.15.31:0/422889502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d742349e0 0x559d74244af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 >> 172.21.15.31:0/422889502 conn(0x559d741a9260 msgr2=0x559d74236020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 shutdown_connections 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.482+0000 7f3eeb5d2b80 1 -- 172.21.15.31:0/422889502 wait complete. 2023-10-06T21:22:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3eeb5d2b80 20 client.25837 trim_cache size 0 max 16384 2023-10-06T21:22:28.489 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 msgr2=0x7f3ebc0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 0x7f3ebc0a3c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ed019a7e0 tx=0x7f3ec0011440 comp rx=0 tx=0).stop 2023-10-06T21:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 msgr2=0x7f3ed019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8008f20 tx=0x7f3ec8008f50 comp rx=0 tx=0).stop 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 shutdown_connections 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed019db70 0x7f3ed019dff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed01010c0 0x7f3ed01991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3ebc0a1790 0x7f3ebc0a3c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 --2- 172.21.15.31:0/4001018612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed01061e0 0x7f3ed0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 >> 172.21.15.31:0/4001018612 conn(0x7f3ed00fc960 msgr2=0x7f3ed00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 shutdown_connections 2023-10-06T21:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.483+0000 7f3ed7bff700 1 -- 172.21.15.31:0/4001018612 wait complete. 2023-10-06T21:22:28.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:22:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 -- 172.21.15.31:0/3763528270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba0101ff0 msgr2=0x7fcba01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 --2- 172.21.15.31:0/3763528270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba0101ff0 0x7fcba01023d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009990 tx=0x7fcb94009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 -- 172.21.15.31:0/3763528270 shutdown_connections 2023-10-06T21:22:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 --2- 172.21.15.31:0/3763528270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba010b410 0x7fcba010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 --2- 172.21.15.31:0/3763528270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba0101ff0 0x7fcba01023d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 --2- 172.21.15.31:0/3763528270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0102910 0x7fcba010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 -- 172.21.15.31:0/3763528270 >> 172.21.15.31:0/3763528270 conn(0x7fcba00fb830 msgr2=0x7fcba00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 -- 172.21.15.31:0/3763528270 shutdown_connections 2023-10-06T21:22:28.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.725+0000 7fcba6bff700 1 -- 172.21.15.31:0/3763528270 wait complete. 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 Processor -- start 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 -- start start 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba0102910 0x7fcba0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba010b410 0x7fcba019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.726+0000 7fcba6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019bed0 con 0x7fcba010b410 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba6bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019c040 con 0x7fcba0102910 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba6bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019c200 con 0x7fcba0101ff0 2023-10-06T21:22:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35990/0 (socket says 172.21.15.31:35990) 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 -- 172.21.15.31:0/4195712431 learned_addr learned my addr 172.21.15.31:0/4195712431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba519c700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba010b410 0x7fcba019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba0102910 0x7fcba0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 -- 172.21.15.31:0/4195712431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba0102910 msgr2=0x7fcba0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba0102910 0x7fcba0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 -- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba010b410 msgr2=0x7fcba019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba010b410 0x7fcba019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 -- 172.21.15.31:0/4195712431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940095f0 con 0x7fcba0101ff0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.727+0000 7fcba499b700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb94005740 tx=0x7fcb94005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.728+0000 7fcb9dffb700 1 -- 172.21.15.31:0/4195712431 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9401d070 con 0x7fcba0101ff0 2023-10-06T21:22:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.728+0000 7fcb9dffb700 1 -- 172.21.15.31:0/4195712431 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb94022470 con 0x7fcba0101ff0 2023-10-06T21:22:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.728+0000 7fcb9dffb700 1 -- 172.21.15.31:0/4195712431 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94017650 con 0x7fcba0101ff0 2023-10-06T21:22:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.728+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba019c480 con 0x7fcba0101ff0 2023-10-06T21:22:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.728+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba019c700 con 0x7fcba0101ff0 2023-10-06T21:22:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.731+0000 7fcb9dffb700 1 -- 172.21.15.31:0/4195712431 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 24) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fcb940225e0 con 0x7fcba0101ff0 2023-10-06T21:22:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.733+0000 7fcb9dffb700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 0x7fcb900a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.733+0000 7fcb9dffb700 1 -- 172.21.15.31:0/4195712431 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcb940ce600 con 0x7fcba0101ff0 2023-10-06T21:22:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.733+0000 7fcbba4b1b80 1 Processor -- start 2023-10-06T21:22:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.733+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 0x7fcb900a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.733+0000 7fcbba4b1b80 1 -- start start 2023-10-06T21:22:28.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a804760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a807090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a75e460 con 0x56046a7f50c0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a74fc20 con 0x56046a804ca0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a74d540 con 0x56046a7f59e0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 0x7fcb900a3bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcba0198500 tx=0x7fcb8c009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51366/0 (socket says 172.21.15.31:51366) 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 -- 172.21.15.31:0/2377356969 learned_addr learned my addr 172.21.15.31:0/2377356969 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba519c700 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a807090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a804760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 -- 172.21.15.31:0/2377356969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 msgr2=0x56046a804760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a804760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 -- 172.21.15.31:0/2377356969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 msgr2=0x56046a807090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.734+0000 7fcba499b700 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a807090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.735+0000 7fcba499b700 1 -- 172.21.15.31:0/2377356969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046a573de0 con 0x56046a7f50c0 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.735+0000 7fcba499b700 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcb940a3140 tx=0x7fcb94022f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3264c9402453772 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.735+0000 7fcb8affd700 1 -- 172.21.15.31:0/2377356969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9409ba70 con 0x56046a7f50c0 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.735+0000 7fcb8affd700 1 -- 172.21.15.31:0/2377356969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb9409b370 con 0x56046a7f50c0 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.735+0000 7fcb8affd700 1 -- 172.21.15.31:0/2377356969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9409b5d0 con 0x56046a7f50c0 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2377356969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 msgr2=0x56046a7f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcb940a3140 tx=0x7fcb94022f70 comp rx=0 tx=0).stop 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2377356969 shutdown_connections 2023-10-06T21:22:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a807090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a7f54a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2377356969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a804760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2377356969 >> 172.21.15.31:0/2377356969 conn(0x56046a76a260 msgr2=0x56046a7f1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2377356969 shutdown_connections 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.736+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2377356969 wait complete. 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 -- start start 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 Processor -- start 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 0x56046a8055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a805b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a748e30 con 0x56046a7f59e0 2023-10-06T21:22:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a6e5710 con 0x56046a804ca0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a6e53b0 con 0x56046a7f50c0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.737+0000 7fcbba4b1b80 10 client.0 _ll_register_callbacks cb 0x56046a542800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36000/0 (socket says 172.21.15.31:36000) 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcb9ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 0x56046a8055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 -- 172.21.15.31:0/2099808227 learned_addr learned my addr 172.21.15.31:0/2099808227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba519c700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a805b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 -- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 msgr2=0x56046a805b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a805b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 -- 172.21.15.31:0/2099808227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 msgr2=0x56046a8055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 0x56046a8055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 -- 172.21.15.31:0/2099808227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba019c480 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 0x56046a8055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcba499b700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb9409b070 tx=0x7fcb94099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c576f59cf415b16c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcb7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940a3670 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56046a7e8f10 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.738+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcb9409f4c0 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb9409b5d0 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 10 client.25849 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56046a55cc60 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56046a7e8f10 con 0x56046a7f50c0 2023-10-06T21:22:28.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcb9400fb40 con 0x56046a7f50c0 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 10 client.25849 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 20 client.25849 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a813fb0 0x56046a8163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 4 client.25849 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.739+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56046a741e10 con 0x56046a813fb0 2023-10-06T21:22:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.740+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a813fb0 0x56046a8163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.740+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a813fb0 0x56046a8163d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3de353373782533 server_cookie=84412ad393d7f7 in_seq=0 out_seq=0 2023-10-06T21:22:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.740+0000 7fcb7ffff700 10 client.25849 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.741+0000 7fcb7ffff700 1 -- 172.21.15.31:0/2099808227 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56046a741e10 con 0x56046a813fb0 2023-10-06T21:22:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.741+0000 7fcb7ffff700 10 client.25849 handle_command_reply: tid=0 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.741+0000 7fcbba4b1b80 1 client.25849 shutdown 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.741+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 msgr2=0x56046a805080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.741+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb9409b070 tx=0x7fcb94099920 comp rx=0 tx=0).stop 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 shutdown_connections 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a805b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f50c0 0x56046a805080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a813fb0 0x56046a8163d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2099808227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f59e0 0x56046a8055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcb7ffff700 0 client.25849 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 >> 172.21.15.31:0/2099808227 conn(0x56046a76a260 msgr2=0x56046a7f1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 shutdown_connections 2023-10-06T21:22:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2099808227 wait complete. 2023-10-06T21:22:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.742+0000 7fcbba4b1b80 20 client.25849 trim_cache size 0 max 16384 2023-10-06T21:22:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.759+0000 7fcbba4b1b80 1 Processor -- start 2023-10-06T21:22:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.759+0000 7fcbba4b1b80 1 -- start start 2023-10-06T21:22:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.759+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a810590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a7ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a5c9920 con 0x56046a7f50c0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a804380 con 0x56046a804ca0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a8095f0 con 0x56046a7f59e0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51392/0 (socket says 172.21.15.31:51392) 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 -- 172.21.15.31:0/2907828686 learned_addr learned my addr 172.21.15.31:0/2907828686 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a810590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba519c700 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a7ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 -- 172.21.15.31:0/2907828686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 msgr2=0x56046a810590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a810590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 -- 172.21.15.31:0/2907828686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 msgr2=0x56046a7ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a7ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.760+0000 7fcba499b700 1 -- 172.21.15.31:0/2907828686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046a59a250 con 0x56046a7f50c0 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.761+0000 7fcba499b700 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb94005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372473611794b8e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.761+0000 7fcb8b7fe700 1 -- 172.21.15.31:0/2907828686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940a3670 con 0x56046a7f50c0 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.761+0000 7fcb8b7fe700 1 -- 172.21.15.31:0/2907828686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb940cba60 con 0x56046a7f50c0 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.761+0000 7fcb8b7fe700 1 -- 172.21.15.31:0/2907828686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940a31e0 con 0x56046a7f50c0 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2907828686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 msgr2=0x56046a8090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb94005910 comp rx=0 tx=0).stop 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2907828686 shutdown_connections 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a804ca0 0x56046a7ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a8090b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/2907828686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a7f59e0 0x56046a810590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2907828686 >> 172.21.15.31:0/2907828686 conn(0x56046a76a260 msgr2=0x56046a7efbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2907828686 shutdown_connections 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/2907828686 wait complete. 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.762+0000 7fcbba4b1b80 1 -- start start 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 Processor -- start 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 0x56046a85c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a804ca0 0x56046a866df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a867330 con 0x56046a7f50c0 2023-10-06T21:22:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a867470 con 0x56046a7f59e0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.763+0000 7fcbba4b1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046a8675b0 con 0x56046a804ca0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcbba4b1b80 10 client.0 _ll_register_callbacks cb 0x7fcb98009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51404/0 (socket says 172.21.15.31:51404) 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 -- 172.21.15.31:0/3520494868 learned_addr learned my addr 172.21.15.31:0/3520494868 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 0x56046a85c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba519c700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a804ca0 0x56046a866df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 -- 172.21.15.31:0/3520494868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a804ca0 msgr2=0x56046a866df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a804ca0 0x56046a866df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 -- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 msgr2=0x56046a85c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 0x56046a85c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcba499b700 1 -- 172.21.15.31:0/3520494868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56046a55cc60 con 0x56046a7f50c0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.764+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 0x56046a85c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcba499b700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcb940095c0 tx=0x7fcb9409ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171a85c2401d8b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940a33e0 con 0x56046a7f50c0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fcb9409f4c0 con 0x56046a7f50c0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb9409b370 con 0x56046a7f50c0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb9409b5d0 con 0x56046a7f50c0 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcbba4b1b80 10 client.16161 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:28.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56046a5722b0 con 0x56046a7f50c0 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb940a9020 con 0x56046a7f50c0 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcb940a1400 con 0x56046a7f50c0 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcbba4b1b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.765+0000 7fcbba4b1b80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcbba4b1b80 20 client.16161 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:28.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a868770 0x56046a86ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcbba4b1b80 4 client.16161 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56046a741b30 con 0x56046a868770 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a868770 0x56046a86ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcb9ffff700 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a868770 0x56046a86ab60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7a9f196edf6e28 server_cookie=82051016acb47aea in_seq=0 out_seq=0 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.766+0000 7fcb7ffff700 10 client.16161 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcb7ffff700 1 -- 172.21.15.31:0/3520494868 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56046a741b30 con 0x56046a868770 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcb7ffff700 10 client.16161 handle_command_reply: tid=0 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 client.16161 shutdown 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 msgr2=0x56046a85bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcb940095c0 tx=0x7fcb9409ad90 comp rx=0 tx=0).stop 2023-10-06T21:22:28.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 shutdown_connections 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046a804ca0 0x56046a866df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56046a868770 0x56046a86ab60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046a7f50c0 0x56046a85bf60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 --2- 172.21.15.31:0/3520494868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56046a7f59e0 0x56046a85c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcb7ffff700 0 client.16161 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 >> 172.21.15.31:0/3520494868 conn(0x56046a76a260 msgr2=0x56046a810590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.767+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 shutdown_connections 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcbba4b1b80 1 -- 172.21.15.31:0/3520494868 wait complete. 2023-10-06T21:22:28.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcbba4b1b80 20 client.16161 trim_cache size 0 max 16384 2023-10-06T21:22:28.775 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:22:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 msgr2=0x7fcb900a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 0x7fcb900a3bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcba0198500 tx=0x7fcb8c009260 comp rx=0 tx=0).stop 2023-10-06T21:22:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 msgr2=0x7fcba0196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb94005740 tx=0x7fcb94005770 comp rx=0 tx=0).stop 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 shutdown_connections 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcb900a1710 0x7fcb900a3bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba010b410 0x7fcba019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba0101ff0 0x7fcba0196ef0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 --2- 172.21.15.31:0/4195712431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcba0102910 0x7fcba0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 >> 172.21.15.31:0/4195712431 conn(0x7fcba00fb830 msgr2=0x7fcba010d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.768+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 shutdown_connections 2023-10-06T21:22:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:28.769+0000 7fcba6bff700 1 -- 172.21.15.31:0/4195712431 wait complete. 2023-10-06T21:22:29.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 7459 3.11 MB/sec warmup 19 sec latency 16.333 ms 2023-10-06T21:22:29.790 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:22:29.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:22:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 -- 172.21.15.31:0/3878995740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 msgr2=0x7f67f0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 --2- 172.21.15.31:0/3878995740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f0101490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009990 tx=0x7f67e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 -- 172.21.15.31:0/3878995740 shutdown_connections 2023-10-06T21:22:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 --2- 172.21.15.31:0/3878995740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f01061d0 0x7f67f010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 --2- 172.21.15.31:0/3878995740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f0101490 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 --2- 172.21.15.31:0/3878995740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f0101a60 0x7f67f0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 -- 172.21.15.31:0/3878995740 >> 172.21.15.31:0/3878995740 conn(0x7f67f00fc950 msgr2=0x7f67f00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 -- 172.21.15.31:0/3878995740 shutdown_connections 2023-10-06T21:22:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.017+0000 7f67f61ff700 1 -- 172.21.15.31:0/3878995740 wait complete. 2023-10-06T21:22:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.018+0000 7f67f61ff700 1 Processor -- start 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.018+0000 7f67f61ff700 1 -- start start 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 0x7f67f00736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 0x7f67f01a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01a5730 con 0x7f67f01010b0 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01a58a0 con 0x7f67f01061d0 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67f61ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01a5a60 con 0x7f67f0073c30 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51426/0 (socket says 172.21.15.31:51426) 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1249366268 learned_addr learned my addr 172.21.15.31:0/1249366268 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67eeffd700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 0x7f67f00736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.019+0000 7f67effff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 0x7f67f01a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1249366268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 msgr2=0x7f67f01a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 0x7f67f01a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1249366268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 msgr2=0x7f67f00736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 0x7f67f00736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1249366268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00095f0 con 0x7f67f01010b0 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67effff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 0x7f67f01a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67eeffd700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 0x7f67f00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005740 tx=0x7f67e0005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ecff9700 1 -- 172.21.15.31:0/1249366268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e001d070 con 0x7f67f01010b0 2023-10-06T21:22:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f01a5ce0 con 0x7f67f01010b0 2023-10-06T21:22:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ecff9700 1 -- 172.21.15.31:0/1249366268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e0022470 con 0x7f67f01010b0 2023-10-06T21:22:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67ecff9700 1 -- 172.21.15.31:0/1249366268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0017650 con 0x7f67f01010b0 2023-10-06T21:22:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.020+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f01a6110 con 0x7f67f01010b0 2023-10-06T21:22:30.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.024+0000 7f67ecff9700 1 -- 172.21.15.31:0/1249366268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f67e00225e0 con 0x7f67f01010b0 2023-10-06T21:22:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.025+0000 7f6809b58b80 1 Processor -- start 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.025+0000 7f6809b58b80 1 -- start start 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.025+0000 7f67ecff9700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 0x7f67d80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.025+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 0x556996f19be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f6809b58b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996e6f4e0 con 0x556996f06a60 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f6809b58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996e60ca0 con 0x556996f1a120 2023-10-06T21:22:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f6809b58b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996e5e5c0 con 0x556996f06140 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ecff9700 1 -- 172.21.15.31:0/1249366268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f67e00ce530 con 0x7f67f01010b0 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 0x556996f19be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36050/0 (socket says 172.21.15.31:36050) 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1003441040 learned_addr learned my addr 172.21.15.31:0/1003441040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 0x556996f19be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51436/0 (socket says 172.21.15.31:51436) 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67effff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 0x7f67d80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67effff700 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1003441040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 msgr2=0x556996f1c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1003441040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 msgr2=0x556996f19be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 0x556996f19be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.026+0000 7f67ef7fe700 1 -- 172.21.15.31:0/1003441040 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556996c84e60 con 0x556996f06140 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.027+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67e00a32c0 tx=0x7f67e009b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3244f0874f039488 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.027+0000 7f67d5ffb700 1 -- 172.21.15.31:0/1003441040 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e009bbf0 con 0x556996f06140 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.027+0000 7f67d5ffb700 1 -- 172.21.15.31:0/1003441040 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e009b4f0 con 0x556996f06140 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.027+0000 7f67effff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 0x7f67d80a3ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67e8015b80 tx=0x7f67e8009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.027+0000 7f67d5ffb700 1 -- 172.21.15.31:0/1003441040 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e009b750 con 0x556996f06140 2023-10-06T21:22:30.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 -- 172.21.15.31:0/1003441040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 msgr2=0x556996f06520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67e00a32c0 tx=0x7f67e009b340 comp rx=0 tx=0).stop 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 -- 172.21.15.31:0/1003441040 shutdown_connections 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06140 0x556996f06520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1003441040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06a60 0x556996f19be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 -- 172.21.15.31:0/1003441040 >> 172.21.15.31:0/1003441040 conn(0x556996e7b2e0 msgr2=0x556996f024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 -- 172.21.15.31:0/1003441040 shutdown_connections 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.028+0000 7f6809b58b80 1 -- 172.21.15.31:0/1003441040 wait complete. 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 -- start start 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 Processor -- start 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996e59eb0 con 0x556996f06140 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996df6790 con 0x556996f1a120 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996df6430 con 0x556996f06a60 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.029+0000 7f6809b58b80 10 client.0 _ll_register_callbacks cb 0x556996c53880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36052/0 (socket says 172.21.15.31:36052) 2023-10-06T21:22:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 -- 172.21.15.31:0/1620157692 learned_addr learned my addr 172.21.15.31:0/1620157692 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67effff700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 -- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 msgr2=0x556996f1af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 -- 172.21.15.31:0/1620157692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 msgr2=0x556996f1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 -- 172.21.15.31:0/1620157692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f01a6110 con 0x556996f06a60 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67eeffd700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67e4012d20 tx=0x7f67e400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e6bdd4c5d946f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.030+0000 7f67ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4017070 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556996ef9f90 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f67e400fb80 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e400fd40 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f6809b58b80 10 client.25867 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556996c6dce0 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.031+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556996ef9f90 con 0x556996f06a60 2023-10-06T21:22:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f67e4022030 con 0x556996f06a60 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 20 client.25867 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f29820 0x556996f2bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 4 client.25867 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556996e52e90 con 0x556996f29820 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.032+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f29820 0x556996f2bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.033+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f29820 0x556996f2bc40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741638ca6d7dc95b server_cookie=6753f416d42dc43c in_seq=0 out_seq=0 2023-10-06T21:22:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.033+0000 7f67ceffd700 10 client.25867 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.033+0000 7f67ceffd700 1 -- 172.21.15.31:0/1620157692 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556996e52e90 con 0x556996f29820 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.033+0000 7f67ceffd700 10 client.25867 handle_command_reply: tid=0 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 client.25867 shutdown 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 msgr2=0x556996f1aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67e4012d20 tx=0x7f67e400f6a0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 shutdown_connections 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f1a120 0x556996f1af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f29820 0x556996f2bc40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 --2- 172.21.15.31:0/1620157692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f1aa40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f67ceffd700 0 client.25867 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 >> 172.21.15.31:0/1620157692 conn(0x556996e7b2e0 msgr2=0x556996f02060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 shutdown_connections 2023-10-06T21:22:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 1 -- 172.21.15.31:0/1620157692 wait complete. 2023-10-06T21:22:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.034+0000 7f6809b58b80 20 client.25867 trim_cache size 0 max 16384 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 Processor -- start 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 -- start start 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 0x556996f1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f14450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996cda9a0 con 0x556996f1a120 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996f197e0 con 0x556996f06140 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.055+0000 7f6809b58b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996f1ea70 con 0x556996f06a60 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f14450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 0x556996f1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51448/0 (socket says 172.21.15.31:51448) 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 -- 172.21.15.31:0/3395567112 learned_addr learned my addr 172.21.15.31:0/3395567112 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 -- 172.21.15.31:0/3395567112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 msgr2=0x556996f14450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f14450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 -- 172.21.15.31:0/3395567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 msgr2=0x556996f1e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 0x556996f1e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67effff700 1 -- 172.21.15.31:0/3395567112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556996f30870 con 0x556996f1a120 2023-10-06T21:22:30.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67eeffd700 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f14450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.056+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 0x556996f1e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.057+0000 7f67effff700 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009450 tx=0x7f67e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5db53f90d8a1e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.057+0000 7f67d67fc700 1 -- 172.21.15.31:0/3395567112 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800e040 con 0x556996f1a120 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.057+0000 7f67d67fc700 1 -- 172.21.15.31:0/3395567112 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e8007990 con 0x556996f1a120 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.057+0000 7f67d67fc700 1 -- 172.21.15.31:0/3395567112 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8019930 con 0x556996f1a120 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- 172.21.15.31:0/3395567112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 msgr2=0x556996f14990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009450 tx=0x7f67e800b040 comp rx=0 tx=0).stop 2023-10-06T21:22:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- 172.21.15.31:0/3395567112 shutdown_connections 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f1a120 0x556996f14990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06140 0x556996f1e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 --2- 172.21.15.31:0/3395567112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f06a60 0x556996f14450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- 172.21.15.31:0/3395567112 >> 172.21.15.31:0/3395567112 conn(0x556996e7b2e0 msgr2=0x556996f25ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- 172.21.15.31:0/3395567112 shutdown_connections 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- 172.21.15.31:0/3395567112 wait complete. 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 -- start start 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.058+0000 7f6809b58b80 1 Processor -- start 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06a60 0x556996f77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f1a120 0x556996f715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996f71bb0 con 0x556996f06140 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996f71d20 con 0x556996f06a60 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556996f71ee0 con 0x556996f1a120 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f6809b58b80 10 client.0 _ll_register_callbacks cb 0x7f67e40127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51452/0 (socket says 172.21.15.31:51452) 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f67ef7fe700 1 -- 172.21.15.31:0/235269896 learned_addr learned my addr 172.21.15.31:0/235269896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f67eeffd700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06a60 0x556996f77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.059+0000 7f67ef7fe700 1 -- 172.21.15.31:0/235269896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f1a120 msgr2=0x556996f715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f1a120 0x556996f715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ef7fe700 1 -- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06a60 msgr2=0x556996f77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06a60 0x556996f77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ef7fe700 1 -- 172.21.15.31:0/235269896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556996c84e60 con 0x556996f06140 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ef7fe700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67e0099840 tx=0x7f67e009b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb0fbd7860b7645 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e009abf0 con 0x556996f06140 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f67e009b750 con 0x556996f06140 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67e009b4f0 con 0x556996f06140 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e009b930 con 0x556996f06140 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f6809b58b80 10 client.16179 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.060+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556996c6dce0 con 0x556996f06140 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e0080080 con 0x556996f06140 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f67e00cbe50 con 0x556996f06140 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 20 client.16179 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f7ed20 0x556996f81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 4 client.16179 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556996e52bb0 con 0x556996f7ed20 2023-10-06T21:22:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.061+0000 7f67eeffd700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f7ed20 0x556996f81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f67eeffd700 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f7ed20 0x556996f81140 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389d7537708005ea server_cookie=4ff58fa3457d8861 in_seq=0 out_seq=0 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f67ceffd700 10 client.16179 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f67ceffd700 1 -- 172.21.15.31:0/235269896 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556996e52bb0 con 0x556996f7ed20 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f67ceffd700 10 client.16179 handle_command_reply: tid=0 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f6809b58b80 1 client.16179 shutdown 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 msgr2=0x556996f77110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.062+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67e0099840 tx=0x7f67e009b2c0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 shutdown_connections 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556996f1a120 0x556996f715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556996f7ed20 0x556996f81140 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556996f06140 0x556996f77110 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 --2- 172.21.15.31:0/235269896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556996f06a60 0x556996f77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f67ceffd700 0 client.16179 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 >> 172.21.15.31:0/235269896 conn(0x556996e7b2e0 msgr2=0x556996f14450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 shutdown_connections 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 1 -- 172.21.15.31:0/235269896 wait complete. 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.063+0000 7f6809b58b80 20 client.16179 trim_cache size 0 max 16384 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-10-06T21:22:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "4267b2dd-bfe4-43a2-a3ca-fc70a15345ff", 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 msgr2=0x7f67d80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 0x7f67d80a3ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67e8015b80 tx=0x7f67e8009920 comp rx=0 tx=0).stop 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 msgr2=0x7f67f00731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005740 tx=0x7f67e0005770 comp rx=0 tx=0).stop 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 shutdown_connections 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67f0073c30 0x7f67f01a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67d80a15f0 0x7f67d80a3ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f01010b0 0x7f67f00731b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 --2- 172.21.15.31:0/1249366268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f01061d0 0x7f67f00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 >> 172.21.15.31:0/1249366268 conn(0x7f67f00fc950 msgr2=0x7f67f00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 shutdown_connections 2023-10-06T21:22:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.064+0000 7f67f61ff700 1 -- 172.21.15.31:0/1249366268 wait complete. 2023-10-06T21:22:30.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:22:30.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:29 smithi142 ceph-mon[102951]: pgmap v49: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:22:30.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:29 smithi031 ceph-mon[98539]: pgmap v49: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:22:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:29 smithi146 ceph-mon[102809]: pgmap v49: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:22:30.299 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.069+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v229 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.679714+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492fac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v84>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v229>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.299 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.071+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v285 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.729093+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492fa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v122>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v285>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.300 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.071+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v223 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.667322+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492fb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v78>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v223>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.300 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.072+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v196 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.741654+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492fb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v60>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v196>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.300 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.072+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v260 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.716710+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149344000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v106>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v260>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.300 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.073+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v186 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.643499+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149344580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v54>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v186>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.301 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.074+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v235 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.691536+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149344b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v90>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v235>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.301 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.074+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v184 ap=1 s=260096 n(v0 rc2023-10-06T21:22:18.157471+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149345080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v52>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v184>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.301 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.074+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v237 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.704545+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149345600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v92>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v237>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.301 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.075+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v198 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.655833+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149345b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v62>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v198>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.075+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v182 ap=1 s=260096 n(v0 rc2023-10-06T21:22:18.179925+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149346100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v50>,<0x10000000009/COREL v957>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v182>,<0x10000000009/COREL v1656>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.075+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth v110 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.761850+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149378b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v8>,<0x10000000009/WORDPRO v810>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v110>,<0x10000000009/WORDPRO v1591>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.076+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v134 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.774728+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@24},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149379080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v22>,<0x10000000009/WORDPRO v810>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v134>,<0x10000000009/WORDPRO v1591>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.076+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v177 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.787071+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@24},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149379600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v46>,<0x10000000009/WORDPRO v810>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v177>,<0x10000000009/WORDPRO v1591>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.076+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v50 ap=1 s=327680 n(v0 rc2023-10-06T21:22:26.663467+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149346680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v26>,<0x10000000009/SEED v862>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v50>,<0x10000000009/SEED v1381>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.076+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v52 ap=1 s=1048576 n(v0 rc2023-10-06T21:22:26.667611+0000 b1048576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149346c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v28>,<0x10000000009/SEED v862>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v52>,<0x10000000009/SEED v1381>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.077+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v232 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.486655+0000 b260096 1=1+0) (iversion lock) caps={15408=pAsLsXsFsc/-@1},l=15408(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492f8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v94>,<0x10000000009/WORD v1013>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v232>,<0x10000000009/WORD v1375>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.077+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v54 ap=1 s=1867776 n(v0 rc2023-10-06T21:22:26.673154+0000 b1867776 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149347180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v30>,<0x10000000009/SEED v862>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v54>,<0x10000000009/SEED v1381>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.302 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.078+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v236 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.500250+0000 b260096 1=1+0) (iversion lock) caps={15408=pAsLsXsFsc/-@1},l=15408(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492f9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v98>,<0x10000000009/WORD v1013>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v236>,<0x10000000009/WORD v1375>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.303 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.078+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v234 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.473068+0000 b260096 1=1+0) (iversion lock) caps={15408=pAsLsXsFsc/-@1},l=15408(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1492f9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v96>,<0x10000000009/WORD v1013>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v234>,<0x10000000009/WORD v1375>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.303 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.078+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v143 ap=1 s=260096 n(v0 rc2023-10-06T21:22:24.597782+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149379b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v16>,<0x10000000009/EXCEL v686>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v143>,<0x10000000009/EXCEL v1808>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.303 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.079+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth v265 ap=1 s=260096 n(v0 rc2023-10-06T21:22:25.359265+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c14937a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v88>,<0x10000000009/EXCEL v686>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v265>,<0x10000000009/EXCEL v1808>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.303 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.080+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v339 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.804254+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@42},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149347700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000226:[<0x10000000225/T1.XLS v92>,<0x10000000009/PM v1089>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000226:[<0x10000000225/T1.XLS v339>,<0x10000000009/PM v1806>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.303 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.081+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v184 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.817277+0000 b260096 1=1+0) (iversion lock) caps={15408=pAsLsXsFscr/-@38},l=15408(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149378000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v10>,<0x10000000009/PM v1089>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v184>,<0x10000000009/PM v1806>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.304 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.081+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v186 ap=1 s=260096 n(v0 rc2023-10-06T21:22:10.829503+0000 b260096 1=1+0) (iversion lock) caps={15408=pAsLsXsFscr/-@38},l=15408(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c149378580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v12>,<0x10000000009/PM v1089>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v186>,<0x10000000009/PM v1806>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.304 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:22:30.082+0000 7f36ee69d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [4,head] /volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v57 ap=1 s=260096 n(v0 rc2023-10-06T21:22:27.420360+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15408=0-4194304@3} caps={15408=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@10},l=15408 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c14937a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v16>,<0x10000000009/ACCESS v842>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v39>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x10000000003/client.0 v22>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v37>,<0x10000000001/sv_0 v57>,<0x10000000000/_nogroup v42>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v57>,<0x10000000009/ACCESS v1611>,<0x10000000008/~dmtmp v348>,<0x10000000007/client0 v63>,<0x10000000006/clients v43>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v46>,<0x10000000002/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a v40>,<0x10000000001/sv_0 v79>,<0x10000000000/_nogroup v62>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/4180645981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 msgr2=0x7f0e78101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/4180645981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e78101490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/4180645981 shutdown_connections 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/4180645981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e781061d0 0x7f0e7810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/4180645981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e78101490 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/4180645981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e78101a60 0x7f0e78105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.323+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/4180645981 >> 172.21.15.31:0/4180645981 conn(0x7f0e780fc930 msgr2=0x7f0e780fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.324+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/4180645981 shutdown_connections 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.324+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/4180645981 wait complete. 2023-10-06T21:22:30.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 Processor -- start 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 -- start start 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 0x7f0e780736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 0x7f0e781a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781a5790 con 0x7f0e781010b0 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781a5900 con 0x7f0e78073c30 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.325+0000 7f0e7d3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781a5ac0 con 0x7f0e781061d0 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 0x7f0e780736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51476/0 (socket says 172.21.15.31:51476) 2023-10-06T21:22:30.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e76ffd700 1 -- 172.21.15.31:0/357099552 learned_addr learned my addr 172.21.15.31:0/357099552 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 7635 2.96 MB/sec warmup 20 sec latency 30.369 ms 2023-10-06T21:22:30.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e777fe700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 0x7f0e781a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 -- 172.21.15.31:0/357099552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 msgr2=0x7f0e781a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 0x7f0e781a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 -- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 msgr2=0x7f0e780731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 -- 172.21.15.31:0/357099552 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680095f0 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e777fe700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 0x7f0e781a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.326+0000 7f0e767fc700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 0x7f0e780736f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9c0 tx=0x7f0e6c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.327+0000 7f0e5ffff700 1 -- 172.21.15.31:0/357099552 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00cbd0 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.327+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e781a5da0 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.327+0000 7f0e5ffff700 1 -- 172.21.15.31:0/357099552 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6c00eeb0 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.327+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e781a6270 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.327+0000 7f0e5ffff700 1 -- 172.21.15.31:0/357099552 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018420 con 0x7f0e781061d0 2023-10-06T21:22:30.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.330+0000 7f0e5ffff700 1 -- 172.21.15.31:0/357099552 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0e6c010430 con 0x7f0e781061d0 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 Processor -- start 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 -- start start 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e5ffff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 0x7f0e600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db9e0 0x5584a03e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e5ffff700 1 -- 172.21.15.31:0/357099552 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0e6c014070 con 0x7f0e781061d0 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a0344460 con 0x5584a03db0c0 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a0335c20 con 0x5584a03db9e0 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.332+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a0333540 con 0x5584a03e6890 2023-10-06T21:22:30.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51480/0 (socket says 172.21.15.31:51480) 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 -- 172.21.15.31:0/455838636 learned_addr learned my addr 172.21.15.31:0/455838636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e777fe700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 0x7f0e600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e767fc700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db9e0 0x5584a03e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e777fe700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 -- 172.21.15.31:0/455838636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 msgr2=0x5584a03e8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 -- 172.21.15.31:0/455838636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db9e0 msgr2=0x5584a03e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db9e0 0x5584a03e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e76ffd700 1 -- 172.21.15.31:0/455838636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a0159de0 con 0x5584a03db0c0 2023-10-06T21:22:30.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.333+0000 7f0e777fe700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:30.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.334+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005c30 tx=0x7f0e68005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1875358c0c487935 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.334+0000 7f0e777fe700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 0x7f0e600a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e70015c00 tx=0x7f0e70009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.334+0000 7f0e5d7fa700 1 -- 172.21.15.31:0/455838636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d070 con 0x5584a03db0c0 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.334+0000 7f0e5d7fa700 1 -- 172.21.15.31:0/455838636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6801e040 con 0x5584a03db0c0 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.334+0000 7f0e5d7fa700 1 -- 172.21.15.31:0/455838636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004c90 con 0x5584a03db0c0 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- 172.21.15.31:0/455838636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 msgr2=0x5584a03db4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005c30 tx=0x7f0e68005c60 comp rx=0 tx=0).stop 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- 172.21.15.31:0/455838636 shutdown_connections 2023-10-06T21:22:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db0c0 0x5584a03db4a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/455838636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db9e0 0x5584a03e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- 172.21.15.31:0/455838636 >> 172.21.15.31:0/455838636 conn(0x5584a0350260 msgr2=0x5584a03d7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- 172.21.15.31:0/455838636 shutdown_connections 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- 172.21.15.31:0/455838636 wait complete. 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.335+0000 7f0e90d11b80 1 -- start start 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 Processor -- start 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 0x5584a03e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a032ee30 con 0x5584a03db9e0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a02cb710 con 0x5584a03e6890 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a02cb3b0 con 0x5584a03db0c0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e90d11b80 10 client.0 _ll_register_callbacks cb 0x5584a0128800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.336+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36120/0 (socket says 172.21.15.31:36120) 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 -- 172.21.15.31:0/1386625600 learned_addr learned my addr 172.21.15.31:0/1386625600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51486/0 (socket says 172.21.15.31:51486) 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e777fe700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 0x5584a03e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 -- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 msgr2=0x5584a03e3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 0x5584a03e3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 -- 172.21.15.31:0/1386625600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 msgr2=0x5584a03e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 -- 172.21.15.31:0/1386625600 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7810e5c0 con 0x5584a03db0c0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e777fe700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 0x5584a03e3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006010 tx=0x7f0e68022b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd16dc4bdb6a0c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.337+0000 7f0e5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d030 con 0x5584a03db0c0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0e68004500 con 0x5584a03db0c0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584a03cef10 con 0x5584a03db0c0 2023-10-06T21:22:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e68005910 con 0x5584a03db0c0 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 10 client.25882 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5584a0142c60 con 0x5584a03db0c0 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5584a03cef10 con 0x5584a03db0c0 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0e6802c4b0 con 0x5584a03db0c0 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.338+0000 7f0e90d11b80 20 client.25882 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a03f5e00 0x5584a03f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e90d11b80 4 client.25882 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584a0327e10 con 0x5584a03f5e00 2023-10-06T21:22:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a03f5e00 0x5584a03f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a03f5e00 0x5584a03f8220 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68973a1f1e765d7d server_cookie=bf635e0d2bdce9e9 in_seq=0 out_seq=0 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.339+0000 7f0e5aff5700 10 client.25882 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1386625600 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5584a0327e10 con 0x5584a03f5e00 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e5aff5700 10 client.25882 handle_command_reply: tid=0 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 client.25882 shutdown 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 msgr2=0x5584a03e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006010 tx=0x7f0e68022b50 comp rx=0 tx=0).stop 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 shutdown_connections 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03e6890 0x5584a03e3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db0c0 0x5584a03e6290 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.340+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a03f5e00 0x5584a03f8220 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1386625600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 >> 172.21.15.31:0/1386625600 conn(0x5584a0350260 msgr2=0x5584a03d7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e5aff5700 0 client.25882 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 shutdown_connections 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1386625600 wait complete. 2023-10-06T21:22:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.341+0000 7f0e90d11b80 20 client.25882 trim_cache size 0 max 16384 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.360+0000 7f0e90d11b80 1 Processor -- start 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.360+0000 7f0e90d11b80 1 -- start start 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ed370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03e6890 0x5584a03e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a01af920 con 0x5584a03e6890 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a03d1550 con 0x5584a03db0c0 2023-10-06T21:22:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a03ed930 con 0x5584a03db9e0 2023-10-06T21:22:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03e6890 0x5584a03e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36128/0 (socket says 172.21.15.31:36128) 2023-10-06T21:22:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 -- 172.21.15.31:0/1590578062 learned_addr learned my addr 172.21.15.31:0/1590578062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ed370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 -- 172.21.15.31:0/1590578062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 msgr2=0x5584a03ed370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ed370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 -- 172.21.15.31:0/1590578062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03e6890 msgr2=0x5584a03e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03e6890 0x5584a03e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.361+0000 7f0e767fc700 1 -- 172.21.15.31:0/1590578062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a0180250 con 0x5584a03db9e0 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c099fa0 tx=0x7f0e6c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0f1c8135f5ea0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e5dffb700 1 -- 172.21.15.31:0/1590578062 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0b2050 con 0x5584a03db9e0 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e5dffb700 1 -- 172.21.15.31:0/1590578062 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6c099c00 con 0x5584a03db9e0 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e5dffb700 1 -- 172.21.15.31:0/1590578062 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c09e120 con 0x5584a03db9e0 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1590578062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 msgr2=0x5584a03e5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.362+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c099fa0 tx=0x7f0e6c000bc0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1590578062 shutdown_connections 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03e6890 0x5584a03e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ed370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1590578062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03db9e0 0x5584a03e5dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1590578062 >> 172.21.15.31:0/1590578062 conn(0x5584a0350260 msgr2=0x5584a03f2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1590578062 shutdown_connections 2023-10-06T21:22:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1590578062 wait complete. 2023-10-06T21:22:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 -- start start 2023-10-06T21:22:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.363+0000 7f0e90d11b80 1 Processor -- start 2023-10-06T21:22:30.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a03e9920 con 0x5584a03db9e0 2023-10-06T21:22:30.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a03e9a90 con 0x5584a03db0c0 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a03e9c50 con 0x5584a03e6890 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e90d11b80 10 client.0 _ll_register_callbacks cb 0x7f0e6c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51508/0 (socket says 172.21.15.31:51508) 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e767fc700 1 -- 172.21.15.31:0/1459820102 learned_addr learned my addr 172.21.15.31:0/1459820102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.364+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e777fe700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 -- 172.21.15.31:0/1459820102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 msgr2=0x5584a03e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 -- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 msgr2=0x5584a03ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:30.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 -- 172.21.15.31:0/1459820102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c0cc040 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e767fc700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c096d10 tx=0x7f0e6c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fba7226d480f58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c09a9a0 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0e6c09a2a0 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e6801e040 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.365+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e6c0a2630 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 10 client.16197 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5584a01582b0 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6801e040 con 0x5584a03db9e0 2023-10-06T21:22:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0e6c099790 con 0x5584a03db9e0 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 20 client.16197 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a044ab00 0x5584a044cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 4 client.16197 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5584a0327b30 con 0x5584a044ab00 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.366+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a044ab00 0x5584a044cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e76ffd700 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a044ab00 0x5584a044cf20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9b073d1788168b server_cookie=1d8e27e5498258cf in_seq=0 out_seq=0 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e5aff5700 10 client.16197 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e5aff5700 1 -- 172.21.15.31:0/1459820102 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5584a0327b30 con 0x5584a044ab00 2023-10-06T21:22:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e5aff5700 10 client.16197 handle_command_reply: tid=0 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 client.16197 shutdown 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 msgr2=0x5584a03ed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c096d10 tx=0x7f0e6c00cda0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 shutdown_connections 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584a03e6890 0x5584a03e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584a03db0c0 0x5584a03ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5584a044ab00 0x5584a044cf20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 --2- 172.21.15.31:0/1459820102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5584a03db9e0 0x5584a03ed030 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 >> 172.21.15.31:0/1459820102 conn(0x5584a0350260 msgr2=0x5584a03d6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.367+0000 7f0e5aff5700 0 client.16197 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 shutdown_connections 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e90d11b80 1 -- 172.21.15.31:0/1459820102 wait complete. 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e90d11b80 20 client.16197 trim_cache size 0 max 16384 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "4267b2dd-bfe4-43a2-a3ca-fc70a15345ff": { 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-10-06T21:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "4267b2dd-bfe4-43a2-a3ca-fc70a15345ff", 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 msgr2=0x7f0e600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 0x7f0e600a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e70015c00 tx=0x7f0e70009920 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 msgr2=0x7f0e780736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 0x7f0e780736f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9c0 tx=0x7f0e6c00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 shutdown_connections 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e78073c30 0x7f0e781a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0e600a1670 0x7f0e600a3b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e781010b0 0x7f0e780731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.368+0000 7f0e7d3ff700 1 --2- 172.21.15.31:0/357099552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e781061d0 0x7f0e780736f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.369+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 >> 172.21.15.31:0/357099552 conn(0x7f0e780fc930 msgr2=0x7f0e780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.369+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 shutdown_connections 2023-10-06T21:22:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:22:30.369+0000 7f0e7d3ff700 1 -- 172.21.15.31:0/357099552 wait complete. 2023-10-06T21:22:30.389 INFO:tasks.cephfs.filesystem:scrub status for tag:4267b2dd-bfe4-43a2-a3ca-fc70a15345ff - {'path': '/', 'tag': '4267b2dd-bfe4-43a2-a3ca-fc70a15345ff', 'options': 'recursive,force'} 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub queued for path: / 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.152 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.154 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: active paths [/] 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.155 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.156 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:30 smithi142 ceph-mon[102951]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-10-06T21:22:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub queued for path: / 2023-10-06T21:22:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:22:31.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:22:31.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: active paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:30 smithi031 ceph-mon[98539]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub queued for path: / 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:22:31.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.302 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:22:31.303 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: active paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.f log and `damage ls` output for details 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.304 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:30 smithi146 ceph-mon[102809]: scrub summary: idle+waiting paths [/] 2023-10-06T21:22:31.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 7930 2.84 MB/sec warmup 21 sec latency 19.721 ms 2023-10-06T21:22:32.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:31 smithi142 ceph-mon[102951]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 818 KiB/s wr, 94 op/s 2023-10-06T21:22:32.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:31 smithi031 ceph-mon[98539]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 818 KiB/s wr, 94 op/s 2023-10-06T21:22:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:31 smithi146 ceph-mon[102809]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 818 KiB/s wr, 94 op/s 2023-10-06T21:22:32.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 8278 2.78 MB/sec warmup 22 sec latency 19.681 ms 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 -- 172.21.15.31:0/314815710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0106f60 msgr2=0x7fc2e0107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/314815710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0106f60 0x7fc2e0107340 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009990 tx=0x7fc2dc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 -- 172.21.15.31:0/314815710 shutdown_connections 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/314815710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0068c90 0x7fc2e010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/314815710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0106f60 0x7fc2e0107340 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/314815710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 0x7fc2e0068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 -- 172.21.15.31:0/314815710 >> 172.21.15.31:0/314815710 conn(0x7fc2e0076b70 msgr2=0x7fc2e0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.409+0000 7fc2e85ff700 1 -- 172.21.15.31:0/314815710 shutdown_connections 2023-10-06T21:22:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.410+0000 7fc2e85ff700 1 -- 172.21.15.31:0/314815710 wait complete. 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 Processor -- start 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 -- start start 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 0x7fc2e01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 0x7fc2e019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e019e2b0 con 0x7fc2e0068c90 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e019e420 con 0x7fc2e0106f60 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.411+0000 7fc2e85ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e019e5e0 con 0x7fc2e00682d0 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51522/0 (socket says 172.21.15.31:51522) 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/2643558432 learned_addr learned my addr 172.21.15.31:0/2643558432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 0x7fc2e019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e639b700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 0x7fc2e01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/2643558432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 msgr2=0x7fc2e01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 0x7fc2e01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/2643558432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 msgr2=0x7fc2e019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 0x7fc2e019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/2643558432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0095f0 con 0x7fc2e0068c90 2023-10-06T21:22:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 0x7fc2e019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.412+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e9a0 tx=0x7fc2d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.413+0000 7fc2d77fe700 1 -- 172.21.15.31:0/2643558432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0009790 con 0x7fc2e0068c90 2023-10-06T21:22:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.413+0000 7fc2d77fe700 1 -- 172.21.15.31:0/2643558432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2d0010430 con 0x7fc2e0068c90 2023-10-06T21:22:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.413+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e019e8c0 con 0x7fc2e0068c90 2023-10-06T21:22:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.413+0000 7fc2d77fe700 1 -- 172.21.15.31:0/2643558432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000cea0 con 0x7fc2e0068c90 2023-10-06T21:22:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.413+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e01a6100 con 0x7fc2e0068c90 2023-10-06T21:22:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.416+0000 7fc2d77fe700 1 -- 172.21.15.31:0/2643558432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc2d00105a0 con 0x7fc2e0068c90 2023-10-06T21:22:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.418+0000 7fc2d77fe700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 0x7fc2cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.418+0000 7fc2d77fe700 1 -- 172.21.15.31:0/2643558432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2d0014070 con 0x7fc2e0068c90 2023-10-06T21:22:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.418+0000 7fc2fbf80b80 1 Processor -- start 2023-10-06T21:22:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.418+0000 7fc2e639b700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 0x7fc2cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.418+0000 7fc2fbf80b80 1 -- start start 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e639b700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 0x7fc2cc0a3ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006010 tx=0x7fc2dc00b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee425400 con 0x55ddee4bbc00 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee416bc0 con 0x55ddee4c73d0 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee4144e0 con 0x55ddee4bc520 2023-10-06T21:22:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36160/0 (socket says 172.21.15.31:36160) 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e6b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/1792981860 learned_addr learned my addr 172.21.15.31:0/1792981860 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/1792981860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 msgr2=0x55ddee4c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/1792981860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 msgr2=0x55ddee4bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4bbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.419+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/1792981860 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddee23ad80 con 0x55ddee4bc520 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.420+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00966e0 tx=0x7fc2d009a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd049feb012f5099 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.420+0000 7fc2d4bfe700 1 -- 172.21.15.31:0/1792981860 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00b1070 con 0x55ddee4bc520 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.420+0000 7fc2d4bfe700 1 -- 172.21.15.31:0/1792981860 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2d009a8e0 con 0x55ddee4bc520 2023-10-06T21:22:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.420+0000 7fc2d4bfe700 1 -- 172.21.15.31:0/1792981860 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d009a440 con 0x55ddee4bc520 2023-10-06T21:22:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/1792981860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 msgr2=0x55ddee4c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00966e0 tx=0x7fc2d009a0e0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/1792981860 shutdown_connections 2023-10-06T21:22:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/1792981860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bc520 0x55ddee4c6e90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/1792981860 >> 172.21.15.31:0/1792981860 conn(0x55ddee431200 msgr2=0x55ddee4b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/1792981860 shutdown_connections 2023-10-06T21:22:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.421+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/1792981860 wait complete. 2023-10-06T21:22:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 -- start start 2023-10-06T21:22:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 Processor -- start 2023-10-06T21:22:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bbc00 0x55ddee4c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee40fdd0 con 0x55ddee4bc520 2023-10-06T21:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee3ac6b0 con 0x55ddee4c73d0 2023-10-06T21:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee3ac350 con 0x55ddee4bbc00 2023-10-06T21:22:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2fbf80b80 10 client.0 _ll_register_callbacks cb 0x55ddee2097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.422+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51534/0 (socket says 172.21.15.31:51534) 2023-10-06T21:22:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/3085237045 learned_addr learned my addr 172.21.15.31:0/3085237045 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e639b700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bbc00 0x55ddee4c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/3085237045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bbc00 msgr2=0x55ddee4c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bbc00 0x55ddee4c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 msgr2=0x55ddee4c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/3085237045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e01a6100 con 0x55ddee4bc520 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00caed0 tx=0x7fc2d0096650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9508cb30ad9b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00b1070 con 0x55ddee4bc520 2023-10-06T21:22:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2d009e330 con 0x55ddee4bc520 2023-10-06T21:22:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.423+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddee4afd50 con 0x55ddee4bc520 2023-10-06T21:22:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d0098580 con 0x55ddee4bc520 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 10 client.16209 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ddee223c00 con 0x55ddee4bc520 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddee4afd50 con 0x55ddee4bc520 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc2d00b7020 con 0x55ddee4bc520 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 20 client.16209 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee4d6b40 0x55ddee4d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 4 client.16209 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.424+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddee408db0 con 0x55ddee4d6b40 2023-10-06T21:22:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.425+0000 7fc2e639b700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee4d6b40 0x55ddee4d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.426+0000 7fc2e639b700 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee4d6b40 0x55ddee4d8f60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91781bcc02f3cdba server_cookie=2c0d610794859327 in_seq=0 out_seq=0 2023-10-06T21:22:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.426+0000 7fc2c9ffb700 10 client.16209 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/3085237045 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ddee408db0 con 0x55ddee4d6b40 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2c9ffb700 10 client.16209 handle_command_reply: tid=0 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 client.16209 shutdown 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 msgr2=0x55ddee4c4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00caed0 tx=0x7fc2d0096650 comp rx=0 tx=0).stop 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 shutdown_connections 2023-10-06T21:22:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4bbc00 0x55ddee4c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee4d6b40 0x55ddee4d8f60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/3085237045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4c4730 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2c9ffb700 0 client.16209 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 >> 172.21.15.31:0/3085237045 conn(0x55ddee431200 msgr2=0x55ddee4b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 shutdown_connections 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.427+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/3085237045 wait complete. 2023-10-06T21:22:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.428+0000 7fc2fbf80b80 20 client.16209 trim_cache size 0 max 16384 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 Processor -- start 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 -- start start 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee2908c0 con 0x55ddee4bc520 2023-10-06T21:22:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee4bf100 con 0x55ddee4bbc00 2023-10-06T21:22:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.445+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee4c0110 con 0x55ddee4c73d0 2023-10-06T21:22:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42746/0 (socket says 172.21.15.31:42746) 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e639b700 1 -- 172.21.15.31:0/217319761 learned_addr learned my addr 172.21.15.31:0/217319761 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/217319761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 msgr2=0x55ddee4c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/217319761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 msgr2=0x55ddee4d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 -- 172.21.15.31:0/217319761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddee2611f0 con 0x55ddee4bc520 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e639b700 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.446+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4beb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00966e0 tx=0x7fc2d00999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c199979b8fa6bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2d53ff700 1 -- 172.21.15.31:0/217319761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00b1020 con 0x55ddee4bc520 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2d53ff700 1 -- 172.21.15.31:0/217319761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2d0099b40 con 0x55ddee4bc520 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2d53ff700 1 -- 172.21.15.31:0/217319761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0096340 con 0x55ddee4bc520 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/217319761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 msgr2=0x55ddee4beb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4beb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00966e0 tx=0x7fc2d00999e0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/217319761 shutdown_connections 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bbc00 0x55ddee4d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/217319761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bc520 0x55ddee4beb80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/217319761 >> 172.21.15.31:0/217319761 conn(0x55ddee431200 msgr2=0x55ddee4d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.447+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/217319761 shutdown_connections 2023-10-06T21:22:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.448+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/217319761 wait complete. 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.448+0000 7fc2fbf80b80 1 -- start start 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.448+0000 7fc2fbf80b80 1 Processor -- start 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bc520 0x55ddee4cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee529a60 con 0x55ddee4bbc00 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee529bd0 con 0x55ddee4bc520 2023-10-06T21:22:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddee529d90 con 0x55ddee4c73d0 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2fbf80b80 10 client.0 _ll_register_callbacks cb 0x7fc2dc005090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2e639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2e6b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2e6b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36188/0 (socket says 172.21.15.31:36188) 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2e6b9c700 1 -- 172.21.15.31:0/503752585 learned_addr learned my addr 172.21.15.31:0/503752585 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.449+0000 7fc2e5b9a700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bc520 0x55ddee4cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 -- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bc520 msgr2=0x55ddee4cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bc520 0x55ddee4cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 -- 172.21.15.31:0/503752585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 msgr2=0x55ddee4cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 -- 172.21.15.31:0/503752585 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e010e5c0 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e639b700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2e6b9c700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8004330 tx=0x7fc2d8004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecfa9934bcd9ebe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8003c50 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fc2d800ec20 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.450+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2d009a8e0 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d800ede0 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 10 client.25906 fetch_fsmap learned FSMap version 29 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ddee223c00 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2d802c080 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc2d8026030 con 0x55ddee4c73d0 2023-10-06T21:22:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 20 client.25906 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee52b680 0x55ddee52daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 4 client.25906 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.451+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ddee408ad0 con 0x55ddee52b680 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.452+0000 7fc2e639b700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee52b680 0x55ddee52daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.452+0000 7fc2e639b700 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee52b680 0x55ddee52daa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f25fe3e1247ab7 server_cookie=c2e5b59262538a9b in_seq=0 out_seq=0 2023-10-06T21:22:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.452+0000 7fc2c9ffb700 10 client.25906 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.472+0000 7fc2c9ffb700 1 -- 172.21.15.31:0/503752585 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ddee408ad0 con 0x55ddee52b680 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.472+0000 7fc2c9ffb700 10 client.25906 handle_command_reply: tid=0 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.472+0000 7fc2fbf80b80 1 client.25906 shutdown 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.472+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 msgr2=0x55ddee4cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.472+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8004330 tx=0x7fc2d8004640 comp rx=0 tx=0).stop 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 shutdown_connections 2023-10-06T21:22:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddee4c73d0 0x55ddee4cc980 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ddee52b680 0x55ddee52daa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddee4bbc00 0x55ddee4cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 --2- 172.21.15.31:0/503752585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddee4bc520 0x55ddee4cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2c9ffb700 0 client.25906 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 >> 172.21.15.31:0/503752585 conn(0x55ddee431200 msgr2=0x55ddee4bf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 shutdown_connections 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 1 -- 172.21.15.31:0/503752585 wait complete. 2023-10-06T21:22:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.473+0000 7fc2fbf80b80 20 client.25906 trim_cache size 0 max 16384 2023-10-06T21:22:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:22:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:22:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:22:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 msgr2=0x7fc2cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 0x7fc2cc0a3ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006010 tx=0x7fc2dc00b260 comp rx=0 tx=0).stop 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 msgr2=0x7fc2e0199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e9a0 tx=0x7fc2d000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 shutdown_connections 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e0106f60 0x7fc2e019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2e00682d0 0x7fc2e01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2cc0a15f0 0x7fc2cc0a3ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 --2- 172.21.15.31:0/2643558432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2e0068c90 0x7fc2e0199600 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 >> 172.21.15.31:0/2643558432 conn(0x7fc2e0076b70 msgr2=0x7fc2e010c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 shutdown_connections 2023-10-06T21:22:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:32.474+0000 7fc2e85ff700 1 -- 172.21.15.31:0/2643558432 wait complete. 2023-10-06T21:22:33.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 8353 2.67 MB/sec warmup 23 sec latency 745.012 ms 2023-10-06T21:22:34.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:33 smithi031 ceph-mon[98539]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 920 KiB/s wr, 106 op/s 2023-10-06T21:22:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:33 smithi146 ceph-mon[102809]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 920 KiB/s wr, 106 op/s 2023-10-06T21:22:34.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 8353 2.56 MB/sec warmup 24 sec latency 1745.158 ms 2023-10-06T21:22:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:33 smithi142 ceph-mon[102951]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 920 KiB/s wr, 106 op/s 2023-10-06T21:22:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:22:34.996+0000 7f51d5f22700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T21:22:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:35 smithi031 ceph-mon[98539]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T21:22:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:35 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:22:35.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:35 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:35.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 8696 2.58 MB/sec warmup 25 sec latency 1854.884 ms 2023-10-06T21:22:35.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:35 smithi142 ceph-mon[102951]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T21:22:35.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:35 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:22:35.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:35 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:35 smithi146 ceph-mon[102809]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T21:22:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:35 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] up:active 2023-10-06T21:22:35.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:35 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:22:36.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 9106 2.49 MB/sec warmup 26 sec latency 19.562 ms 2023-10-06T21:22:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:36 smithi142 ceph-mon[102951]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:36.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:36 smithi031 ceph-mon[98539]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:36.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:36 smithi146 ceph-mon[102809]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:37.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 9493 2.45 MB/sec warmup 27 sec latency 16.939 ms 2023-10-06T21:22:38.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 9870 2.51 MB/sec warmup 28 sec latency 23.221 ms 2023-10-06T21:22:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:38 smithi142 ceph-mon[102951]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 812 KiB/s wr, 96 op/s 2023-10-06T21:22:38.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:38 smithi031 ceph-mon[98539]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 812 KiB/s wr, 96 op/s 2023-10-06T21:22:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:38 smithi146 ceph-mon[102809]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 812 KiB/s wr, 96 op/s 2023-10-06T21:22:39.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 10392 2.54 MB/sec warmup 29 sec latency 20.082 ms 2023-10-06T21:22:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:39 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:22:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:39 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:22:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:39 smithi142 ceph-mon[102951]: scrub summary: idle 2023-10-06T21:22:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:39 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:22:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:39 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:22:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:39 smithi031 ceph-mon[98539]: scrub summary: idle 2023-10-06T21:22:39.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:39 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:22:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:39 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:22:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:39 smithi146 ceph-mon[102809]: scrub summary: idle 2023-10-06T21:22:40.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 10653 2.56 MB/sec warmup 30 sec latency 24.923 ms 2023-10-06T21:22:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:40 smithi142 ceph-mon[102951]: pgmap v54: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2023-10-06T21:22:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:40 smithi031 ceph-mon[98539]: pgmap v54: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2023-10-06T21:22:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:40 smithi146 ceph-mon[102809]: pgmap v54: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2023-10-06T21:22:41.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 10925 2.52 MB/sec warmup 31 sec latency 20.007 ms 2023-10-06T21:22:42.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 11100 2.45 MB/sec warmup 32 sec latency 19.230 ms 2023-10-06T21:22:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:42 smithi142 ceph-mon[102951]: pgmap v55: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 564 KiB/s wr, 89 op/s 2023-10-06T21:22:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:42 smithi031 ceph-mon[98539]: pgmap v55: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 564 KiB/s wr, 89 op/s 2023-10-06T21:22:42.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:42 smithi146 ceph-mon[102809]: pgmap v55: 97 pgs: 97 active+clean; 27 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 564 KiB/s wr, 89 op/s 2023-10-06T21:22:43.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 11332 2.38 MB/sec warmup 33 sec latency 17.571 ms 2023-10-06T21:22:44.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 11704 2.36 MB/sec warmup 34 sec latency 16.452 ms 2023-10-06T21:22:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:44 smithi142 ceph-mon[102951]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 852 KiB/s wr, 112 op/s 2023-10-06T21:22:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:44 smithi031 ceph-mon[98539]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 852 KiB/s wr, 112 op/s 2023-10-06T21:22:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:44 smithi146 ceph-mon[102809]: pgmap v56: 97 pgs: 97 active+clean; 29 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 852 KiB/s wr, 112 op/s 2023-10-06T21:22:45.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 12084 2.38 MB/sec warmup 35 sec latency 20.758 ms 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.637+0000 7f10e1fff700 1 -- 172.21.15.31:0/1364752946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1010d0 msgr2=0x7f10dc1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.637+0000 7f10e1fff700 1 --2- 172.21.15.31:0/1364752946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1010d0 0x7f10dc1014b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009990 tx=0x7f10cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 -- 172.21.15.31:0/1364752946 shutdown_connections 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 --2- 172.21.15.31:0/1364752946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1061f0 0x7f10dc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 --2- 172.21.15.31:0/1364752946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1010d0 0x7f10dc1014b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 --2- 172.21.15.31:0/1364752946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc101a80 0x7f10dc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 -- 172.21.15.31:0/1364752946 >> 172.21.15.31:0/1364752946 conn(0x7f10dc0fc950 msgr2=0x7f10dc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 -- 172.21.15.31:0/1364752946 shutdown_connections 2023-10-06T21:22:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.638+0000 7f10e1fff700 1 -- 172.21.15.31:0/1364752946 wait complete. 2023-10-06T21:22:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.639+0000 7f10e1fff700 1 Processor -- start 2023-10-06T21:22:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.639+0000 7f10e1fff700 1 -- start start 2023-10-06T21:22:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc1010d0 0x7f10dc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1a30d0 0x7f10dc1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc1a5a00 con 0x7f10dc1061f0 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc1a5b40 con 0x7f10dc1010d0 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10e1fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc1a5cb0 con 0x7f10dc1a30d0 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60736/0 (socket says 172.21.15.31:60736) 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.640+0000 7f10daffd700 1 -- 172.21.15.31:0/4177540435 learned_addr learned my addr 172.21.15.31:0/4177540435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10db7fe700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc1010d0 0x7f10dc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10dbfff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1a30d0 0x7f10dc1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 -- 172.21.15.31:0/4177540435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1a30d0 msgr2=0x7f10dc1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1a30d0 0x7f10dc1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 -- 172.21.15.31:0/4177540435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc1010d0 msgr2=0x7f10dc073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc1010d0 0x7f10dc073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 -- 172.21.15.31:0/4177540435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc0095f0 con 0x7f10dc1061f0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10daffd700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e970 tx=0x7f10d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10d8ff9700 1 -- 172.21.15.31:0/4177540435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0009790 con 0x7f10dc1061f0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10d8ff9700 1 -- 172.21.15.31:0/4177540435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10d000eec0 con 0x7f10dc1061f0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.641+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc1a5f90 con 0x7f10dc1061f0 2023-10-06T21:22:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.642+0000 7f10d8ff9700 1 -- 172.21.15.31:0/4177540435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00054b0 con 0x7f10dc1061f0 2023-10-06T21:22:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.642+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc1a6410 con 0x7f10dc1061f0 2023-10-06T21:22:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.646+0000 7f10d8ff9700 1 -- 172.21.15.31:0/4177540435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f10d0004750 con 0x7f10dc1061f0 2023-10-06T21:22:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.646+0000 7f10f59cab80 1 Processor -- start 2023-10-06T21:22:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 -- start start 2023-10-06T21:22:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78edcdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ed37400 con 0x55c78edcdc00 2023-10-06T21:22:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ed28bc0 con 0x55c78edce520 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.647+0000 7f10f59cab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ed264e0 con 0x55c78edd93d0 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51092/0 (socket says 172.21.15.31:51092) 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 -- 172.21.15.31:0/4243719231 learned_addr learned my addr 172.21.15.31:0/4243719231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10dbfff700 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78edcdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10db7fe700 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10d8ff9700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 0x7f10c40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 -- 172.21.15.31:0/4243719231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 msgr2=0x55c78edd8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 -- 172.21.15.31:0/4243719231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 msgr2=0x55c78edcdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78edcdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.648+0000 7f10daffd700 1 -- 172.21.15.31:0/4243719231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c78eb4cd80 con 0x55c78edd93d0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10dbfff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 0x7f10c40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10d8ff9700 1 -- 172.21.15.31:0/4177540435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f10d0014070 con 0x7f10dc1061f0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10daffd700 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ca9a0 tx=0x7f10d006d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b6859fa526f844 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10c1ffb700 1 -- 172.21.15.31:0/4243719231 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d009a0c0 con 0x55c78edd93d0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10c1ffb700 1 -- 172.21.15.31:0/4243719231 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10d0099e30 con 0x55c78edd93d0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.649+0000 7f10c1ffb700 1 -- 172.21.15.31:0/4243719231 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00633e0 con 0x55c78edd93d0 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 -- 172.21.15.31:0/4243719231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 msgr2=0x55c78eddb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10dbfff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 0x7f10c40a3b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10d40075c0 tx=0x7f10d4009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ca9a0 tx=0x7f10d006d260 comp rx=0 tx=0).stop 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 -- 172.21.15.31:0/4243719231 shutdown_connections 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78eddb7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78edcdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 --2- 172.21.15.31:0/4243719231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 -- 172.21.15.31:0/4243719231 >> 172.21.15.31:0/4243719231 conn(0x55c78ed43200 msgr2=0x55c78edca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 -- 172.21.15.31:0/4243719231 shutdown_connections 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.650+0000 7f10f59cab80 1 -- 172.21.15.31:0/4243719231 wait complete. 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 -- start start 2023-10-06T21:22:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 Processor -- start 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ed21dd0 con 0x55c78edd93d0 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ecbe6b0 con 0x55c78edce520 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78ecbe350 con 0x55c78edcdc00 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.651+0000 7f10f59cab80 10 client.0 _ll_register_callbacks cb 0x55c78eb1b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60766/0 (socket says 172.21.15.31:60766) 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10dbfff700 1 -- 172.21.15.31:0/1670204929 learned_addr learned my addr 172.21.15.31:0/1670204929 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51106/0 (socket says 172.21.15.31:51106) 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10daffd700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 -- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 msgr2=0x55c78edd60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 -- 172.21.15.31:0/1670204929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 msgr2=0x55c78edd6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10db7fe700 1 -- 172.21.15.31:0/1670204929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc1a6410 con 0x55c78edcdc00 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.652+0000 7f10dbfff700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10db7fe700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10cc006010 tx=0x7f10cc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394a567899bd6665 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10baffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01d030 con 0x55c78edcdc00 2023-10-06T21:22:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f10cc00bda0 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c78edc1d50 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10cc004500 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10f59cab80 10 client.25918 fetch_fsmap learned FSMap version 30 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.653+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c78eb35c00 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c78edc1d50 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f10cc00f6c0 con 0x55c78edcdc00 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 10 client.25918 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 20 client.25918 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ede8de0 0x55c78edeb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 4 client.25918 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c78ed1adb0 con 0x55c78ede8de0 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.654+0000 7f10daffd700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ede8de0 0x55c78edeb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.655+0000 7f10daffd700 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ede8de0 0x55c78edeb200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34163be66f14fcdb server_cookie=8ced54c5aefdedb0 in_seq=0 out_seq=0 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.655+0000 7f10baffd700 10 client.25918 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10baffd700 1 -- 172.21.15.31:0/1670204929 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c78ed1adb0 con 0x55c78ede8de0 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10baffd700 10 client.25918 handle_command_reply: tid=0 2023-10-06T21:22:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10f59cab80 1 client.25918 shutdown 2023-10-06T21:22:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 msgr2=0x55c78edd8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10cc006010 tx=0x7f10cc0057c0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 shutdown_connections 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.656+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ede8de0 0x55c78edeb200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edd93d0 0x55c78edd6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10baffd700 0 client.25918 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd8db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 --2- 172.21.15.31:0/1670204929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 >> 172.21.15.31:0/1670204929 conn(0x55c78ed43200 msgr2=0x55c78edc9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 shutdown_connections 2023-10-06T21:22:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 1 -- 172.21.15.31:0/1670204929 wait complete. 2023-10-06T21:22:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.657+0000 7f10f59cab80 20 client.25918 trim_cache size 0 max 16384 2023-10-06T21:22:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.673+0000 7f10f59cab80 1 Processor -- start 2023-10-06T21:22:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.673+0000 7f10f59cab80 1 -- start start 2023-10-06T21:22:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.673+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78edd1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10f59cab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78eddbc20 con 0x55c78edcdc00 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10f59cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78eba28c0 con 0x55c78edce520 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10f59cab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78edcadc0 con 0x55c78edd93d0 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60780/0 (socket says 172.21.15.31:60780) 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 -- 172.21.15.31:0/2570793600 learned_addr learned my addr 172.21.15.31:0/2570793600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10daffd700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10dbfff700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78edd1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 -- 172.21.15.31:0/2570793600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 msgr2=0x55c78edd1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78edd1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 -- 172.21.15.31:0/2570793600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 msgr2=0x55c78edd13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10db7fe700 1 -- 172.21.15.31:0/2570793600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c78eb731f0 con 0x55c78edcdc00 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.674+0000 7f10daffd700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.675+0000 7f10db7fe700 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b360 tx=0x7f10cc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b5a3789163fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.675+0000 7f10c27fc700 1 -- 172.21.15.31:0/2570793600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01d070 con 0x55c78edcdc00 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.675+0000 7f10c27fc700 1 -- 172.21.15.31:0/2570793600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10cc01e040 con 0x55c78edcdc00 2023-10-06T21:22:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.675+0000 7f10c27fc700 1 -- 172.21.15.31:0/2570793600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0048c0 con 0x55c78edcdc00 2023-10-06T21:22:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- 172.21.15.31:0/2570793600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 msgr2=0x55c78ede5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b360 tx=0x7f10cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- 172.21.15.31:0/2570793600 shutdown_connections 2023-10-06T21:22:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edd93d0 0x55c78edd1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edcdc00 0x55c78ede5050 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 --2- 172.21.15.31:0/2570793600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edce520 0x55c78edd13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- 172.21.15.31:0/2570793600 >> 172.21.15.31:0/2570793600 conn(0x55c78ed43200 msgr2=0x55c78edefb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- 172.21.15.31:0/2570793600 shutdown_connections 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- 172.21.15.31:0/2570793600 wait complete. 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.676+0000 7f10f59cab80 1 -- start start 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 Processor -- start 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edd93d0 0x55c78edd2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78edd32e0 con 0x55c78edce520 2023-10-06T21:22:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78edd3450 con 0x55c78edd93d0 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78edd3610 con 0x55c78edcdc00 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10f59cab80 10 client.0 _ll_register_callbacks cb 0x7f10cc009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.677+0000 7f10daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60788/0 (socket says 172.21.15.31:60788) 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 -- 172.21.15.31:0/3801355021 learned_addr learned my addr 172.21.15.31:0/3801355021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10db7fe700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10dbfff700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edd93d0 0x55c78edd2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 -- 172.21.15.31:0/3801355021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 msgr2=0x55c78edd2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 -- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edd93d0 msgr2=0x55c78edd2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edd93d0 0x55c78edd2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 -- 172.21.15.31:0/3801355021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c78eb35c00 con 0x55c78edce520 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10daffd700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c5b0 tx=0x7f10d0096730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5369701b111282f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10baffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00b0070 con 0x55c78edce520 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10d0099e30 con 0x55c78edce520 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.678+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f10d000cbd0 con 0x55c78edce520 2023-10-06T21:22:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10d009a9f0 con 0x55c78edce520 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 10 client.16233 fetch_fsmap learned FSMap version 30 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c78eb4b250 con 0x55c78edce520 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10d0099e30 con 0x55c78edce520 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f10d00c9f20 con 0x55c78edce520 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 10 client.16233 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 20 client.16233 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ee3d2d0 0x55c78ee3f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 4 client.16233 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.679+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c78ed1aad0 con 0x55c78ee3d2d0 2023-10-06T21:22:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.680+0000 7f10db7fe700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ee3d2d0 0x55c78ee3f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.680+0000 7f10db7fe700 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ee3d2d0 0x55c78ee3f6f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31943aacaedff184 server_cookie=d04f238f3615da7f in_seq=0 out_seq=0 2023-10-06T21:22:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.680+0000 7f10baffd700 10 client.16233 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.703+0000 7f10baffd700 1 -- 172.21.15.31:0/3801355021 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c78ed1aad0 con 0x55c78ee3d2d0 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.703+0000 7f10baffd700 10 client.16233 handle_command_reply: tid=0 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.703+0000 7f10f59cab80 1 client.16233 shutdown 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 msgr2=0x55c78edd2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c5b0 tx=0x7f10d0096730 comp rx=0 tx=0).stop 2023-10-06T21:22:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 shutdown_connections 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c78edd93d0 0x55c78edd2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c78edcdc00 0x55c78edd2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c78ee3d2d0 0x55c78ee3f6f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 --2- 172.21.15.31:0/3801355021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c78edce520 0x55c78edd2860 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10baffd700 0 client.16233 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 >> 172.21.15.31:0/3801355021 conn(0x55c78ed43200 msgr2=0x55c78ede5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 shutdown_connections 2023-10-06T21:22:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 1 -- 172.21.15.31:0/3801355021 wait complete. 2023-10-06T21:22:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.704+0000 7f10f59cab80 20 client.16233 trim_cache size 0 max 16384 2023-10-06T21:22:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:22:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:22:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:22:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:22:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 msgr2=0x7f10c40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 0x7f10c40a3b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10d40075c0 tx=0x7f10d4009040 comp rx=0 tx=0).stop 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 msgr2=0x7f10dc0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e970 tx=0x7f10d000ec80 comp rx=0 tx=0).stop 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 shutdown_connections 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10dc1a30d0 0x7f10dc1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10dc1010d0 0x7f10dc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f10c40a1670 0x7f10c40a3b30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 --2- 172.21.15.31:0/4177540435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10dc1061f0 0x7f10dc0738d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 >> 172.21.15.31:0/4177540435 conn(0x7f10dc0fc950 msgr2=0x7f10dc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 shutdown_connections 2023-10-06T21:22:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:45.705+0000 7f10e1fff700 1 -- 172.21.15.31:0/4177540435 wait complete. 2023-10-06T21:22:46.328 INFO:tasks.workunit.client.0.smithi031.stdout: 1 12478 2.32 MB/sec warmup 36 sec latency 27.766 ms 2023-10-06T21:22:46.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:46 smithi142 ceph-mon[102951]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 889 KiB/s wr, 120 op/s 2023-10-06T21:22:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:46 smithi031 ceph-mon[98539]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 889 KiB/s wr, 120 op/s 2023-10-06T21:22:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:46 smithi146 ceph-mon[102809]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 889 KiB/s wr, 120 op/s 2023-10-06T21:22:47.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 12902 2.27 MB/sec warmup 37 sec latency 20.336 ms 2023-10-06T21:22:48.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 13223 2.27 MB/sec warmup 38 sec latency 17.438 ms 2023-10-06T21:22:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:48 smithi142 ceph-mon[102951]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 643 KiB/s wr, 73 op/s 2023-10-06T21:22:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:48 smithi031 ceph-mon[98539]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 643 KiB/s wr, 73 op/s 2023-10-06T21:22:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:48 smithi146 ceph-mon[102809]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 643 KiB/s wr, 73 op/s 2023-10-06T21:22:49.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 13817 2.35 MB/sec warmup 39 sec latency 19.837 ms 2023-10-06T21:22:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:49 smithi142 ceph-mon[102951]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 970 KiB/s wr, 122 op/s 2023-10-06T21:22:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:49 smithi031 ceph-mon[98539]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 970 KiB/s wr, 122 op/s 2023-10-06T21:22:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:49 smithi146 ceph-mon[102809]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 970 KiB/s wr, 122 op/s 2023-10-06T21:22:50.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 14010 2.32 MB/sec warmup 40 sec latency 19.235 ms 2023-10-06T21:22:51.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 14412 2.37 MB/sec warmup 41 sec latency 20.063 ms 2023-10-06T21:22:52.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:52 smithi031 ceph-mon[98539]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 754 KiB/s wr, 92 op/s 2023-10-06T21:22:52.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:52 smithi146 ceph-mon[102809]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 754 KiB/s wr, 92 op/s 2023-10-06T21:22:52.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 14582 2.32 MB/sec warmup 42 sec latency 18.449 ms 2023-10-06T21:22:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:52 smithi142 ceph-mon[102951]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 754 KiB/s wr, 92 op/s 2023-10-06T21:22:53.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 14767 2.27 MB/sec warmup 43 sec latency 17.197 ms 2023-10-06T21:22:54.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:54 smithi031 ceph-mon[98539]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:54 smithi146 ceph-mon[102809]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:54.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 15108 2.23 MB/sec warmup 44 sec latency 19.043 ms 2023-10-06T21:22:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:54 smithi142 ceph-mon[102951]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2023-10-06T21:22:55.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 15327 2.21 MB/sec warmup 45 sec latency 382.846 ms 2023-10-06T21:22:56.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:56 smithi031 ceph-mon[98539]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 939 KiB/s wr, 111 op/s 2023-10-06T21:22:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:56 smithi146 ceph-mon[102809]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 939 KiB/s wr, 111 op/s 2023-10-06T21:22:56.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 15704 2.23 MB/sec warmup 46 sec latency 20.309 ms 2023-10-06T21:22:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:56 smithi142 ceph-mon[102951]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 939 KiB/s wr, 111 op/s 2023-10-06T21:22:57.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 16072 2.19 MB/sec warmup 47 sec latency 22.158 ms 2023-10-06T21:22:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:58 smithi146 ceph-mon[102809]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 799 KiB/s wr, 91 op/s 2023-10-06T21:22:58.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 16530 2.17 MB/sec warmup 48 sec latency 19.237 ms 2023-10-06T21:22:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:58 smithi142 ceph-mon[102951]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 799 KiB/s wr, 91 op/s 2023-10-06T21:22:58.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:58 smithi031 ceph-mon[98539]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 799 KiB/s wr, 91 op/s 2023-10-06T21:22:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 -- 172.21.15.31:0/4270033333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 msgr2=0x7f2d08068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 --2- 172.21.15.31:0/4270033333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d08068750 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80098f0 tx=0x7f2cf8009c00 comp rx=0 tx=0).stop 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 -- 172.21.15.31:0/4270033333 shutdown_connections 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 --2- 172.21.15.31:0/4270033333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08068c90 0x7f2d0810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 --2- 172.21.15.31:0/4270033333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08106f60 0x7f2d08107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.864+0000 7f2d103ff700 1 --2- 172.21.15.31:0/4270033333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d08068750 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.865+0000 7f2d103ff700 1 -- 172.21.15.31:0/4270033333 >> 172.21.15.31:0/4270033333 conn(0x7f2d08076b70 msgr2=0x7f2d08076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.865+0000 7f2d103ff700 1 -- 172.21.15.31:0/4270033333 shutdown_connections 2023-10-06T21:22:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.865+0000 7f2d103ff700 1 -- 172.21.15.31:0/4270033333 wait complete. 2023-10-06T21:22:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.866+0000 7f2d103ff700 1 Processor -- start 2023-10-06T21:22:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.866+0000 7f2d103ff700 1 -- start start 2023-10-06T21:22:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d081990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08106f60 0x7f2d0819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819e2c0 con 0x7f2d080682d0 2023-10-06T21:22:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819e430 con 0x7f2d08106f60 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d103ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819e5f0 con 0x7f2d08068c90 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d0e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d081990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48374/0 (socket says 172.21.15.31:48374) 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d0d99a700 1 -- 172.21.15.31:0/66393273 learned_addr learned my addr 172.21.15.31:0/66393273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.867+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08106f60 0x7f2d0819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 -- 172.21.15.31:0/66393273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 msgr2=0x7f2d08199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 -- 172.21.15.31:0/66393273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08106f60 msgr2=0x7f2d0819dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08106f60 0x7f2d0819dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 -- 172.21.15.31:0/66393273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009c40 con 0x7f2d080682d0 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d081990d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400e9c0 tx=0x7f2d0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2cff7fe700 1 -- 172.21.15.31:0/66393273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004220 con 0x7f2d080682d0 2023-10-06T21:22:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2cff7fe700 1 -- 172.21.15.31:0/66393273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d04004cc0 con 0x7f2d080682d0 2023-10-06T21:22:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2cff7fe700 1 -- 172.21.15.31:0/66393273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400cc60 con 0x7f2d080682d0 2023-10-06T21:22:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.868+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf8009520 con 0x7f2d080682d0 2023-10-06T21:22:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.869+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d081a5f30 con 0x7f2d080682d0 2023-10-06T21:22:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.872+0000 7f2cff7fe700 1 -- 172.21.15.31:0/66393273 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2d04004750 con 0x7f2d080682d0 2023-10-06T21:22:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 Processor -- start 2023-10-06T21:22:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 -- start start 2023-10-06T21:22:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 0x7f2cf40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2cff7fe700 1 -- 172.21.15.31:0/66393273 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2d04014070 con 0x7f2d080682d0 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb113400 con 0x55ebfb1aa520 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb104bc0 con 0x55ebfb1b53d0 2023-10-06T21:22:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.874+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb1024e0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38920/0 (socket says 172.21.15.31:38920) 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0d99a700 1 -- 172.21.15.31:0/996011522 learned_addr learned my addr 172.21.15.31:0/996011522 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 0x7f2cf40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.875+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 -- 172.21.15.31:0/996011522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1b53d0 msgr2=0x55ebfb1b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 -- 172.21.15.31:0/996011522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 msgr2=0x55ebfb1b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 -- 172.21.15.31:0/996011522 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfaf28d80 con 0x55ebfb1a9c00 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 0x7f2cf40a3ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011f80 tx=0x7f2d00009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.876+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1a9fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d040ca610 tx=0x7f2d040999e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c5c5115f060544 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.877+0000 7f2cfcbfe700 1 -- 172.21.15.31:0/996011522 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04099ea0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.877+0000 7f2cfcbfe700 1 -- 172.21.15.31:0/996011522 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d0409dff0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.877+0000 7f2cfcbfe700 1 -- 172.21.15.31:0/996011522 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0409a100 con 0x55ebfb1a9c00 2023-10-06T21:22:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 -- 172.21.15.31:0/996011522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 msgr2=0x55ebfb1a9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1a9fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d040ca610 tx=0x7f2d040999e0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 -- 172.21.15.31:0/996011522 shutdown_connections 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1a9fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/996011522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 -- 172.21.15.31:0/996011522 >> 172.21.15.31:0/996011522 conn(0x55ebfb11f200 msgr2=0x55ebfb1a6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 -- 172.21.15.31:0/996011522 shutdown_connections 2023-10-06T21:22:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.878+0000 7f2d23d92b80 1 -- 172.21.15.31:0/996011522 wait complete. 2023-10-06T21:22:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.879+0000 7f2d23d92b80 1 -- start start 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.879+0000 7f2d23d92b80 1 Processor -- start 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb0fddd0 con 0x55ebfb1aa520 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb09a6b0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb09a350 con 0x55ebfb1b53d0 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d23d92b80 10 client.0 _ll_register_callbacks cb 0x55ebfaef77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38924/0 (socket says 172.21.15.31:38924) 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d0d99a700 1 -- 172.21.15.31:0/454675702 learned_addr learned my addr 172.21.15.31:0/454675702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.880+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 -- 172.21.15.31:0/454675702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 msgr2=0x55ebfb1b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 -- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 msgr2=0x55ebfb1b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 -- 172.21.15.31:0/454675702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0810e5c0 con 0x55ebfb1aa520 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800b790 tx=0x7f2cf800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6e3fcd6e85ecd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.881+0000 7f2cf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:22:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf801c040 con 0x55ebfb1aa520 2023-10-06T21:22:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2cf8005240 con 0x55ebfb1aa520 2023-10-06T21:22:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebfb19dd50 con 0x55ebfb1aa520 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf800bee0 con 0x55ebfb1aa520 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2d23d92b80 10 client.16245 fetch_fsmap learned FSMap version 30 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.882+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2d081a5f30 con 0x55ebfb1aa520 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ebfb19dd50 con 0x55ebfb1aa520 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2cf8015020 con 0x55ebfb1aa520 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 20 client.16245 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb1c4fc0 0x55ebfb1c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 4 client.16245 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.883+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebfb0f6db0 con 0x55ebfb1c4fc0 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.884+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb1c4fc0 0x55ebfb1c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.884+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb1c4fc0 0x55ebfb1c73e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e4b11918be5ac8 server_cookie=8b2edd9036ef15ef in_seq=0 out_seq=0 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.884+0000 7f2cf1ffb700 10 client.16245 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/454675702 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ebfb0f6db0 con 0x55ebfb1c4fc0 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2cf1ffb700 10 client.16245 handle_command_reply: tid=0 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 client.16245 shutdown 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 msgr2=0x55ebfb1b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800b790 tx=0x7f2cf800b7c0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 shutdown_connections 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb1c4fc0 0x55ebfb1c73e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/454675702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1aa520 0x55ebfb1b26f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 >> 172.21.15.31:0/454675702 conn(0x55ebfb11f200 msgr2=0x55ebfb1a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2cf1ffb700 0 client.16245 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 shutdown_connections 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.885+0000 7f2d23d92b80 1 -- 172.21.15.31:0/454675702 wait complete. 2023-10-06T21:22:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.886+0000 7f2d23d92b80 20 client.16245 trim_cache size 0 max 16384 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 Processor -- start 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 -- start start 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb1af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfaf7e8c0 con 0x55ebfb1b53d0 2023-10-06T21:22:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb1a2860 con 0x55ebfb1aa520 2023-10-06T21:22:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.904+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb1a6dc0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb1af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48400/0 (socket says 172.21.15.31:48400) 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 -- 172.21.15.31:0/1654273190 learned_addr learned my addr 172.21.15.31:0/1654273190 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 -- 172.21.15.31:0/1654273190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 msgr2=0x55ebfb1af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb1af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 -- 172.21.15.31:0/1654273190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1b53d0 msgr2=0x55ebfb1af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 -- 172.21.15.31:0/1654273190 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfaf11c00 con 0x55ebfb1a9c00 2023-10-06T21:22:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.905+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d04099b50 tx=0x7f2d04009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6958e995733cba05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.906+0000 7f2cfd3ff700 1 -- 172.21.15.31:0/1654273190 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04099f20 con 0x55ebfb1a9c00 2023-10-06T21:22:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.906+0000 7f2cfd3ff700 1 -- 172.21.15.31:0/1654273190 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d04098240 con 0x55ebfb1a9c00 2023-10-06T21:22:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.906+0000 7f2cfd3ff700 1 -- 172.21.15.31:0/1654273190 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040995f0 con 0x55ebfb1a9c00 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.906+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1654273190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 msgr2=0x55ebfb1bbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.906+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d04099b50 tx=0x7f2d04009f50 comp rx=0 tx=0).stop 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1654273190 shutdown_connections 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1b53d0 0x55ebfb1af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1a9c00 0x55ebfb1bbf20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1654273190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb1af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1654273190 >> 172.21.15.31:0/1654273190 conn(0x55ebfb11f200 msgr2=0x55ebfb1ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1654273190 shutdown_connections 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1654273190 wait complete. 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 -- start start 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.907+0000 7f2d23d92b80 1 Processor -- start 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1a9c00 0x55ebfb20cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb217e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb218340 con 0x55ebfb1a9c00 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb218480 con 0x55ebfb1aa520 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb2185c0 con 0x55ebfb1b53d0 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d23d92b80 10 client.0 _ll_register_callbacks cb 0x7f2cf800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d0e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1a9c00 0x55ebfb20cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d0d99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39992/0 (socket says 172.21.15.31:39992) 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d0d99a700 1 -- 172.21.15.31:0/1357093402 learned_addr learned my addr 172.21.15.31:0/1357093402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.908+0000 7f2d0e99c700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb217e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 -- 172.21.15.31:0/1357093402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 msgr2=0x55ebfb217e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb217e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 -- 172.21.15.31:0/1357093402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1a9c00 msgr2=0x55ebfb20cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1a9c00 0x55ebfb20cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 -- 172.21.15.31:0/1357093402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebfaf28d80 con 0x55ebfb1aa520 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d0d99a700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005e90 tx=0x7f2cf8003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c02683134058c52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2cf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf801c040 con 0x55ebfb1aa520 2023-10-06T21:22:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.909+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d0409dff0 con 0x55ebfb1aa520 2023-10-06T21:22:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2cf80058e0 con 0x55ebfb1aa520 2023-10-06T21:22:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf800bee0 con 0x55ebfb1aa520 2023-10-06T21:22:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 10 client.25985 fetch_fsmap learned FSMap version 30 2023-10-06T21:22:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ebfaf4f1f0 con 0x55ebfb1aa520 2023-10-06T21:22:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d0409dff0 con 0x55ebfb1aa520 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2cf8009c70 con 0x55ebfb1aa520 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 20 client.25985 populate_metadata read hostname 'smithi031' 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb219920 0x55ebfb21bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 4 client.25985 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.910+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebfb0f6ad0 con 0x55ebfb219920 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.911+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb219920 0x55ebfb21bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.911+0000 7f2d0e19b700 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb219920 0x55ebfb21bd40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b503060edefc3433 server_cookie=d5344b23e3948f97 in_seq=0 out_seq=0 2023-10-06T21:22:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.911+0000 7f2cf1ffb700 10 client.25985 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2cf1ffb700 1 -- 172.21.15.31:0/1357093402 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebfb0f6ad0 con 0x55ebfb219920 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2cf1ffb700 10 client.25985 handle_command_reply: tid=0 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 client.25985 shutdown 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 msgr2=0x55ebfb20d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005e90 tx=0x7f2cf8003ce0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 shutdown_connections 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ebfb1b53d0 0x55ebfb217e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ebfb219920 0x55ebfb21bd40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ebfb1a9c00 0x55ebfb20cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 --2- 172.21.15.31:0/1357093402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ebfb1aa520 0x55ebfb20d4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2cf1ffb700 0 client.25985 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 >> 172.21.15.31:0/1357093402 conn(0x55ebfb11f200 msgr2=0x55ebfb1b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 shutdown_connections 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 1 -- 172.21.15.31:0/1357093402 wait complete. 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.933+0000 7f2d23d92b80 20 client.25985 trim_cache size 0 max 16384 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:22:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:22:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 msgr2=0x7f2cf40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 0x7f2cf40a3ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011f80 tx=0x7f2d00009920 comp rx=0 tx=0).stop 2023-10-06T21:22:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 msgr2=0x7f2d081990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d081990d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400e9c0 tx=0x7f2d0400ecd0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 shutdown_connections 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d08106f60 0x7f2d0819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2cf40a15f0 0x7f2cf40a3ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d080682d0 0x7f2d081990d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 --2- 172.21.15.31:0/66393273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d08068c90 0x7f2d08199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 >> 172.21.15.31:0/66393273 conn(0x7f2d08076b70 msgr2=0x7f2d080feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 shutdown_connections 2023-10-06T21:22:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:22:58.934+0000 7f2d103ff700 1 -- 172.21.15.31:0/66393273 wait complete. 2023-10-06T21:22:59.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 16795 2.15 MB/sec warmup 49 sec latency 24.033 ms 2023-10-06T21:23:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:22:59 smithi142 ceph-mon[102951]: pgmap v64: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-06T21:23:00.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:22:59 smithi031 ceph-mon[98539]: pgmap v64: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-06T21:23:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:22:59 smithi146 ceph-mon[102809]: pgmap v64: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-06T21:23:00.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 17398 2.22 MB/sec warmup 50 sec latency 25.276 ms 2023-10-06T21:23:00.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:23:00.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 -- 172.21.15.31:0/4021292076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 msgr2=0x7fec5810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 --2- 172.21.15.31:0/4021292076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec5810ae50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fec48009990 tx=0x7fec48009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 -- 172.21.15.31:0/4021292076 shutdown_connections 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 --2- 172.21.15.31:0/4021292076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec5810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 --2- 172.21.15.31:0/4021292076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec58102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 --2- 172.21.15.31:0/4021292076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec5810ae50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 -- 172.21.15.31:0/4021292076 >> 172.21.15.31:0/4021292076 conn(0x7fec580fb890 msgr2=0x7fec580fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 -- 172.21.15.31:0/4021292076 shutdown_connections 2023-10-06T21:23:00.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.617+0000 7fec5fbff700 1 -- 172.21.15.31:0/4021292076 wait complete. 2023-10-06T21:23:00.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.618+0000 7fec5fbff700 1 Processor -- start 2023-10-06T21:23:00.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 -- start start 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec581001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec581a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58100a10 con 0x7fec58102970 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a5660 con 0x7fec58102050 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a57a0 con 0x7fec5810b470 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40008/0 (socket says 172.21.15.31:40008) 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5d99b700 1 -- 172.21.15.31:0/3562783130 learned_addr learned my addr 172.21.15.31:0/3562783130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.619+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec581001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5e19c700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec581a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 -- 172.21.15.31:0/3562783130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 msgr2=0x7fec581a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec581a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 -- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 msgr2=0x7fec581001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec581001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 -- 172.21.15.31:0/3562783130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec50009c40 con 0x7fec58102050 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5e19c700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec581a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d19a700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec581001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e9a0 tx=0x7fec5400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec4effd700 1 -- 172.21.15.31:0/3562783130 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54009790 con 0x7fec58102050 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec480095f0 con 0x7fec58102050 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec4effd700 1 -- 172.21.15.31:0/3562783130 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec5400cbd0 con 0x7fec58102050 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.620+0000 7fec4effd700 1 -- 172.21.15.31:0/3562783130 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54018680 con 0x7fec58102050 2023-10-06T21:23:00.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.621+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a5d50 con 0x7fec58102050 2023-10-06T21:23:00.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.624+0000 7fec4effd700 1 -- 172.21.15.31:0/3562783130 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fec5400cd40 con 0x7fec58102050 2023-10-06T21:23:00.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.625+0000 7fec7357cb80 1 Processor -- start 2023-10-06T21:23:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 -- start start 2023-10-06T21:23:00.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec4effd700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec4effd700 1 -- 172.21.15.31:0/3562783130 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fec54014070 con 0x7fec58102050 2023-10-06T21:23:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 0x563079534350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563079534890 0x563079536c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079492460 con 0x563079534890 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079483c20 con 0x5630795299e0 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec7357cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079481540 con 0x5630795290c0 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.626+0000 7fec5e19c700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48430/0 (socket says 172.21.15.31:48430) 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 -- 172.21.15.31:0/3510791844 learned_addr learned my addr 172.21.15.31:0/3510791844 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5e19c700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563079534890 0x563079536c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d19a700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 0x563079534350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 -- 172.21.15.31:0/3510791844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 msgr2=0x563079534350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 0x563079534350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 -- 172.21.15.31:0/3510791844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563079534890 msgr2=0x563079536c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563079534890 0x563079536c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 -- 172.21.15.31:0/3510791844 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630792a7de0 con 0x5630795290c0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d19a700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 0x563079534350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5e19c700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 0x7fec440a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec58101db0 tx=0x7fec50007c80 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec5d99b700 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec54096de0 tx=0x7fec54099cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a84fe58989b28b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.627+0000 7fec43fff700 1 -- 172.21.15.31:0/3510791844 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5409e4c0 con 0x5630795290c0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.628+0000 7fec43fff700 1 -- 172.21.15.31:0/3510791844 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec540a2700 con 0x5630795290c0 2023-10-06T21:23:00.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.628+0000 7fec43fff700 1 -- 172.21.15.31:0/3510791844 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54099860 con 0x5630795290c0 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.628+0000 7fec7357cb80 1 -- 172.21.15.31:0/3510791844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 msgr2=0x5630795294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.628+0000 7fec7357cb80 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec54096de0 tx=0x7fec54099cd0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 -- 172.21.15.31:0/3510791844 shutdown_connections 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563079534890 0x563079536c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x5630795294a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 --2- 172.21.15.31:0/3510791844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795299e0 0x563079534350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 -- 172.21.15.31:0/3510791844 >> 172.21.15.31:0/3510791844 conn(0x56307949e260 msgr2=0x563079525450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 -- 172.21.15.31:0/3510791844 shutdown_connections 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 -- 172.21.15.31:0/3510791844 wait complete. 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 -- start start 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.629+0000 7fec7357cb80 1 Processor -- start 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 0x5630795342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307947ce30 con 0x5630795290c0 2023-10-06T21:23:00.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079419710 con 0x563079534890 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630794193b0 con 0x5630795299e0 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec7357cb80 10 client.0 _ll_register_callbacks cb 0x563079276800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 0x5630795342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec5e19c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48434/0 (socket says 172.21.15.31:48434) 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.630+0000 7fec5d19a700 1 -- 172.21.15.31:0/2434508348 learned_addr learned my addr 172.21.15.31:0/2434508348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 -- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 msgr2=0x563079531cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 -- 172.21.15.31:0/2434508348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 msgr2=0x5630795342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 0x5630795342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 -- 172.21.15.31:0/2434508348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a5d50 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d99b700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 0x5630795342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec5d19a700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec48009990 tx=0x7fec48005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a866413d049ac7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.631+0000 7fec417fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48004910 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56307951cf10 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fec48004a70 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec48004c30 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec7357cb80 10 client.25954 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563079290c60 con 0x5630795299e0 2023-10-06T21:23:00.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.632+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56307951cf10 con 0x5630795299e0 2023-10-06T21:23:00.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fec4800f9d0 con 0x5630795299e0 2023-10-06T21:23:00.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 10 client.25954 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:00.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:00.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 20 client.25954 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079544380 0x5630795467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 4 client.25954 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563079475e10 con 0x563079544380 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.633+0000 7fec5d99b700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079544380 0x5630795467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.634+0000 7fec417fa700 10 client.25954 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.634+0000 7fec5d99b700 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079544380 0x5630795467a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3ad2ec98f0ad81 server_cookie=573b6ac99ed35ad2 in_seq=0 out_seq=0 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec417fa700 1 -- 172.21.15.31:0/2434508348 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563079475e10 con 0x563079544380 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec417fa700 10 client.25954 handle_command_reply: tid=0 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 client.25954 shutdown 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 msgr2=0x563079531570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec48009990 tx=0x7fec48005270 comp rx=0 tx=0).stop 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 shutdown_connections 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079544380 0x5630795467a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795290c0 0x5630795342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 --2- 172.21.15.31:0/2434508348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795299e0 0x563079531570 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec417fa700 0 client.25954 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 >> 172.21.15.31:0/2434508348 conn(0x56307949e260 msgr2=0x563079524fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 shutdown_connections 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.635+0000 7fec7357cb80 1 -- 172.21.15.31:0/2434508348 wait complete. 2023-10-06T21:23:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.636+0000 7fec7357cb80 20 client.25954 trim_cache size 0 max 16384 2023-10-06T21:23:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 Processor -- start 2023-10-06T21:23:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 -- start start 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x56307952c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630792fd920 con 0x5630795299e0 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079528f30 con 0x563079534890 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.659+0000 7fec7357cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630795262b0 con 0x5630795290c0 2023-10-06T21:23:00.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5e19c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38986/0 (socket says 172.21.15.31:38986) 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 -- 172.21.15.31:0/259813001 learned_addr learned my addr 172.21.15.31:0/259813001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d99b700 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x56307952c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 -- 172.21.15.31:0/259813001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 msgr2=0x56307952c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x56307952c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 -- 172.21.15.31:0/259813001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 msgr2=0x563079531290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 -- 172.21.15.31:0/259813001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630792ce250 con 0x5630795299e0 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d99b700 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x56307952c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:00.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.660+0000 7fec5d19a700 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21e393594064192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec4cbff700 1 -- 172.21.15.31:0/259813001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48005bc0 con 0x5630795299e0 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec4cbff700 1 -- 172.21.15.31:0/259813001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4801e050 con 0x5630795299e0 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec4cbff700 1 -- 172.21.15.31:0/259813001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480239c0 con 0x5630795299e0 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 -- 172.21.15.31:0/259813001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 msgr2=0x563079530d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48009310 comp rx=0 tx=0).stop 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 -- 172.21.15.31:0/259813001 shutdown_connections 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563079534890 0x563079531290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630795290c0 0x56307952c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 --2- 172.21.15.31:0/259813001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079530d50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 -- 172.21.15.31:0/259813001 >> 172.21.15.31:0/259813001 conn(0x56307949e260 msgr2=0x56307952c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 -- 172.21.15.31:0/259813001 shutdown_connections 2023-10-06T21:23:00.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.661+0000 7fec7357cb80 1 -- 172.21.15.31:0/259813001 wait complete. 2023-10-06T21:23:00.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.662+0000 7fec7357cb80 1 -- start start 2023-10-06T21:23:00.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.662+0000 7fec7357cb80 1 Processor -- start 2023-10-06T21:23:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079539900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563079534890 0x563079597190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307953a1f0 con 0x5630795299e0 2023-10-06T21:23:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630795976d0 con 0x5630795290c0 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563079597840 con 0x563079534890 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec7357cb80 10 client.0 _ll_register_callbacks cb 0x7fec5806a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec5d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079539900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.663+0000 7fec5d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40048/0 (socket says 172.21.15.31:40048) 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 -- 172.21.15.31:0/1712471231 learned_addr learned my addr 172.21.15.31:0/1712471231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5e19c700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563079534890 0x563079597190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 -- 172.21.15.31:0/1712471231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563079534890 msgr2=0x563079597190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563079534890 0x563079597190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 -- 172.21.15.31:0/1712471231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 msgr2=0x563079539900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079539900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 -- 172.21.15.31:0/1712471231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630792a7de0 con 0x5630795290c0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d19a700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079539900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec5d99b700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c5b0 tx=0x7fec54099fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea76762c64e5c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.664+0000 7fec417fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5409e4c0 con 0x5630795290c0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec540a2700 con 0x5630795290c0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fec540b6070 con 0x5630795290c0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec54098710 con 0x5630795290c0 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 10 client.26000 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563079290c60 con 0x5630795290c0 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec540a2700 con 0x5630795290c0 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fec5409a5d0 con 0x5630795290c0 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 20 client.26000 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079598d90 0x56307959b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 4 client.26000 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.665+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563079475b30 con 0x563079598d90 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.666+0000 7fec5d19a700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079598d90 0x56307959b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.666+0000 7fec417fa700 10 client.26000 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.666+0000 7fec5d19a700 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079598d90 0x56307959b1b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c38886bf6280800 server_cookie=86807d52a147c6db in_seq=0 out_seq=0 2023-10-06T21:23:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec417fa700 1 -- 172.21.15.31:0/1712471231 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563079475b30 con 0x563079598d90 2023-10-06T21:23:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec417fa700 10 client.26000 handle_command_reply: tid=0 2023-10-06T21:23:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 client.26000 shutdown 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 msgr2=0x5630795393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c5b0 tx=0x7fec54099fb0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 shutdown_connections 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563079534890 0x563079597190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630795290c0 0x5630795393c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563079598d90 0x56307959b1b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 --2- 172.21.15.31:0/1712471231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630795299e0 0x563079539900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 >> 172.21.15.31:0/1712471231 conn(0x56307949e260 msgr2=0x563079532e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec417fa700 0 client.26000 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 shutdown_connections 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 1 -- 172.21.15.31:0/1712471231 wait complete. 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.667+0000 7fec7357cb80 20 client.26000 trim_cache size 0 max 16384 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-10-06T21:23:00.675 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 msgr2=0x7fec440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 0x7fec440a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec58101db0 tx=0x7fec50007c80 comp rx=0 tx=0).stop 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 msgr2=0x7fec580ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e9a0 tx=0x7fec5400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 shutdown_connections 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5810b470 0x7fec581a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec58102050 0x7fec580ffca0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 --2- 172.21.15.31:0/3562783130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58102970 0x7fec581001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 >> 172.21.15.31:0/3562783130 conn(0x7fec580fb890 msgr2=0x7fec580fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 shutdown_connections 2023-10-06T21:23:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.668+0000 7fec5fbff700 1 -- 172.21.15.31:0/3562783130 wait complete. 2023-10-06T21:23:00.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:23:00.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 -- 172.21.15.31:0/1203136342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc101a80 msgr2=0x7f42dc105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 --2- 172.21.15.31:0/1203136342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc101a80 0x7f42dc105ac0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009990 tx=0x7f42cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 -- 172.21.15.31:0/1203136342 shutdown_connections 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 --2- 172.21.15.31:0/1203136342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 --2- 172.21.15.31:0/1203136342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc1010d0 0x7f42dc1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 --2- 172.21.15.31:0/1203136342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc101a80 0x7f42dc105ac0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 -- 172.21.15.31:0/1203136342 >> 172.21.15.31:0/1203136342 conn(0x7f42dc0fc990 msgr2=0x7f42dc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 -- 172.21.15.31:0/1203136342 shutdown_connections 2023-10-06T21:23:00.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.940+0000 7f42e3dff700 1 -- 172.21.15.31:0/1203136342 wait complete. 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 Processor -- start 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 -- start start 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 0x7f42dc0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 0x7f42dc1a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1a5790 con 0x7f42dc1010d0 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1a5900 con 0x7f42dc1061f0 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.941+0000 7f42e3dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1a5ac0 con 0x7f42dc073c30 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e1b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 0x7f42dc0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40058/0 (socket says 172.21.15.31:40058) 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 -- 172.21.15.31:0/160304773 learned_addr learned my addr 172.21.15.31:0/160304773 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e239c700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 0x7f42dc1a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 -- 172.21.15.31:0/160304773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 msgr2=0x7f42dc1a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 0x7f42dc1a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 -- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 msgr2=0x7f42dc0731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 0x7f42dc0731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e139a700 1 -- 172.21.15.31:0/160304773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0095f0 con 0x7f42dc1061f0 2023-10-06T21:23:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e239c700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 0x7f42dc1a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.942+0000 7f42e1b9b700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 0x7f42dc0731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:00.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42e139a700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005770 tx=0x7f42cc0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc01d070 con 0x7f42dc1061f0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc1a5d40 con 0x7f42dc1061f0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42cc022470 con 0x7f42dc1061f0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc017650 con 0x7f42dc1061f0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.943+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1a6160 con 0x7f42dc1061f0 2023-10-06T21:23:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.944+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc067cd0 con 0x7f42dc1061f0 2023-10-06T21:23:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.946+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f42cc022850 con 0x7f42dc1061f0 2023-10-06T21:23:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.948+0000 7f42d2ffd700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 0x7f42c80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.948+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f42cc0ce820 con 0x7f42dc1061f0 2023-10-06T21:23:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.948+0000 7f42e1b9b700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 0x7f42c80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.949+0000 7f42e1b9b700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 0x7f42c80a3af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42d8007fd0 tx=0x7f42d80073d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:00.953+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f42cc097b20 con 0x7f42dc1061f0 2023-10-06T21:23:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.276+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42dc101b80 con 0x7f42dc1061f0 2023-10-06T21:23:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.277+0000 7f42d2ffd700 1 -- 172.21.15.31:0/160304773 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11382 (secure 0 0 0) 0x7f42cc09b140 con 0x7f42dc1061f0 2023-10-06T21:23:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:23:01.280 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3223954371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3223954371},{"type":"v1","addr":"172.21.15.31:6835","nonce":3223954371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2454240095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":2454240095},{"type":"v1","addr":"172.21.15.31:6837","nonce":2454240095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:18:54.346752+0000","modified":"2023-10-06T21:22:34.852095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24526,"mds_3":24518,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14694":{"gid":14694,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/1272583602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":1272583602},{"type":"v1","addr":"172.21.15.31:6839","nonce":1272583602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/847340369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":847340369},{"type":"v1","addr":"172.21.15.31:6841","nonce":847340369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1755294522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1755294522},{"type":"v1","addr":"172.21.15.146:6833","nonce":1755294522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6833/1989266338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1989266338},{"type":"v1","addr":"172.21.15.142:6833","nonce":1989266338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.142:6835/4077995395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4077995395},{"type":"v1","addr":"172.21.15.142:6835","nonce":4077995395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":63,"addr":"172.21.15.146:6835/3172848071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3172848071},{"type":"v1","addr":"172.21.15.146:6835","nonce":3172848071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/824494679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":824494679},{"type":"v1","addr":"172.21.15.146:6837","nonce":824494679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/1999670279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1999670279},{"type":"v1","addr":"172.21.15.142:6837","nonce":1999670279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/1025732473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1025732473},{"type":"v1","addr":"172.21.15.146:6839","nonce":1025732473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2643221655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2643221655},{"type":"v1","addr":"172.21.15.142:6839","nonce":2643221655}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 msgr2=0x7f42c80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 0x7f42c80a3af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42d8007fd0 tx=0x7f42d80073d0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 msgr2=0x7f42dc0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005770 tx=0x7f42cc0057a0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 shutdown_connections 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42dc073c30 0x7f42dc1a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f42c80a1630 0x7f42c80a3af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc1010d0 0x7f42dc0731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 --2- 172.21.15.31:0/160304773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc1061f0 0x7f42dc0736f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 >> 172.21.15.31:0/160304773 conn(0x7f42dc0fc990 msgr2=0x7f42dc104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 shutdown_connections 2023-10-06T21:23:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.280+0000 7f42e3dff700 1 -- 172.21.15.31:0/160304773 wait complete. 2023-10-06T21:23:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2023-10-06T21:23:01.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T21:23:01.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 17583 2.20 MB/sec warmup 51 sec latency 26.725 ms 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.525+0000 7f2be6bff700 1 -- 172.21.15.31:0/293610504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01010e0 msgr2=0x7f2be01014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.525+0000 7f2be6bff700 1 --2- 172.21.15.31:0/293610504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01010e0 0x7f2be01014c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40098f0 tx=0x7f2bd4009c00 comp rx=0 tx=0).stop 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 -- 172.21.15.31:0/293610504 shutdown_connections 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 --2- 172.21.15.31:0/293610504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 0x7f2be010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 --2- 172.21.15.31:0/293610504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01010e0 0x7f2be01014c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 --2- 172.21.15.31:0/293610504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be0101a90 0x7f2be0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 -- 172.21.15.31:0/293610504 >> 172.21.15.31:0/293610504 conn(0x7f2be00fc960 msgr2=0x7f2be00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 -- 172.21.15.31:0/293610504 shutdown_connections 2023-10-06T21:23:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.526+0000 7f2be6bff700 1 -- 172.21.15.31:0/293610504 wait complete. 2023-10-06T21:23:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.527+0000 7f2be6bff700 1 Processor -- start 2023-10-06T21:23:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.527+0000 7f2be6bff700 1 -- start start 2023-10-06T21:23:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.527+0000 7f2be6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be6bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 0x7f2be010d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be6bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 0x7f2be01a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a2950 con 0x7f2be01010e0 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be6bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a2ac0 con 0x7f2be01a1f00 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be6bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a2c80 con 0x7f2be0106200 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39028/0 (socket says 172.21.15.31:39028) 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 -- 172.21.15.31:0/4072255172 learned_addr learned my addr 172.21.15.31:0/4072255172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be519c700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 0x7f2be01a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2bdffff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 0x7f2be010d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 -- 172.21.15.31:0/4072255172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 msgr2=0x7f2be010d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 0x7f2be010d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 -- 172.21.15.31:0/4072255172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 msgr2=0x7f2be01a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 0x7f2be01a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be499b700 1 -- 172.21.15.31:0/4072255172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4009520 con 0x7f2be01010e0 2023-10-06T21:23:01.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.528+0000 7f2be519c700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 0x7f2be01a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:01.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2be499b700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009810 tx=0x7f2bd400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2bddffb700 1 -- 172.21.15.31:0/4072255172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd401c070 con 0x7f2be01010e0 2023-10-06T21:23:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be01aa1b0 con 0x7f2be01010e0 2023-10-06T21:23:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2bddffb700 1 -- 172.21.15.31:0/4072255172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bd400bba0 con 0x7f2be01010e0 2023-10-06T21:23:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2bddffb700 1 -- 172.21.15.31:0/4072255172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4004930 con 0x7f2be01010e0 2023-10-06T21:23:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.529+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01aa680 con 0x7f2be01010e0 2023-10-06T21:23:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.533+0000 7f2bddffb700 1 -- 172.21.15.31:0/4072255172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2bd4020070 con 0x7f2be01010e0 2023-10-06T21:23:01.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bfa59eb80 1 Processor -- start 2023-10-06T21:23:01.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bddffb700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 0x7f2bd00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bfa59eb80 1 -- start start 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bddffb700 1 -- 172.21.15.31:0/4072255172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2bd40cdeb0 con 0x7f2be01010e0 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2be519c700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 0x7f2bd00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.535+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c958460 con 0x55f97c9fa890 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c949c20 con 0x55f97c9ef0c0 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c947540 con 0x55f97c9ef9e0 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40086/0 (socket says 172.21.15.31:40086) 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 -- 172.21.15.31:0/1701007554 learned_addr learned my addr 172.21.15.31:0/1701007554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2bdffff700 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be519c700 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 -- 172.21.15.31:0/1701007554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 msgr2=0x55f97c9fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 -- 172.21.15.31:0/1701007554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 msgr2=0x55f97c9fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be499b700 1 -- 172.21.15.31:0/1701007554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97c76dde0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.536+0000 7f2be519c700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 0x7f2bd00a3ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8007610 tx=0x7f2bd80079d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.537+0000 7f2be499b700 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40a2c40 tx=0x7f2bd409aab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465f47f7195f812e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.537+0000 7f2bcaffd700 1 -- 172.21.15.31:0/1701007554 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd409ae70 con 0x55f97c9ef0c0 2023-10-06T21:23:01.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.537+0000 7f2bcaffd700 1 -- 172.21.15.31:0/1701007554 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bd409efc0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.537+0000 7f2bcaffd700 1 -- 172.21.15.31:0/1701007554 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40051d0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/1701007554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 msgr2=0x55f97c9ef4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40a2c40 tx=0x7f2bd409aab0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/1701007554 shutdown_connections 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ef4a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/1701007554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/1701007554 >> 172.21.15.31:0/1701007554 conn(0x55f97c964260 msgr2=0x55f97c9eb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/1701007554 shutdown_connections 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/1701007554 wait complete. 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.538+0000 7f2bfa59eb80 1 -- start start 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 Processor -- start 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 0x55f97c9fac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c942e30 con 0x55f97c9fa890 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c8df710 con 0x55f97c9ef9e0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c8df3b0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.539+0000 7f2bfa59eb80 10 client.0 _ll_register_callbacks cb 0x55f97c73c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 0x55f97c9fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39050/0 (socket says 172.21.15.31:39050) 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be519c700 1 -- 172.21.15.31:0/3499409256 learned_addr learned my addr 172.21.15.31:0/3499409256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 -- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef9e0 msgr2=0x55f97c9fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 -- 172.21.15.31:0/3499409256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 msgr2=0x55f97c9fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 -- 172.21.15.31:0/3499409256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01aa680 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2be499b700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 0x55f97c9fac70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40a2c40 tx=0x7f2bd40972e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f8e0ff93173767 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.540+0000 7f2bbffff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd409ae70 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2bd40051d0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f97c9e2f10 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd4005390 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 10 client.25978 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f97c756c60 con 0x55f97c9ef0c0 2023-10-06T21:23:01.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f97c9e2f10 con 0x55f97c9ef0c0 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2bd40a2ce0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 10 client.25978 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 10 client.25978 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 20 client.25978 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca09a20 0x55f97ca0be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 4 client.25978 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.541+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f97c93be10 con 0x55f97ca09a20 2023-10-06T21:23:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.542+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca09a20 0x55f97ca0be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.543+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca09a20 0x55f97ca0be10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3456648bd66d6d74 server_cookie=62183323bd2b9fa0 in_seq=0 out_seq=0 2023-10-06T21:23:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.543+0000 7f2bbffff700 10 client.25978 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.543+0000 7f2bbffff700 1 -- 172.21.15.31:0/3499409256 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f97c93be10 con 0x55f97ca09a20 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.543+0000 7f2bbffff700 10 client.25978 handle_command_reply: tid=0 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 client.25978 shutdown 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 msgr2=0x55f97c9fac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 0x55f97c9fac70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40a2c40 tx=0x7f2bd40972e0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 shutdown_connections 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca09a20 0x55f97ca0be10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9fa890 0x55f97c9fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef0c0 0x55f97c9fac70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3499409256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bbffff700 0 client.25978 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:23:01.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 >> 172.21.15.31:0/3499409256 conn(0x55f97c964260 msgr2=0x55f97c9eafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 shutdown_connections 2023-10-06T21:23:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3499409256 wait complete. 2023-10-06T21:23:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.544+0000 7f2bfa59eb80 20 client.25978 trim_cache size 0 max 16384 2023-10-06T21:23:01.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.558+0000 7f2bfa59eb80 1 Processor -- start 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.558+0000 7f2bfa59eb80 1 -- start start 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97ca06260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 0x55f97c9f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c7c3920 con 0x55f97c9ef9e0 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c9e7a80 con 0x55f97c9ef0c0 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c9ea240 con 0x55f97c9fa890 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39064/0 (socket says 172.21.15.31:39064) 2023-10-06T21:23:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 -- 172.21.15.31:0/3222272608 learned_addr learned my addr 172.21.15.31:0/3222272608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2be499b700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97ca06260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2be519c700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 0x55f97c9f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 -- 172.21.15.31:0/3222272608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 msgr2=0x55f97c9f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 0x55f97c9f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 -- 172.21.15.31:0/3222272608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 msgr2=0x55f97ca06260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.559+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97ca06260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2bdffff700 1 -- 172.21.15.31:0/3222272608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97c9fa300 con 0x55f97c9ef9e0 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2be519c700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 0x55f97c9f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:23:01.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2bdffff700 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae96b155947b138 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2bcb7fe700 1 -- 172.21.15.31:0/3222272608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc017040 con 0x55f97c9ef9e0 2023-10-06T21:23:01.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2bcb7fe700 1 -- 172.21.15.31:0/3222272608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bcc023070 con 0x55f97c9ef9e0 2023-10-06T21:23:01.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.560+0000 7f2bcb7fe700 1 -- 172.21.15.31:0/3222272608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005280 con 0x55f97c9ef9e0 2023-10-06T21:23:01.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3222272608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 msgr2=0x55f97ca05410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc009310 comp rx=0 tx=0).stop 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3222272608 shutdown_connections 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9fa890 0x55f97c9f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9ef0c0 0x55f97ca06260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/3222272608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef9e0 0x55f97ca05410 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3222272608 >> 172.21.15.31:0/3222272608 conn(0x55f97c964260 msgr2=0x55f97c9fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3222272608 shutdown_connections 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/3222272608 wait complete. 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.561+0000 7f2bfa59eb80 1 -- start start 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 Processor -- start 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9fa890 0x55f97c9fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c9fe7d0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c9fe940 con 0x55f97c9fa890 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c9feb00 con 0x55f97c9ef9e0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.562+0000 7f2bfa59eb80 10 client.0 _ll_register_callbacks cb 0x7f2bcc009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39066/0 (socket says 172.21.15.31:39066) 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 -- 172.21.15.31:0/765496882 learned_addr learned my addr 172.21.15.31:0/765496882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2bdffff700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be519c700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9fa890 0x55f97c9fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 -- 172.21.15.31:0/765496882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 msgr2=0x55f97c9fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 -- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9fa890 msgr2=0x55f97c9fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9fa890 0x55f97c9fe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 -- 172.21.15.31:0/765496882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc009c50 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2be499b700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4097370 tx=0x7f2bd409aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81ba770beafb1a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2bbffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.563+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd409b800 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2bd409ae70 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd409efc0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd40053d0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 10 client.16293 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f97c794250 con 0x55f97c9ef0c0 2023-10-06T21:23:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bd409efc0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2bd4099dd0 con 0x55f97c9ef0c0 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 10 client.16293 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 10 client.16293 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 20 client.16293 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca5e5f0 0x55f97ca609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 4 client.16293 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:23:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.564+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 --> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f97c93bb30 con 0x55f97ca5e5f0 2023-10-06T21:23:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bdffff700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca5e5f0 0x55f97ca609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bdffff700 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca5e5f0 0x55f97ca609e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db324fad1b5441fc server_cookie=329cca2cdcc935e in_seq=0 out_seq=0 2023-10-06T21:23:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bbffff700 10 client.16293 ms_handle_connect on v2:172.21.15.142:6832/1989266338 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bbffff700 1 -- 172.21.15.31:0/765496882 <== mds.4 v2:172.21.15.142:6832/1989266338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f97c93bb30 con 0x55f97ca5e5f0 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bbffff700 10 client.16293 handle_command_reply: tid=0 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.565+0000 7f2bfa59eb80 1 client.16293 shutdown 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 msgr2=0x55f97c9ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4097370 tx=0x7f2bd409aab0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 shutdown_connections 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:6832/1989266338,v1:172.21.15.142:6833/1989266338] conn(0x55f97ca5e5f0 0x55f97ca609e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f97c9fa890 0x55f97c9fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f97c9ef0c0 0x55f97c9ffa20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 --2- 172.21.15.31:0/765496882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c9ef9e0 0x55f97c9fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 >> 172.21.15.31:0/765496882 conn(0x55f97c964260 msgr2=0x55f97c9fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bbffff700 0 client.16293 ms_handle_reset on v2:172.21.15.142:6832/1989266338 2023-10-06T21:23:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 shutdown_connections 2023-10-06T21:23:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 1 -- 172.21.15.31:0/765496882 wait complete. 2023-10-06T21:23:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.566+0000 7f2bfa59eb80 20 client.16293 trim_cache size 0 max 16384 2023-10-06T21:23:01.574 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 msgr2=0x7f2bd00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 0x7f2bd00a3ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8007610 tx=0x7f2bd80079d0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 msgr2=0x7f2be010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009810 tx=0x7f2bd400fa00 comp rx=0 tx=0).stop 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 shutdown_connections 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be01a1f00 0x7f2be01a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2bd00a15f0 0x7f2bd00a3ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be01010e0 0x7f2be010cdc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 --2- 172.21.15.31:0/4072255172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2be0106200 0x7f2be010d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 >> 172.21.15.31:0/4072255172 conn(0x7f2be00fc960 msgr2=0x7f2be0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 shutdown_connections 2023-10-06T21:23:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.567+0000 7f2be6bff700 1 -- 172.21.15.31:0/4072255172 wait complete. 2023-10-06T21:23:01.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 -- 172.21.15.31:0/3466636857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c8101a60 msgr2=0x7f12c8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 --2- 172.21.15.31:0/3466636857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c8101a60 0x7f12c8105aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009990 tx=0x7f12b8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 -- 172.21.15.31:0/3466636857 shutdown_connections 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 --2- 172.21.15.31:0/3466636857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c81061d0 0x7f12c810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 --2- 172.21.15.31:0/3466636857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81010b0 0x7f12c8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 --2- 172.21.15.31:0/3466636857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c8101a60 0x7f12c8105aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.854+0000 7f12d07ff700 1 -- 172.21.15.31:0/3466636857 >> 172.21.15.31:0/3466636857 conn(0x7f12c80fc950 msgr2=0x7f12c80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.855+0000 7f12d07ff700 1 -- 172.21.15.31:0/3466636857 shutdown_connections 2023-10-06T21:23:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.855+0000 7f12d07ff700 1 -- 172.21.15.31:0/3466636857 wait complete. 2023-10-06T21:23:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.856+0000 7f12d07ff700 1 Processor -- start 2023-10-06T21:23:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.856+0000 7f12d07ff700 1 -- start start 2023-10-06T21:23:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 0x7f12c81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c819b830 0x7f12c819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c819c2b0 con 0x7f12c81061d0 2023-10-06T21:23:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c819c420 con 0x7f12c81010b0 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12d07ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c819c5e0 con 0x7f12c819b830 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12cdd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 0x7f12c81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40128/0 (socket says 172.21.15.31:40128) 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12ce59b700 1 -- 172.21.15.31:0/428204941 learned_addr learned my addr 172.21.15.31:0/428204941 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.857+0000 7f12ced9c700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c819b830 0x7f12c819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 -- 172.21.15.31:0/428204941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c819b830 msgr2=0x7f12c819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c819b830 0x7f12c819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 -- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 msgr2=0x7f12c81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 0x7f12c81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 -- 172.21.15.31:0/428204941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009520 con 0x7f12c81010b0 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 0x7f12c81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12ce59b700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12c400eb30 tx=0x7f12c400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12bf7fe700 1 -- 172.21.15.31:0/428204941 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4009880 con 0x7f12c81010b0 2023-10-06T21:23:01.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12bf7fe700 1 -- 172.21.15.31:0/428204941 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c40044b0 con 0x7f12c81010b0 2023-10-06T21:23:01.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12bf7fe700 1 -- 172.21.15.31:0/428204941 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4003ce0 con 0x7f12c81010b0 2023-10-06T21:23:01.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.858+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b80095f0 con 0x7f12c81010b0 2023-10-06T21:23:01.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.859+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c8073440 con 0x7f12c81010b0 2023-10-06T21:23:01.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.862+0000 7f12bf7fe700 1 -- 172.21.15.31:0/428204941 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f12c4003e40 con 0x7f12c81010b0 2023-10-06T21:23:01.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.863+0000 7f12e4096b80 1 Processor -- start 2023-10-06T21:23:01.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 -- start start 2023-10-06T21:23:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12bf7fe700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 0x7f12b40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a2a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a097d460 con 0x5580a0a1d1f0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a096ec20 con 0x5580a0a280a0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12e4096b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a096c540 con 0x5580a0a1c8d0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12bf7fe700 1 -- 172.21.15.31:0/428204941 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f12c4014070 con 0x7f12c81010b0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12ced9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a2a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39110/0 (socket says 172.21.15.31:39110) 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.864+0000 7f12ce59b700 1 -- 172.21.15.31:0/2279866666 learned_addr learned my addr 172.21.15.31:0/2279866666 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ced9c700 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 0x7f12b40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 -- 172.21.15.31:0/2279866666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 msgr2=0x5580a0a1ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 -- 172.21.15.31:0/2279866666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 msgr2=0x5580a0a2a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a2a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 -- 172.21.15.31:0/2279866666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a0792de0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12ce59b700 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f12c4097040 tx=0x7f12c40cb2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246765eb6f0ba137 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 0x7f12b40a3ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b800b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12bcff9700 1 -- 172.21.15.31:0/2279866666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400cdd0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.865+0000 7f12bcff9700 1 -- 172.21.15.31:0/2279866666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c409acd0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12bcff9700 1 -- 172.21.15.31:0/2279866666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40a2960 con 0x5580a0a1d1f0 2023-10-06T21:23:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 -- 172.21.15.31:0/2279866666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 msgr2=0x5580a0a27b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f12c4097040 tx=0x7f12c40cb2a0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 -- 172.21.15.31:0/2279866666 shutdown_connections 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a2a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 --2- 172.21.15.31:0/2279866666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a27b60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 -- 172.21.15.31:0/2279866666 >> 172.21.15.31:0/2279866666 conn(0x5580a0989260 msgr2=0x5580a0a10450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 -- 172.21.15.31:0/2279866666 shutdown_connections 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.866+0000 7f12e4096b80 1 -- 172.21.15.31:0/2279866666 wait complete. 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.867+0000 7f12e4096b80 1 -- start start 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.867+0000 7f12e4096b80 1 Processor -- start 2023-10-06T21:23:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1d1f0 0x5580a0a253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0967e30 con 0x5580a0a280a0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0904710 con 0x5580a0a1c8d0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a09043b0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.868+0000 7f12e4096b80 10 client.0 _ll_register_callbacks cb 0x5580a0761800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ce59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40158/0 (socket says 172.21.15.31:40158) 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ce59b700 1 -- 172.21.15.31:0/410200036 learned_addr learned my addr 172.21.15.31:0/410200036 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1d1f0 0x5580a0a253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 -- 172.21.15.31:0/410200036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1d1f0 msgr2=0x5580a0a253e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1d1f0 0x5580a0a253e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 -- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 msgr2=0x5580a0a24ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 -- 172.21.15.31:0/410200036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c8073440 con 0x5580a0a280a0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ce59b700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12ced9c700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007ba0 tx=0x7f12c0007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78a940db7d563a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.869+0000 7f12b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c001e070 con 0x5580a0a280a0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580a0a07f10 con 0x5580a0a280a0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f12c0004bb0 con 0x5580a0a280a0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12c000e040 con 0x5580a0a280a0 2023-10-06T21:23:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12e4096b80 10 client.16305 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580a077bc60 con 0x5580a0a280a0 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5580a0a07f10 con 0x5580a0a280a0 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f12c0021020 con 0x5580a0a280a0 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12e4096b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.870+0000 7f12e4096b80 10 client.16305 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.871+0000 7f12e4096b80 20 client.16305 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.871+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a37d20 0x5580a0a3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.871+0000 7f12e4096b80 4 client.16305 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.871+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580a0960e10 con 0x5580a0a37d20 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.871+0000 7f12ce59b700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a37d20 0x5580a0a3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.872+0000 7f12ce59b700 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a37d20 0x5580a0a3a140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4716971f1bdddb7a server_cookie=900a6ca0f89c28d5 in_seq=0 out_seq=0 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.872+0000 7f12b1ffb700 10 client.16305 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.872+0000 7f12b1ffb700 1 -- 172.21.15.31:0/410200036 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580a0960e10 con 0x5580a0a37d20 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.872+0000 7f12b1ffb700 10 client.16305 handle_command_reply: tid=0 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.872+0000 7f12e4096b80 1 client.16305 shutdown 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 msgr2=0x5580a0a25920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007ba0 tx=0x7f12c0007eb0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 shutdown_connections 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25920 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a37d20 0x5580a0a3a140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1c8d0 0x5580a0a24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 --2- 172.21.15.31:0/410200036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1d1f0 0x5580a0a253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12b1ffb700 0 client.16305 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 >> 172.21.15.31:0/410200036 conn(0x5580a0989260 msgr2=0x5580a0a0ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 shutdown_connections 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 1 -- 172.21.15.31:0/410200036 wait complete. 2023-10-06T21:23:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.873+0000 7f12e4096b80 20 client.16305 trim_cache size 0 max 16384 2023-10-06T21:23:01.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 Processor -- start 2023-10-06T21:23:01.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 -- start start 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1d1f0 0x5580a0a24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a07e8920 con 0x5580a0a280a0 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0a1db50 con 0x5580a0a1d1f0 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.889+0000 7f12e4096b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0a07de0 con 0x5580a0a1c8d0 2023-10-06T21:23:01.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39136/0 (socket says 172.21.15.31:39136) 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 -- 172.21.15.31:0/3887248219 learned_addr learned my addr 172.21.15.31:0/3887248219 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ce59b700 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1d1f0 0x5580a0a24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 -- 172.21.15.31:0/3887248219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 msgr2=0x5580a0a1f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 -- 172.21.15.31:0/3887248219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1d1f0 msgr2=0x5580a0a24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1d1f0 0x5580a0a24680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 -- 172.21.15.31:0/3887248219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a0a0c140 con 0x5580a0a280a0 2023-10-06T21:23:01.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.890+0000 7f12ced9c700 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009010 tx=0x7f12c0009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ffb81f2f25c952 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.891+0000 7f12bd7fa700 1 -- 172.21.15.31:0/3887248219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c001e070 con 0x5580a0a280a0 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.891+0000 7f12bd7fa700 1 -- 172.21.15.31:0/3887248219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c000f040 con 0x5580a0a280a0 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.891+0000 7f12bd7fa700 1 -- 172.21.15.31:0/3887248219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000e040 con 0x5580a0a280a0 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.891+0000 7f12e4096b80 1 -- 172.21.15.31:0/3887248219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 msgr2=0x5580a0a25570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.891+0000 7f12e4096b80 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009010 tx=0x7f12c0009410 comp rx=0 tx=0).stop 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 -- 172.21.15.31:0/3887248219 shutdown_connections 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a280a0 0x5580a0a25570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a1f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 --2- 172.21.15.31:0/3887248219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a1d1f0 0x5580a0a24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 -- 172.21.15.31:0/3887248219 >> 172.21.15.31:0/3887248219 conn(0x5580a0989260 msgr2=0x5580a0a24e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 -- 172.21.15.31:0/3887248219 shutdown_connections 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 -- 172.21.15.31:0/3887248219 wait complete. 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 -- start start 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.892+0000 7f12e4096b80 1 Processor -- start 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a2c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a8ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0a2d550 con 0x5580a0a1d1f0 2023-10-06T21:23:01.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0a8b050 con 0x5580a0a280a0 2023-10-06T21:23:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a0a8b210 con 0x5580a0a1c8d0 2023-10-06T21:23:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12e4096b80 10 client.0 _ll_register_callbacks cb 0x7f12c00075b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12cdd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12cdd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39152/0 (socket says 172.21.15.31:39152) 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.893+0000 7f12cdd9a700 1 -- 172.21.15.31:0/1094931214 learned_addr learned my addr 172.21.15.31:0/1094931214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12ce59b700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a2c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12ced9c700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a8ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 -- 172.21.15.31:0/1094931214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 msgr2=0x5580a0a2c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a2c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 -- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 msgr2=0x5580a0a8ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a8ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 -- 172.21.15.31:0/1094931214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580a077bc60 con 0x5580a0a1d1f0 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12cdd9a700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009990 tx=0x7f12b8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938e882f16e5cd7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.894+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b802b070 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12c409acd0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f12b8005440 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b8024770 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 10 client.16311 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580a07b9250 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c409acd0 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f12b8004a40 con 0x5580a0a1d1f0 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 10 client.16311 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 20 client.16311 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a8c9f0 0x5580a0a8ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 4 client.16311 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.895+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 --> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5580a0960b30 con 0x5580a0a8c9f0 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12ce59b700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a8c9f0 0x5580a0a8ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12ce59b700 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a8c9f0 0x5580a0a8ee10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a076ed363dc5e565 server_cookie=a042a0516b4e7bfd in_seq=0 out_seq=0 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12b1ffb700 10 client.16311 ms_handle_connect on v2:172.21.15.142:6834/4077995395 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12b1ffb700 1 -- 172.21.15.31:0/1094931214 <== mds.3 v2:172.21.15.142:6834/4077995395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5580a0960b30 con 0x5580a0a8c9f0 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12b1ffb700 10 client.16311 handle_command_reply: tid=0 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.896+0000 7f12e4096b80 1 client.16311 shutdown 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 msgr2=0x5580a0a2ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009990 tx=0x7f12b8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 shutdown_connections 2023-10-06T21:23:01.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:6834/4077995395,v1:172.21.15.142:6835/4077995395] conn(0x5580a0a8c9f0 0x5580a0a8ee10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a0a280a0 0x5580a0a8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580a0a1c8d0 0x5580a0a2c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12b1ffb700 0 client.16311 ms_handle_reset on v2:172.21.15.142:6834/4077995395 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 --2- 172.21.15.31:0/1094931214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5580a0a1d1f0 0x5580a0a2ce00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 >> 172.21.15.31:0/1094931214 conn(0x5580a0989260 msgr2=0x5580a0a25680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 shutdown_connections 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 1 -- 172.21.15.31:0/1094931214 wait complete. 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.897+0000 7f12e4096b80 20 client.16311 trim_cache size 0 max 16384 2023-10-06T21:23:01.904 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:23:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 msgr2=0x7f12b40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 0x7f12b40a3ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b800b260 comp rx=0 tx=0).stop 2023-10-06T21:23:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 msgr2=0x7f12c8196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12c400eb30 tx=0x7f12c400ee40 comp rx=0 tx=0).stop 2023-10-06T21:23:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 shutdown_connections 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12c819b830 0x7f12c819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c81010b0 0x7f12c8196e90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f12b40a15f0 0x7f12b40a3ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 --2- 172.21.15.31:0/428204941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12c81061d0 0x7f12c81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 >> 172.21.15.31:0/428204941 conn(0x7f12c80fc950 msgr2=0x7f12c80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 shutdown_connections 2023-10-06T21:23:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:01.898+0000 7f12d07ff700 1 -- 172.21.15.31:0/428204941 wait complete. 2023-10-06T21:23:01.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.138+0000 7fe9e61ff700 1 -- 172.21.15.31:0/4096445951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 msgr2=0x7fe9e0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.138+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/4096445951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0101490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009990 tx=0x7fe9d0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 -- 172.21.15.31:0/4096445951 shutdown_connections 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/4096445951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 0x7fe9e010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/4096445951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0101490 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/4096445951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e0101a60 0x7fe9e0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 -- 172.21.15.31:0/4096445951 >> 172.21.15.31:0/4096445951 conn(0x7fe9e00fc930 msgr2=0x7fe9e00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 -- 172.21.15.31:0/4096445951 shutdown_connections 2023-10-06T21:23:02.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.139+0000 7fe9e61ff700 1 -- 172.21.15.31:0/4096445951 wait complete. 2023-10-06T21:23:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 Processor -- start 2023-10-06T21:23:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 -- start start 2023-10-06T21:23:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 0x7fe9e01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e019db30 0x7fe9e019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e019e5b0 con 0x7fe9e01010b0 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e019e720 con 0x7fe9e019db30 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9e61ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e019e8e0 con 0x7fe9e01061d0 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.140+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39178/0 (socket says 172.21.15.31:39178) 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3420147813 learned_addr learned my addr 172.21.15.31:0/3420147813 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 0x7fe9e01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e019db30 0x7fe9e019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3420147813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 msgr2=0x7fe9e01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 0x7fe9e01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3420147813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e019db30 msgr2=0x7fe9e019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e019db30 0x7fe9e019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3420147813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00095f0 con 0x7fe9e01010b0 2023-10-06T21:23:02.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.141+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00036c0 tx=0x7fe9d0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.142+0000 7fe9dcff9700 1 -- 172.21.15.31:0/3420147813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001d070 con 0x7fe9e01010b0 2023-10-06T21:23:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.142+0000 7fe9dcff9700 1 -- 172.21.15.31:0/3420147813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9d0003ec0 con 0x7fe9e01010b0 2023-10-06T21:23:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.142+0000 7fe9dcff9700 1 -- 172.21.15.31:0/3420147813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0022ac0 con 0x7fe9e01010b0 2023-10-06T21:23:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.142+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e01a5de0 con 0x7fe9e01010b0 2023-10-06T21:23:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.142+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01a61b0 con 0x7fe9e01010b0 2023-10-06T21:23:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.146+0000 7fe9dcff9700 1 -- 172.21.15.31:0/3420147813 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe9d0004030 con 0x7fe9e01010b0 2023-10-06T21:23:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.147+0000 7fe9f9b6db80 1 Processor -- start 2023-10-06T21:23:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.147+0000 7fe9f9b6db80 1 -- start start 2023-10-06T21:23:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64b5890 0x557ea64b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6413460 con 0x557ea64aa0c0 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6404c20 con 0x557ea64aa9e0 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6402540 con 0x557ea64b5890 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9dcff9700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 0x7fe9c80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40208/0 (socket says 172.21.15.31:40208) 2023-10-06T21:23:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.148+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3555630966 learned_addr learned my addr 172.21.15.31:0/3555630966 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9dcff9700 1 -- 172.21.15.31:0/3420147813 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fe9d00ce720 con 0x7fe9e01010b0 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 0x7fe9c80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9deffd700 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64b5890 0x557ea64b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3555630966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64b5890 msgr2=0x557ea64b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64b5890 0x557ea64b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3555630966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 msgr2=0x557ea64aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 -- 172.21.15.31:0/3555630966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea6228de0 con 0x557ea64aa9e0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.149+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004300 tx=0x7fe9d0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec8ef449887d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 0x7fe9c80a3b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80041e0 tx=0x7fe9d8004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9c5ffb700 1 -- 172.21.15.31:0/3555630966 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d009bf00 con 0x557ea64aa9e0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9c5ffb700 1 -- 172.21.15.31:0/3555630966 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9d009b800 con 0x557ea64aa9e0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9c5ffb700 1 -- 172.21.15.31:0/3555630966 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d009af50 con 0x557ea64aa9e0 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3555630966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 msgr2=0x557ea64b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.150+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004300 tx=0x7fe9d0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3555630966 shutdown_connections 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64b5890 0x557ea64b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3555630966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b5350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3555630966 >> 172.21.15.31:0/3555630966 conn(0x557ea641f260 msgr2=0x557ea64a6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3555630966 shutdown_connections 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3555630966 wait complete. 2023-10-06T21:23:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.151+0000 7fe9f9b6db80 1 -- start start 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 Processor -- start 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa9e0 0x557ea64b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea63fde30 con 0x557ea64aa0c0 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea639a710 con 0x557ea64b5890 2023-10-06T21:23:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea639a3b0 con 0x557ea64aa9e0 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9f9b6db80 10 client.0 _ll_register_callbacks cb 0x557ea61f7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39194/0 (socket says 172.21.15.31:39194) 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.152+0000 7fe9df7fe700 1 -- 172.21.15.31:0/4041378774 learned_addr learned my addr 172.21.15.31:0/4041378774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9deffd700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa9e0 0x557ea64b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9dffff700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 -- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa9e0 msgr2=0x557ea64b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa9e0 0x557ea64b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 -- 172.21.15.31:0/4041378774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 msgr2=0x557ea64b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 -- 172.21.15.31:0/4041378774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d8011040 con 0x557ea64aa0c0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9d009b790 tx=0x7fe9d00cbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251b5476361f7578 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9beffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d009bf00 con 0x557ea64aa0c0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fe9d009f950 con 0x557ea64aa0c0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.153+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ea649df10 con 0x557ea64aa0c0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d00a3b00 con 0x557ea64aa0c0 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 10 client.16323 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe9e01a61b0 con 0x557ea64aa0c0 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ea649df10 con 0x557ea64aa0c0 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe9d00975f0 con 0x557ea64aa0c0 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 20 client.16323 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea64c4a00 0x557ea64c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 4 client.16323 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.154+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ea63f6e10 con 0x557ea64c4a00 2023-10-06T21:23:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.155+0000 7fe9deffd700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea64c4a00 0x557ea64c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.155+0000 7fe9deffd700 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea64c4a00 0x557ea64c6df0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3960d9752578bc2 server_cookie=c6311dc0ec6fb893 in_seq=0 out_seq=0 2023-10-06T21:23:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.155+0000 7fe9beffd700 10 client.16323 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9beffd700 1 -- 172.21.15.31:0/4041378774 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ea63f6e10 con 0x557ea64c4a00 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9beffd700 10 client.16323 handle_command_reply: tid=0 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 client.16323 shutdown 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 msgr2=0x557ea64b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9d009b790 tx=0x7fe9d00cbdb0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 shutdown_connections 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea64c4a00 0x557ea64c6df0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa0c0 0x557ea64b5c70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9beffd700 0 client.16323 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/4041378774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa9e0 0x557ea64b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 >> 172.21.15.31:0/4041378774 conn(0x557ea641f260 msgr2=0x557ea64a6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 shutdown_connections 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/4041378774 wait complete. 2023-10-06T21:23:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.157+0000 7fe9f9b6db80 20 client.16323 trim_cache size 0 max 16384 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.174+0000 7fe9f9b6db80 1 Processor -- start 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 -- start start 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea627e920 con 0x557ea64aa9e0 2023-10-06T21:23:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea64b4b70 con 0x557ea64b5890 2023-10-06T21:23:02.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea64c7470 con 0x557ea64aa0c0 2023-10-06T21:23:02.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39198/0 (socket says 172.21.15.31:39198) 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9deffd700 1 -- 172.21.15.31:0/657899963 learned_addr learned my addr 172.21.15.31:0/657899963 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9dffff700 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.175+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 -- 172.21.15.31:0/657899963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 msgr2=0x557ea64a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 -- 172.21.15.31:0/657899963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 msgr2=0x557ea64c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64c11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 -- 172.21.15.31:0/657899963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea64a2140 con 0x557ea64aa9e0 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9deffd700 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9e019a7a0 tx=0x7fe9d40073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214cc62f699bfc0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9c67fc700 1 -- 172.21.15.31:0/657899963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4016070 con 0x557ea64aa9e0 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9c67fc700 1 -- 172.21.15.31:0/657899963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9d4017040 con 0x557ea64aa9e0 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.176+0000 7fe9c67fc700 1 -- 172.21.15.31:0/657899963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4011c40 con 0x557ea64aa9e0 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/657899963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 msgr2=0x557ea64af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9e019a7a0 tx=0x7fe9d40073d0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/657899963 shutdown_connections 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64b5890 0x557ea64c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/657899963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64aa9e0 0x557ea64af160 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/657899963 >> 172.21.15.31:0/657899963 conn(0x557ea641f260 msgr2=0x557ea64b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/657899963 shutdown_connections 2023-10-06T21:23:02.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.177+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/657899963 wait complete. 2023-10-06T21:23:02.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.178+0000 7fe9f9b6db80 1 -- start start 2023-10-06T21:23:02.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.178+0000 7fe9f9b6db80 1 Processor -- start 2023-10-06T21:23:02.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.178+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.178+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9f9b6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6517d40 con 0x557ea64b5890 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6517e80 con 0x557ea64aa9e0 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9f9b6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea6517ff0 con 0x557ea64aa0c0 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9f9b6db80 10 client.0 _ll_register_callbacks cb 0x7fe9d8003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9df7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39212/0 (socket says 172.21.15.31:39212) 2023-10-06T21:23:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 -- 172.21.15.31:0/3681361521 learned_addr learned my addr 172.21.15.31:0/3681361521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9deffd700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 -- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 msgr2=0x557ea64b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 -- 172.21.15.31:0/3681361521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 msgr2=0x557ea64b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 -- 172.21.15.31:0/3681361521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d4007bb0 con 0x557ea64b5890 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.179+0000 7fe9dffff700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8004930 tx=0x7fe9d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147283398240b1bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9beffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8020040 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9d009b800 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fe9d8007e00 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d801b450 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9f9b6db80 10 client.16329 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ea6211c60 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.180+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9d009b800 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe9d8027750 con 0x557ea64b5890 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 10 client.16329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 20 client.16329 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea65197a0 0x557ea651bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 4 client.16329 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 --> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ea63f6b30 con 0x557ea65197a0 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.181+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea65197a0 0x557ea651bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.182+0000 7fe9df7fe700 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea65197a0 0x557ea651bbc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9d5d1e5e23ace6b server_cookie=5dee810332f77c76 in_seq=0 out_seq=0 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.182+0000 7fe9beffd700 10 client.16329 ms_handle_connect on v2:172.21.15.146:6834/3172848071 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9beffd700 1 -- 172.21.15.31:0/3681361521 <== mds.2 v2:172.21.15.146:6834/3172848071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6135 (crc 0 0 0) 0x557ea63f6b30 con 0x557ea65197a0 2023-10-06T21:23:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9beffd700 10 client.16329 handle_command_reply: tid=0 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 client.16329 shutdown 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 msgr2=0x557ea6517800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8004930 tx=0x7fe9d8004ad0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 shutdown_connections 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:6834/3172848071,v1:172.21.15.146:6835/3172848071] conn(0x557ea65197a0 0x557ea651bbc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ea64b5890 0x557ea6517800 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ea64aa0c0 0x557ea64b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 --2- 172.21.15.31:0/3681361521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ea64aa9e0 0x557ea64b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 >> 172.21.15.31:0/3681361521 conn(0x557ea641f260 msgr2=0x557ea64adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9beffd700 0 client.16329 ms_handle_reset on v2:172.21.15.146:6834/3172848071 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 shutdown_connections 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 1 -- 172.21.15.31:0/3681361521 wait complete. 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.183+0000 7fe9f9b6db80 20 client.16329 trim_cache size 0 max 16384 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 475420284, 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628320, 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 768616312, 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628299, 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2023-10-06T21:23:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 784891703, 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628298, 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1078894879, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628326, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1320097941, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628311, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1362107070, 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628313, 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1396320325, 2023-10-06T21:23:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628314, 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1724282706, 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628351, 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1735194928, 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628322, 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 1829424146, 2023-10-06T21:23:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628354, 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2057071126, 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628323, 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2240972172, 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628327, 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2359548126, 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628324, 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2023-10-06T21:23:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2418594655, 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628302, 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2548908049, 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628353, 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2666914736, 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628328, 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2023-10-06T21:23:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 2863683827, 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628310, 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3055924542, 2023-10-06T21:23:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628317, 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3080208503, 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628318, 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3216720589, 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628316, 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3305505859, 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628319, 2023-10-06T21:23:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3386454148, 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628315, 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3661709194, 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628301, 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3742581999, 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628297, 2023-10-06T21:23:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 3890576528, 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628355, 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:23:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "damage_type": "backtrace", 2023-10-06T21:23:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4042609579, 2023-10-06T21:23:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1099511628312, 2023-10-06T21:23:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2023-10-06T21:23:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:23:02.209 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 msgr2=0x7fe9c80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 0x7fe9c80a3b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80041e0 tx=0x7fe9d8004b40 comp rx=0 tx=0).stop 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 msgr2=0x7fe9e0199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00036c0 tx=0x7fe9d0005740 comp rx=0 tx=0).stop 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 shutdown_connections 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9e019db30 0x7fe9e019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9c80a1670 0x7fe9c80a3b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e01010b0 0x7fe9e0199190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 --2- 172.21.15.31:0/3420147813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9e01061d0 0x7fe9e01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 >> 172.21.15.31:0/3420147813 conn(0x7fe9e00fc930 msgr2=0x7fe9e00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 shutdown_connections 2023-10-06T21:23:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.184+0000 7fe9e61ff700 1 -- 172.21.15.31:0/3420147813 wait complete. 2023-10-06T21:23:02.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:02 smithi031 ceph-mon[98539]: pgmap v65: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 701 KiB/s wr, 85 op/s 2023-10-06T21:23:02.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/160304773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:23:02.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:23:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:02 smithi146 ceph-mon[102809]: pgmap v65: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 701 KiB/s wr, 85 op/s 2023-10-06T21:23:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:02 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/160304773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:23:02.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 17973 2.24 MB/sec warmup 52 sec latency 16.598 ms 2023-10-06T21:23:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:02 smithi142 ceph-mon[102951]: pgmap v65: 97 pgs: 97 active+clean; 30 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 701 KiB/s wr, 85 op/s 2023-10-06T21:23:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:02 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/160304773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:23:02.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 -- 172.21.15.31:0/2894020781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc101ff0 msgr2=0x7f6bbc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/2894020781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc1023d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009990 tx=0x7f6bb0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 -- 172.21.15.31:0/2894020781 shutdown_connections 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/2894020781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc10b410 0x7f6bbc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/2894020781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc1023d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/2894020781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc102910 0x7f6bbc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.429+0000 7f6bc2dff700 1 -- 172.21.15.31:0/2894020781 >> 172.21.15.31:0/2894020781 conn(0x7f6bbc0fb830 msgr2=0x7f6bbc0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.430+0000 7f6bc2dff700 1 -- 172.21.15.31:0/2894020781 shutdown_connections 2023-10-06T21:23:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.430+0000 7f6bc2dff700 1 -- 172.21.15.31:0/2894020781 wait complete. 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 Processor -- start 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 -- start start 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1009b0 con 0x7f6bbc10b410 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1a55e0 con 0x7f6bbc101ff0 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.431+0000 7f6bc2dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1a5720 con 0x7f6bbc102910 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39226/0 (socket says 172.21.15.31:39226) 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48632/0 (socket says 172.21.15.31:48632) 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bc0b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bc139c700 1 -- 172.21.15.31:0/762288309 learned_addr learned my addr 172.21.15.31:0/762288309 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 -- 172.21.15.31:0/762288309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc101ff0 msgr2=0x7f6bbc0ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc0ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 -- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 msgr2=0x7f6bbc1a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 -- 172.21.15.31:0/762288309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00095f0 con 0x7f6bbc102910 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bc139c700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.432+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9f0 tx=0x7f6ba800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.433+0000 7f6bb9ffb700 1 -- 172.21.15.31:0/762288309 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800c9c0 con 0x7f6bbc102910 2023-10-06T21:23:02.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.433+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc1a5980 con 0x7f6bbc102910 2023-10-06T21:23:02.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.433+0000 7f6bb9ffb700 1 -- 172.21.15.31:0/762288309 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ba8010430 con 0x7f6bbc102910 2023-10-06T21:23:02.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.433+0000 7f6bb9ffb700 1 -- 172.21.15.31:0/762288309 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8018680 con 0x7f6bbc102910 2023-10-06T21:23:02.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.433+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc1a5e60 con 0x7f6bbc102910 2023-10-06T21:23:02.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.436+0000 7f6bb9ffb700 1 -- 172.21.15.31:0/762288309 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6ba80105a0 con 0x7f6bbc102910 2023-10-06T21:23:02.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.437+0000 7f6bd6686b80 1 Processor -- start 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.437+0000 7f6bb9ffb700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 0x7f6bac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.437+0000 7f6bd6686b80 1 -- start start 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.437+0000 7f6bb9ffb700 1 -- 172.21.15.31:0/762288309 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f6ba8014070 con 0x7f6bbc102910 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bc139c700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 0x7f6bac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 0x560e2c412350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 0x560e2c414c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c370460 con 0x560e2c4079e0 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c361c20 con 0x560e2c412890 2023-10-06T21:23:02.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c35f540 con 0x560e2c4070c0 2023-10-06T21:23:02.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bc0b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 0x560e2c412350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bc0b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48634/0 (socket says 172.21.15.31:48634) 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bc0b9b700 1 -- 172.21.15.31:0/1364779963 learned_addr learned my addr 172.21.15.31:0/1364779963 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.438+0000 7f6bc139c700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 0x560e2c414c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 -- 172.21.15.31:0/1364779963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 msgr2=0x560e2c4074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 -- 172.21.15.31:0/1364779963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 msgr2=0x560e2c414c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 0x560e2c414c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 -- 172.21.15.31:0/1364779963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2c185de0 con 0x560e2c4079e0 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bc139c700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 0x560e2c414c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bc139c700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 0x7f6bac0a3ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc101d50 tx=0x7f6bb4009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 0x560e2c412350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80ca8d0 tx=0x7f6ba8099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07e22084fe3964f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6ba6ffd700 1 -- 172.21.15.31:0/1364779963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba809a860 con 0x560e2c4079e0 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6ba6ffd700 1 -- 172.21.15.31:0/1364779963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ba809a160 con 0x560e2c4079e0 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.439+0000 7f6ba6ffd700 1 -- 172.21.15.31:0/1364779963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8099650 con 0x560e2c4079e0 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 -- 172.21.15.31:0/1364779963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 msgr2=0x560e2c412350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 0x560e2c412350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80ca8d0 tx=0x7f6ba8099dc0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 -- 172.21.15.31:0/1364779963 shutdown_connections 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c412890 0x560e2c414c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c4074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/1364779963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4079e0 0x560e2c412350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 -- 172.21.15.31:0/1364779963 >> 172.21.15.31:0/1364779963 conn(0x560e2c37c260 msgr2=0x560e2c403450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 -- 172.21.15.31:0/1364779963 shutdown_connections 2023-10-06T21:23:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.440+0000 7f6bd6686b80 1 -- 172.21.15.31:0/1364779963 wait complete. 2023-10-06T21:23:02.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.441+0000 7f6bd6686b80 1 -- start start 2023-10-06T21:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.441+0000 7f6bd6686b80 1 Processor -- start 2023-10-06T21:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c40f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c40fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c35ae30 con 0x560e2c412890 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c2f7710 con 0x560e2c4079e0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c2f73b0 con 0x560e2c4070c0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bd6686b80 10 client.0 _ll_register_callbacks cb 0x560e2c154800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c40fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39234/0 (socket says 172.21.15.31:39234) 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 -- 172.21.15.31:0/336574865 learned_addr learned my addr 172.21.15.31:0/336574865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c40f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 -- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 msgr2=0x560e2c40f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c40f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 -- 172.21.15.31:0/336574865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 msgr2=0x560e2c40fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c40fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bc139c700 1 -- 172.21.15.31:0/336574865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc10e590 con 0x560e2c412890 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.442+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c40fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6bc139c700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400f000 tx=0x7f6bb4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d71a40709ef842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6b9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4017440 con 0x560e2c412890 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e2c3faf10 con 0x560e2c412890 2023-10-06T21:23:02.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f6bb40175a0 con 0x560e2c412890 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb4017760 con 0x560e2c412890 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6bd6686b80 10 client.16341 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.443+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f6bbc1a5e60 con 0x560e2c412890 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560e2c3faf10 con 0x560e2c412890 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6bb4022800 con 0x560e2c412890 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 10 client.16341 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 20 client.16341 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c422500 0x560e2c424920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 4 client.16341 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:02.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e2c353e10 con 0x560e2c422500 2023-10-06T21:23:02.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.444+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c422500 0x560e2c424920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.445+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c422500 0x560e2c424920 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=988603aaaa9ff86d server_cookie=4f0f51d2314d7465 in_seq=0 out_seq=0 2023-10-06T21:23:02.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.445+0000 7f6b9bfff700 10 client.16341 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6b9bfff700 1 -- 172.21.15.31:0/336574865 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e2c353e10 con 0x560e2c422500 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6b9bfff700 10 client.16341 handle_command_reply: tid=0 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 client.16341 shutdown 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 msgr2=0x560e2c4100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400f000 tx=0x7f6bb4004b40 comp rx=0 tx=0).stop 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 shutdown_connections 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c422500 0x560e2c424920 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c4100c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c40f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/336574865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c40fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.446+0000 7f6b9bfff700 0 client.16341 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.447+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 >> 172.21.15.31:0/336574865 conn(0x560e2c37c260 msgr2=0x560e2c403040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.447+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 shutdown_connections 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.447+0000 7f6bd6686b80 1 -- 172.21.15.31:0/336574865 wait complete. 2023-10-06T21:23:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.447+0000 7f6bd6686b80 20 client.16341 trim_cache size 0 max 16384 2023-10-06T21:23:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 Processor -- start 2023-10-06T21:23:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 -- start start 2023-10-06T21:23:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 0x560e2c40a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c1db920 con 0x560e2c4070c0 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c419920 con 0x560e2c4079e0 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c40b490 con 0x560e2c412890 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.461+0000 7f6bc0b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 0x560e2c40a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40310/0 (socket says 172.21.15.31:40310) 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bc0b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39246/0 (socket says 172.21.15.31:39246) 2023-10-06T21:23:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 -- 172.21.15.31:0/3817029532 learned_addr learned my addr 172.21.15.31:0/3817029532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 -- 172.21.15.31:0/3817029532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 msgr2=0x560e2c40a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 0x560e2c40a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 -- 172.21.15.31:0/3817029532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 msgr2=0x560e2c40a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 -- 172.21.15.31:0/3817029532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2c3ff140 con 0x560e2c4079e0 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bc139c700 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 0x560e2c40a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8099e60 tx=0x7f6ba80cae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b4bf57862ac349 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.462+0000 7f6ba77fe700 1 -- 172.21.15.31:0/3817029532 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba809a860 con 0x560e2c4079e0 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6ba77fe700 1 -- 172.21.15.31:0/3817029532 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ba80a24f0 con 0x560e2c4079e0 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6ba77fe700 1 -- 172.21.15.31:0/3817029532 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80a2060 con 0x560e2c4079e0 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 -- 172.21.15.31:0/3817029532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 msgr2=0x560e2c4193e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8099e60 tx=0x7f6ba80cae30 comp rx=0 tx=0).stop 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 -- 172.21.15.31:0/3817029532 shutdown_connections 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c412890 0x560e2c40a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c4070c0 0x560e2c40a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/3817029532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c4193e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 -- 172.21.15.31:0/3817029532 >> 172.21.15.31:0/3817029532 conn(0x560e2c37c260 msgr2=0x560e2c41db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 -- 172.21.15.31:0/3817029532 shutdown_connections 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.463+0000 7f6bd6686b80 1 -- 172.21.15.31:0/3817029532 wait complete. 2023-10-06T21:23:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.464+0000 7f6bd6686b80 1 -- start start 2023-10-06T21:23:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.464+0000 7f6bd6686b80 1 Processor -- start 2023-10-06T21:23:02.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.464+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c46ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.464+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c46f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bd6686b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c418e20 con 0x560e2c412890 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c418f90 con 0x560e2c4079e0 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bd6686b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c419150 con 0x560e2c4070c0 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bd6686b80 10 client.0 _ll_register_callbacks cb 0x7f6bb400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39256/0 (socket says 172.21.15.31:39256) 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 -- 172.21.15.31:0/569366079 learned_addr learned my addr 172.21.15.31:0/569366079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c46ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bbbfff700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c46f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 -- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 msgr2=0x560e2c46ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c46ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 -- 172.21.15.31:0/569366079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 msgr2=0x560e2c46f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c46f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 -- 172.21.15.31:0/569366079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb0005380 con 0x560e2c412890 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6bc139c700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4022be0 tx=0x7f6bb4022660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b3e20cc8aec54e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.465+0000 7f6b9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4007e70 con 0x560e2c412890 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba809a160 con 0x560e2c412890 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f6bb4017440 con 0x560e2c412890 2023-10-06T21:23:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb4019a40 con 0x560e2c412890 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6bd6686b80 10 client.16347 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f6bbc1a5e60 con 0x560e2c412890 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.466+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bb4048080 con 0x560e2c412890 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6bb4042030 con 0x560e2c412890 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 10 client.16347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 20 client.16347 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c476b60 0x560e2c478f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 4 client.16347 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 --> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e2c353b30 con 0x560e2c476b60 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.467+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c476b60 0x560e2c478f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bc0b9b700 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c476b60 0x560e2c478f80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8aff515cd4727601 server_cookie=493fbcbaca014f24 in_seq=0 out_seq=0 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6b9bfff700 10 client.16347 ms_handle_connect on v2:172.21.15.146:6838/1025732473 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6b9bfff700 1 -- 172.21.15.31:0/569366079 <== mds.1 v2:172.21.15.146:6838/1025732473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e2c353b30 con 0x560e2c476b60 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6b9bfff700 10 client.16347 handle_command_reply: tid=0 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 client.16347 shutdown 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 msgr2=0x560e2c418810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4022be0 tx=0x7f6bb4022660 comp rx=0 tx=0).stop 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 shutdown_connections 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:6838/1025732473,v1:172.21.15.146:6839/1025732473] conn(0x560e2c476b60 0x560e2c478f80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e2c412890 0x560e2c418810 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e2c4070c0 0x560e2c46ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 --2- 172.21.15.31:0/569366079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2c4079e0 0x560e2c46f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 >> 172.21.15.31:0/569366079 conn(0x560e2c37c260 msgr2=0x560e2c40a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.468+0000 7f6b9bfff700 0 client.16347 ms_handle_reset on v2:172.21.15.146:6838/1025732473 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 shutdown_connections 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bd6686b80 1 -- 172.21.15.31:0/569366079 wait complete. 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bd6686b80 20 client.16347 trim_cache size 0 max 16384 2023-10-06T21:23:02.476 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 msgr2=0x7f6bac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 0x7f6bac0a3ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc101d50 tx=0x7f6bb4009920 comp rx=0 tx=0).stop 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 msgr2=0x7f6bbc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9f0 tx=0x7f6ba800ed00 comp rx=0 tx=0).stop 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 shutdown_connections 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6bac0a15f0 0x7f6bac0a3ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bbc10b410 0x7f6bbc1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bbc101ff0 0x7f6bbc0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 --2- 172.21.15.31:0/762288309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc102910 0x7f6bbc100180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.469+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 >> 172.21.15.31:0/762288309 conn(0x7f6bbc0fb830 msgr2=0x7f6bbc0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.470+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 shutdown_connections 2023-10-06T21:23:02.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.470+0000 7f6bc2dff700 1 -- 172.21.15.31:0/762288309 wait complete. 2023-10-06T21:23:02.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:23:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 -- 172.21.15.31:0/2943204393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 msgr2=0x7efe0c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 --2- 172.21.15.31:0/2943204393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1036e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009990 tx=0x7efdfc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 -- 172.21.15.31:0/2943204393 shutdown_connections 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 --2- 172.21.15.31:0/2943204393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c108420 0x7efe0c10ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 --2- 172.21.15.31:0/2943204393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1036e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 --2- 172.21.15.31:0/2943204393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c103cb0 0x7efe0c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 -- 172.21.15.31:0/2943204393 >> 172.21.15.31:0/2943204393 conn(0x7efe0c0feb40 msgr2=0x7efe0c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 -- 172.21.15.31:0/2943204393 shutdown_connections 2023-10-06T21:23:02.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.712+0000 7efe115ff700 1 -- 172.21.15.31:0/2943204393 wait complete. 2023-10-06T21:23:02.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.713+0000 7efe115ff700 1 Processor -- start 2023-10-06T21:23:02.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.713+0000 7efe115ff700 1 -- start start 2023-10-06T21:23:02.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c108420 0x7efe0c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c19db80 0x7efe0c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19e600 con 0x7efe0c103300 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19e770 con 0x7efe0c108420 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe115ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19e930 con 0x7efe0c19db80 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe0a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c108420 0x7efe0c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39276/0 (socket says 172.21.15.31:39276) 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe0b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c19db80 0x7efe0c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.714+0000 7efe0affd700 1 -- 172.21.15.31:0/268371604 learned_addr learned my addr 172.21.15.31:0/268371604 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 -- 172.21.15.31:0/268371604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c19db80 msgr2=0x7efe0c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c19db80 0x7efe0c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 -- 172.21.15.31:0/268371604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c108420 msgr2=0x7efe0c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c108420 0x7efe0c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 -- 172.21.15.31:0/268371604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0095f0 con 0x7efe0c103300 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe0affd700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005740 tx=0x7efdfc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efdf3fff700 1 -- 172.21.15.31:0/268371604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc01d070 con 0x7efe0c103300 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efdf3fff700 1 -- 172.21.15.31:0/268371604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdfc022470 con 0x7efe0c103300 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efdf3fff700 1 -- 172.21.15.31:0/268371604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc017650 con 0x7efe0c103300 2023-10-06T21:23:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c1a5e30 con 0x7efe0c103300 2023-10-06T21:23:02.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.715+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1a61a0 con 0x7efe0c103300 2023-10-06T21:23:02.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.719+0000 7efdf3fff700 1 -- 172.21.15.31:0/268371604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efdfc0225e0 con 0x7efe0c103300 2023-10-06T21:23:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.720+0000 7efdf3fff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 0x7efdf40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.720+0000 7efdf3fff700 1 -- 172.21.15.31:0/268371604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7efdfc0ce480 con 0x7efe0c103300 2023-10-06T21:23:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.720+0000 7efe24e55b80 1 Processor -- start 2023-10-06T21:23:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.720+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 0x7efdf40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 -- start start 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece500460 con 0x564ece5ab0a0 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece4f1c20 con 0x564ece59f8d0 2023-10-06T21:23:02.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe24e55b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece4ef540 con 0x564ece5a01f0 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40344/0 (socket says 172.21.15.31:40344) 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0affd700 1 -- 172.21.15.31:0/3248194592 learned_addr learned my addr 172.21.15.31:0/3248194592 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48702/0 (socket says 172.21.15.31:48702) 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.721+0000 7efe0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 0x7efdf40a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe0c19a7f0 tx=0x7efe00009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 -- 172.21.15.31:0/3248194592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 msgr2=0x564ece5aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 -- 172.21.15.31:0/3248194592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 msgr2=0x564ece5ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 -- 172.21.15.31:0/3248194592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ece315de0 con 0x564ece59f8d0 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efe0affd700 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0a3210 tx=0x7efdfc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adde54a78798ee74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efdf17fa700 1 -- 172.21.15.31:0/3248194592 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc09bb40 con 0x564ece59f8d0 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efdf17fa700 1 -- 172.21.15.31:0/3248194592 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdfc09b440 con 0x564ece59f8d0 2023-10-06T21:23:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.722+0000 7efdf17fa700 1 -- 172.21.15.31:0/3248194592 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc09f380 con 0x564ece59f8d0 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 -- 172.21.15.31:0/3248194592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 msgr2=0x564ece59fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0a3210 tx=0x7efdfc00bfd0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 -- 172.21.15.31:0/3248194592 shutdown_connections 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece59f8d0 0x564ece59fcb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 --2- 172.21.15.31:0/3248194592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 -- 172.21.15.31:0/3248194592 >> 172.21.15.31:0/3248194592 conn(0x564ece50c260 msgr2=0x564ece593450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 -- 172.21.15.31:0/3248194592 shutdown_connections 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.723+0000 7efe24e55b80 1 -- 172.21.15.31:0/3248194592 wait complete. 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.724+0000 7efe24e55b80 1 -- start start 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.724+0000 7efe24e55b80 1 Processor -- start 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece5abf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece4eae30 con 0x564ece59f8d0 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece487710 con 0x564ece5ab0a0 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece4873b0 con 0x564ece5a01f0 2023-10-06T21:23:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe24e55b80 10 client.0 _ll_register_callbacks cb 0x564ece2e4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39298/0 (socket says 172.21.15.31:39298) 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 -- 172.21.15.31:0/467943588 learned_addr learned my addr 172.21.15.31:0/467943588 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0b7fe700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece5abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 -- 172.21.15.31:0/467943588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 msgr2=0x564ece5ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 -- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 msgr2=0x564ece5abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece5abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.725+0000 7efe0affd700 1 -- 172.21.15.31:0/467943588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe00017070 con 0x564ece59f8d0 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efe0b7fe700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece5abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efe0affd700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efdfc097940 tx=0x7efdfc005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de32319be9f885c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efdeeff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc09f590 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ece58af10 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7efdfc0b0020 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdfc0a3740 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efe24e55b80 10 client.16359 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.726+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efe0c1a61a0 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564ece58af10 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efdfc0a3340 con 0x564ece59f8d0 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 20 client.16359 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece5ba340 0x564ece5bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 4 client.16359 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ece4e3e10 con 0x564ece5ba340 2023-10-06T21:23:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.727+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece5ba340 0x564ece5bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.728+0000 7efdeeff5700 10 client.16359 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.728+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece5ba340 0x564ece5bc760 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad66c99bb92060 server_cookie=d53b653487f889e3 in_seq=0 out_seq=0 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efdeeff5700 1 -- 172.21.15.31:0/467943588 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ece4e3e10 con 0x564ece5ba340 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efdeeff5700 10 client.16359 handle_command_reply: tid=0 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 client.16359 shutdown 2023-10-06T21:23:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 msgr2=0x564ece5ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efdfc097940 tx=0x7efdfc005c40 comp rx=0 tx=0).stop 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 shutdown_connections 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece5abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece5ba340 0x564ece5bc760 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece5ab480 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 --2- 172.21.15.31:0/467943588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece5ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efdeeff5700 0 client.16359 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 >> 172.21.15.31:0/467943588 conn(0x564ece50c260 msgr2=0x564ece592f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 shutdown_connections 2023-10-06T21:23:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.729+0000 7efe24e55b80 1 -- 172.21.15.31:0/467943588 wait complete. 2023-10-06T21:23:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.730+0000 7efe24e55b80 20 client.16359 trim_cache size 0 max 16384 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.744+0000 7efe24e55b80 1 Processor -- start 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 -- start start 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece59f8d0 0x564ece5af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece36b920 con 0x564ece5ab0a0 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece5b65d0 con 0x564ece5a01f0 2023-10-06T21:23:02.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe24e55b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece5a10c0 con 0x564ece59f8d0 2023-10-06T21:23:02.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe0a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe0a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40354/0 (socket says 172.21.15.31:40354) 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.745+0000 7efe0a7fc700 1 -- 172.21.15.31:0/51081482 learned_addr learned my addr 172.21.15.31:0/51081482 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0affd700 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece59f8d0 0x564ece5af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 -- 172.21.15.31:0/51081482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece59f8d0 msgr2=0x564ece5af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece59f8d0 0x564ece5af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 -- 172.21.15.31:0/51081482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 msgr2=0x564ece5a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 -- 172.21.15.31:0/51081482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ece5aad40 con 0x564ece5a01f0 2023-10-06T21:23:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0b7fe700 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe00009b20 tx=0x7efe00014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5487c9e9eb96863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efdf1ffb700 1 -- 172.21.15.31:0/51081482 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0002a070 con 0x564ece5a01f0 2023-10-06T21:23:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efdf1ffb700 1 -- 172.21.15.31:0/51081482 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe00023040 con 0x564ece5a01f0 2023-10-06T21:23:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.746+0000 7efdf1ffb700 1 -- 172.21.15.31:0/51081482 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00029650 con 0x564ece5a01f0 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 -- 172.21.15.31:0/51081482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 msgr2=0x564ece5a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe00009b20 tx=0x7efe00014000 comp rx=0 tx=0).stop 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 -- 172.21.15.31:0/51081482 shutdown_connections 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece5ab0a0 0x564ece5a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece59f8d0 0x564ece5af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 --2- 172.21.15.31:0/51081482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5a01f0 0x564ece5a4780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 -- 172.21.15.31:0/51081482 >> 172.21.15.31:0/51081482 conn(0x564ece50c260 msgr2=0x564ece5b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 -- 172.21.15.31:0/51081482 shutdown_connections 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.747+0000 7efe24e55b80 1 -- 172.21.15.31:0/51081482 wait complete. 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 -- start start 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 Processor -- start 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece6027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece60d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece602f20 con 0x564ece59f8d0 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece60d680 con 0x564ece5ab0a0 2023-10-06T21:23:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ece60d810 con 0x564ece5a01f0 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.748+0000 7efe24e55b80 10 client.0 _ll_register_callbacks cb 0x7efe04009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39326/0 (socket says 172.21.15.31:39326) 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 -- 172.21.15.31:0/185403602 learned_addr learned my addr 172.21.15.31:0/185403602 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0b7fe700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece60d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece6027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 -- 172.21.15.31:0/185403602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 msgr2=0x564ece6027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece6027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 -- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 msgr2=0x564ece60d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:02.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece60d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 -- 172.21.15.31:0/185403602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe040117e0 con 0x564ece59f8d0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece6027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efe0affd700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efdfc097eb0 tx=0x7efdfc005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae7523b25f4a223 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.749+0000 7efdeeff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0a18a0 con 0x564ece59f8d0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdfc09b440 con 0x564ece59f8d0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7efdfc0b0070 con 0x564ece59f8d0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdfc09f590 con 0x564ece59f8d0 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 10 client.16365 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efe00011b20 con 0x564ece59f8d0 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdfc09b440 con 0x564ece59f8d0 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efdfc09a930 con 0x564ece59f8d0 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 20 client.16365 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece60ebb0 0x564ece610fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 4 client.16365 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.750+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ece4e3b30 con 0x564ece60ebb0 2023-10-06T21:23:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece60ebb0 0x564ece610fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efe0a7fc700 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece60ebb0 0x564ece610fd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba2e5e4f8bc59d9 server_cookie=34947684ab90c908 in_seq=0 out_seq=0 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efdeeff5700 10 client.16365 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efdeeff5700 1 -- 172.21.15.31:0/185403602 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ece4e3b30 con 0x564ece60ebb0 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efdeeff5700 10 client.16365 handle_command_reply: tid=0 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.751+0000 7efe24e55b80 1 client.16365 shutdown 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 msgr2=0x564ece6022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efdfc097eb0 tx=0x7efdfc005910 comp rx=0 tx=0).stop 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 shutdown_connections 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ece5ab0a0 0x564ece60d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564ece60ebb0 0x564ece610fd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ece59f8d0 0x564ece6022b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 --2- 172.21.15.31:0/185403602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ece5a01f0 0x564ece6027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 >> 172.21.15.31:0/185403602 conn(0x564ece50c260 msgr2=0x564ece5a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efdeeff5700 0 client.16365 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 shutdown_connections 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 1 -- 172.21.15.31:0/185403602 wait complete. 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.752+0000 7efe24e55b80 20 client.16365 trim_cache size 0 max 16384 2023-10-06T21:23:02.765 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T21:23:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 msgr2=0x7efdf40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 0x7efdf40a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe0c19a7f0 tx=0x7efe00009260 comp rx=0 tx=0).stop 2023-10-06T21:23:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 msgr2=0x7efe0c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005740 tx=0x7efdfc005770 comp rx=0 tx=0).stop 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 shutdown_connections 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe0c19db80 0x7efe0c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7efdf40a1670 0x7efdf40a3b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c103300 0x7efe0c1991e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 --2- 172.21.15.31:0/268371604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c108420 0x7efe0c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 >> 172.21.15.31:0/268371604 conn(0x7efe0c0feb40 msgr2=0x7efe0c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 shutdown_connections 2023-10-06T21:23:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:23:02.753+0000 7efe115ff700 1 -- 172.21.15.31:0/268371604 wait complete. 2023-10-06T21:23:02.774 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 475420284, 'ino': 1099511628320, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 768616312, 'ino': 1099511628299, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 784891703, 'ino': 1099511628298, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 1078894879, 'ino': 1099511628326, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 1320097941, 'ino': 1099511628311, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 1362107070, 'ino': 1099511628313, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 1396320325, 'ino': 1099511628314, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 1724282706, 'ino': 1099511628351, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 1735194928, 'ino': 1099511628322, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 1829424146, 'ino': 1099511628354, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 2057071126, 'ino': 1099511628323, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 2240972172, 'ino': 1099511628327, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 2359548126, 'ino': 1099511628324, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 2418594655, 'ino': 1099511628302, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 2548908049, 'ino': 1099511628353, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 2666914736, 'ino': 1099511628328, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 2863683827, 'ino': 1099511628310, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 3055924542, 'ino': 1099511628317, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 3080208503, 'ino': 1099511628318, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 3216720589, 'ino': 1099511628316, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 3305505859, 'ino': 1099511628319, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 3386454148, 'ino': 1099511628315, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 3661709194, 'ino': 1099511628301, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 3742581999, 'ino': 1099511628297, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 3890576528, 'ino': 1099511628355, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 4042609579, 'ino': 1099511628312, 'path': '/volumes/_nogroup/sv_0/0c5bd00c-9cf2-4f36-b104-6b12ec1d378a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}] 2023-10-06T21:23:02.775 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-10-06T21:23:03.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 18142 2.20 MB/sec warmup 53 sec latency 17.432 ms 2023-10-06T21:23:04.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:04 smithi031 ceph-mon[98539]: pgmap v66: 97 pgs: 97 active+clean; 28 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1020 KiB/s wr, 140 op/s 2023-10-06T21:23:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:04 smithi146 ceph-mon[102809]: pgmap v66: 97 pgs: 97 active+clean; 28 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1020 KiB/s wr, 140 op/s 2023-10-06T21:23:04.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 18340 2.16 MB/sec warmup 54 sec latency 18.797 ms 2023-10-06T21:23:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:04 smithi142 ceph-mon[102951]: pgmap v66: 97 pgs: 97 active+clean; 28 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1020 KiB/s wr, 140 op/s 2023-10-06T21:23:05.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 18685 2.13 MB/sec warmup 55 sec latency 22.286 ms 2023-10-06T21:23:06.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:06 smithi146 ceph-mon[102809]: pgmap v67: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 975 KiB/s wr, 102 op/s 2023-10-06T21:23:06.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 19020 2.12 MB/sec warmup 56 sec latency 20.800 ms 2023-10-06T21:23:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:06 smithi142 ceph-mon[102951]: pgmap v67: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 975 KiB/s wr, 102 op/s 2023-10-06T21:23:06.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:06 smithi031 ceph-mon[98539]: pgmap v67: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 975 KiB/s wr, 102 op/s 2023-10-06T21:23:07.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 19432 2.14 MB/sec warmup 57 sec latency 18.804 ms 2023-10-06T21:23:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:08 smithi146 ceph-mon[102809]: pgmap v68: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 802 KiB/s wr, 100 op/s 2023-10-06T21:23:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:08 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:08.329 INFO:tasks.workunit.client.0.smithi031.stdout: 1 19829 2.11 MB/sec warmup 58 sec latency 20.844 ms 2023-10-06T21:23:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:08 smithi142 ceph-mon[102951]: pgmap v68: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 802 KiB/s wr, 100 op/s 2023-10-06T21:23:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:08 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:08 smithi031 ceph-mon[98539]: pgmap v68: 97 pgs: 97 active+clean; 30 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 802 KiB/s wr, 100 op/s 2023-10-06T21:23:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:08 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:23:09.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 20217 2.10 MB/sec warmup 59 sec latency 18.970 ms 2023-10-06T21:23:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:23:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:23:10.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:10 smithi146 ceph-mon[102809]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2023-10-06T21:23:10.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 20697 2.15 MB/sec warmup 60 sec latency 18.441 ms 2023-10-06T21:23:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:10 smithi142 ceph-mon[102951]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2023-10-06T21:23:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:10 smithi031 ceph-mon[98539]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2023-10-06T21:23:11.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 21092 2.15 MB/sec warmup 61 sec latency 19.324 ms 2023-10-06T21:23:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 -- 172.21.15.31:0/1266978058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01010d0 msgr2=0x7f27d01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1266978058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01010d0 0x7f27d01014b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009990 tx=0x7f27c0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 -- 172.21.15.31:0/1266978058 shutdown_connections 2023-10-06T21:23:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1266978058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1266978058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01010d0 0x7f27d01014b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1266978058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d0101a80 0x7f27d0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 -- 172.21.15.31:0/1266978058 >> 172.21.15.31:0/1266978058 conn(0x7f27d00fc950 msgr2=0x7f27d00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 -- 172.21.15.31:0/1266978058 shutdown_connections 2023-10-06T21:23:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.110+0000 7f27d69ff700 1 -- 172.21.15.31:0/1266978058 wait complete. 2023-10-06T21:23:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.111+0000 7f27d69ff700 1 Processor -- start 2023-10-06T21:23:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.111+0000 7f27d69ff700 1 -- start start 2023-10-06T21:23:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d01010d0 0x7f27d0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01a30e0 0x7f27d01a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a5a10 con 0x7f27d01a30e0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a5b50 con 0x7f27d01061f0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d69ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01a5cc0 con 0x7f27d01010d0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27d4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01a30e0 0x7f27d01a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53290/0 (socket says 172.21.15.31:53290) 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 -- 172.21.15.31:0/1062623736 learned_addr learned my addr 172.21.15.31:0/1062623736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cffff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d01010d0 0x7f27d0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 -- 172.21.15.31:0/1062623736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d01010d0 msgr2=0x7f27d0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d01010d0 0x7f27d0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 -- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01a30e0 msgr2=0x7f27d01a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01a30e0 0x7f27d01a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.112+0000 7f27cf7fe700 1 -- 172.21.15.31:0/1062623736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00095f0 con 0x7f27d01061f0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27cd7fa700 1 -- 172.21.15.31:0/1062623736 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4009790 con 0x7f27d01061f0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27cd7fa700 1 -- 172.21.15.31:0/1062623736 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c400c7e0 con 0x7f27d01061f0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d01a5fa0 con 0x7f27d01061f0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27cd7fa700 1 -- 172.21.15.31:0/1062623736 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4005440 con 0x7f27d01061f0 2023-10-06T21:23:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.113+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d01a6420 con 0x7f27d01061f0 2023-10-06T21:23:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.118+0000 7f27cd7fa700 1 -- 172.21.15.31:0/1062623736 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f27c400c950 con 0x7f27d01061f0 2023-10-06T21:23:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.119+0000 7f27ea41db80 1 Processor -- start 2023-10-06T21:23:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.119+0000 7f27ea41db80 1 -- start start 2023-10-06T21:23:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab30c00 0x56300ab30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab31520 0x56300ab3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa9a400 con 0x56300ab3c3d0 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa8bbc0 con 0x56300ab31520 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27ea41db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa894e0 con 0x56300ab30c00 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cd7fa700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39522/0 (socket says 172.21.15.31:39522) 2023-10-06T21:23:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cf7fe700 1 -- 172.21.15.31:0/2476973872 learned_addr learned my addr 172.21.15.31:0/2476973872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab30c00 0x56300ab30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cffff700 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab31520 0x56300ab3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.120+0000 7f27cd7fa700 1 -- 172.21.15.31:0/1062623736 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f27c4014070 con 0x7f27d01061f0 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 -- 172.21.15.31:0/2476973872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab30c00 msgr2=0x56300ab30fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab30c00 0x56300ab30fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 -- 172.21.15.31:0/2476973872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab31520 msgr2=0x56300ab3be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab31520 0x56300ab3be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 -- 172.21.15.31:0/2476973872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300a8afd80 con 0x56300ab3c3d0 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ca910 tx=0x7f27c4099e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70740829e1f3cbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27b67fc700 1 -- 172.21.15.31:0/2476973872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c409a960 con 0x56300ab3c3d0 2023-10-06T21:23:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27b67fc700 1 -- 172.21.15.31:0/2476973872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c409a260 con 0x56300ab3c3d0 2023-10-06T21:23:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.121+0000 7f27b67fc700 1 -- 172.21.15.31:0/2476973872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40cac50 con 0x56300ab3c3d0 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 -- 172.21.15.31:0/2476973872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 msgr2=0x56300ab3e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 0x7f27b80a3ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009ec0 tx=0x7f27c8004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ca910 tx=0x7f27c4099e20 comp rx=0 tx=0).stop 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 -- 172.21.15.31:0/2476973872 shutdown_connections 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab3e7c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab30c00 0x56300ab30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 --2- 172.21.15.31:0/2476973872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab31520 0x56300ab3be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 -- 172.21.15.31:0/2476973872 >> 172.21.15.31:0/2476973872 conn(0x56300aaa6200 msgr2=0x56300ab2d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 -- 172.21.15.31:0/2476973872 shutdown_connections 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.122+0000 7f27ea41db80 1 -- 172.21.15.31:0/2476973872 wait complete. 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.123+0000 7f27ea41db80 1 -- start start 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.123+0000 7f27ea41db80 1 Processor -- start 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab3bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab3c3d0 0x56300ab39710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa84dd0 con 0x56300ab31520 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa216b0 con 0x56300ab30c00 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300aa21350 con 0x56300ab3c3d0 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27ea41db80 10 client.0 _ll_register_callbacks cb 0x56300a87e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39528/0 (socket says 172.21.15.31:39528) 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab3bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 -- 172.21.15.31:0/91836029 learned_addr learned my addr 172.21.15.31:0/91836029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab3c3d0 0x56300ab39710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 -- 172.21.15.31:0/91836029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab3c3d0 msgr2=0x56300ab39710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab3c3d0 0x56300ab39710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 -- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 msgr2=0x56300ab3bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab3bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.124+0000 7f27cf7fe700 1 -- 172.21.15.31:0/91836029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d01a6420 con 0x56300ab31520 2023-10-06T21:23:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27c4099f10 tx=0x7f27c409e2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e50aeb54cdbbc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40cae60 con 0x56300ab31520 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f27c40a25f0 con 0x56300ab31520 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56300ab24d50 con 0x56300ab31520 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c40b8350 con 0x56300ab31520 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27ea41db80 10 client.16377 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.125+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56300a898c00 con 0x56300ab31520 2023-10-06T21:23:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56300ab24d50 con 0x56300ab31520 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f27c40b2d20 con 0x56300ab31520 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 10 client.16377 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 20 client.16377 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300ab4bae0 0x56300ab4df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 4 client.16377 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56300aa7ddb0 con 0x56300ab4bae0 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.126+0000 7f27cffff700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300ab4bae0 0x56300ab4df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.127+0000 7f27cffff700 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300ab4bae0 0x56300ab4df00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c806a86d3a1fd42c server_cookie=42a3754de847dbda in_seq=0 out_seq=0 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.127+0000 7f27af7fe700 10 client.16377 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27af7fe700 1 -- 172.21.15.31:0/91836029 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56300aa7ddb0 con 0x56300ab4bae0 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27af7fe700 10 client.16377 handle_command_reply: tid=0 2023-10-06T21:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 client.16377 shutdown 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 msgr2=0x56300ab39070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27c4099f10 tx=0x7f27c409e2c0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 shutdown_connections 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab3c3d0 0x56300ab39710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab3bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300ab4bae0 0x56300ab4df00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 --2- 172.21.15.31:0/91836029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab31520 0x56300ab39070 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 >> 172.21.15.31:0/91836029 conn(0x56300aaa6200 msgr2=0x56300ab2cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27af7fe700 0 client.16377 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.128+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 shutdown_connections 2023-10-06T21:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.129+0000 7f27ea41db80 1 -- 172.21.15.31:0/91836029 wait complete. 2023-10-06T21:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.129+0000 7f27ea41db80 20 client.16377 trim_cache size 0 max 16384 2023-10-06T21:23:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.146+0000 7f27ea41db80 1 Processor -- start 2023-10-06T21:23:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.146+0000 7f27ea41db80 1 -- start start 2023-10-06T21:23:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300a9058c0 con 0x56300ab3c3d0 2023-10-06T21:23:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300ab2be30 con 0x56300ab30c00 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27ea41db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300ab47d80 con 0x56300ab31520 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27d4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27d4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39538/0 (socket says 172.21.15.31:39538) 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27d4f9c700 1 -- 172.21.15.31:0/1081782733 learned_addr learned my addr 172.21.15.31:0/1081782733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.147+0000 7f27cffff700 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 -- 172.21.15.31:0/1081782733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 msgr2=0x56300ab34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 -- 172.21.15.31:0/1081782733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 msgr2=0x56300ab4ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab4ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 -- 172.21.15.31:0/1081782733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300a8d61f0 con 0x56300ab3c3d0 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f27c80093b0 tx=0x7f27c8009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fe59808f565ca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27b6ffd700 1 -- 172.21.15.31:0/1081782733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8021050 con 0x56300ab3c3d0 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27b6ffd700 1 -- 172.21.15.31:0/1081782733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c8022040 con 0x56300ab3c3d0 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.148+0000 7f27b6ffd700 1 -- 172.21.15.31:0/1081782733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c800e040 con 0x56300ab3c3d0 2023-10-06T21:23:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 -- 172.21.15.31:0/1081782733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 msgr2=0x56300ab34890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f27c80093b0 tx=0x7f27c8009070 comp rx=0 tx=0).stop 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 -- 172.21.15.31:0/1081782733 shutdown_connections 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab3c3d0 0x56300ab34890 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab30c00 0x56300ab4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 --2- 172.21.15.31:0/1081782733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 -- 172.21.15.31:0/1081782733 >> 172.21.15.31:0/1081782733 conn(0x56300aaa6200 msgr2=0x56300ab47670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 -- 172.21.15.31:0/1081782733 shutdown_connections 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.149+0000 7f27ea41db80 1 -- 172.21.15.31:0/1081782733 wait complete. 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 -- start start 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 Processor -- start 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab3c3d0 0x56300ab40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300ab412e0 con 0x56300ab30c00 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300ab41450 con 0x56300ab3c3d0 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300ab41610 con 0x56300ab31520 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.150+0000 7f27ea41db80 10 client.0 _ll_register_callbacks cb 0x7f27c80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39542/0 (socket says 172.21.15.31:39542) 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cffff700 1 -- 172.21.15.31:0/481576117 learned_addr learned my addr 172.21.15.31:0/481576117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27d4f9c700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab3c3d0 0x56300ab40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 -- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab3c3d0 msgr2=0x56300ab40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab3c3d0 0x56300ab40d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 -- 172.21.15.31:0/481576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 msgr2=0x56300ab42850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cf7fe700 1 -- 172.21.15.31:0/481576117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56300a898c00 con 0x56300ab31520 2023-10-06T21:23:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.151+0000 7f27cffff700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27cf7fe700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab42d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27c400cef0 tx=0x7f27c409a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaef9ec30785707d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:23:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4099a00 con 0x56300ab31520 2023-10-06T21:23:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f27c40a25f0 con 0x56300ab31520 2023-10-06T21:23:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27c409a260 con 0x56300ab31520 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c40cae60 con 0x56300ab31520 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27ea41db80 10 client.26071 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f27c8009120 con 0x56300ab31520 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c409a260 con 0x56300ab31520 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.152+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f27c4096970 con 0x56300ab31520 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 10 client.26071 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 20 client.26071 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300aba06c0 0x56300aba2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 4 client.26071 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56300aa7dad0 con 0x56300aba06c0 2023-10-06T21:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.153+0000 7f27cffff700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300aba06c0 0x56300aba2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.154+0000 7f27af7fe700 10 client.26071 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.154+0000 7f27cffff700 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300aba06c0 0x56300aba2ae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0798fbff7b633d5 server_cookie=6d8e920a9ff34f28 in_seq=0 out_seq=0 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27af7fe700 1 -- 172.21.15.31:0/481576117 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56300aa7dad0 con 0x56300aba06c0 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27af7fe700 10 client.26071 handle_command_reply: tid=0 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 client.26071 shutdown 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 msgr2=0x56300ab42d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab42d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27c400cef0 tx=0x7f27c409a6d0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 shutdown_connections 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56300ab3c3d0 0x56300ab40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56300aba06c0 0x56300aba2ae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56300ab30c00 0x56300ab42850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 --2- 172.21.15.31:0/481576117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56300ab31520 0x56300ab42d90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27af7fe700 0 client.26071 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 >> 172.21.15.31:0/481576117 conn(0x56300aaa6200 msgr2=0x56300ab34350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 shutdown_connections 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 1 -- 172.21.15.31:0/481576117 wait complete. 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.175+0000 7f27ea41db80 20 client.26071 trim_cache size 0 max 16384 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:23:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:23:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:23:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 msgr2=0x7f27b80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 0x7f27b80a3ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009ec0 tx=0x7f27c8004b40 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 msgr2=0x7f27d00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ec80 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 shutdown_connections 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27d01a30e0 0x7f27d01a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27d01010d0 0x7f27d0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 --2- 172.21.15.31:0/1062623736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27d01061f0 0x7f27d00738d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 >> 172.21.15.31:0/1062623736 conn(0x7f27d00fc950 msgr2=0x7f27d0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 shutdown_connections 2023-10-06T21:23:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:12.176+0000 7f27d69ff700 1 -- 172.21.15.31:0/1062623736 wait complete. 2023-10-06T21:23:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:12 smithi146 ceph-mon[102809]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-10-06T21:23:12.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 21415 2.17 MB/sec warmup 62 sec latency 24.325 ms 2023-10-06T21:23:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:12 smithi142 ceph-mon[102951]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-10-06T21:23:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:12 smithi031 ceph-mon[98539]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-10-06T21:23:13.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 21639 2.15 MB/sec warmup 63 sec latency 18.050 ms 2023-10-06T21:23:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:14 smithi146 ceph-mon[102809]: pgmap v71: 97 pgs: 97 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:23:14.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 21810 2.12 MB/sec warmup 64 sec latency 18.529 ms 2023-10-06T21:23:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:14 smithi142 ceph-mon[102951]: pgmap v71: 97 pgs: 97 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:23:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:14 smithi031 ceph-mon[98539]: pgmap v71: 97 pgs: 97 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:23:15.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 22105 2.09 MB/sec warmup 65 sec latency 18.804 ms 2023-10-06T21:23:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:16 smithi146 ceph-mon[102809]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2023-10-06T21:23:16.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 22450 2.09 MB/sec warmup 66 sec latency 21.810 ms 2023-10-06T21:23:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:16 smithi142 ceph-mon[102951]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2023-10-06T21:23:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:16 smithi031 ceph-mon[98539]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2023-10-06T21:23:17.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 22832 2.10 MB/sec warmup 67 sec latency 19.571 ms 2023-10-06T21:23:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:18 smithi146 ceph-mon[102809]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 836 KiB/s wr, 89 op/s 2023-10-06T21:23:18.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 23198 2.08 MB/sec warmup 68 sec latency 20.983 ms 2023-10-06T21:23:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:18 smithi142 ceph-mon[102951]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 836 KiB/s wr, 89 op/s 2023-10-06T21:23:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:18 smithi031 ceph-mon[98539]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 836 KiB/s wr, 89 op/s 2023-10-06T21:23:19.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 23483 2.05 MB/sec warmup 69 sec latency 271.670 ms 2023-10-06T21:23:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:19 smithi142 ceph-mon[102951]: pgmap v74: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-10-06T21:23:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:19 smithi031 ceph-mon[98539]: pgmap v74: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-10-06T21:23:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:19 smithi146 ceph-mon[102809]: pgmap v74: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-10-06T21:23:20.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 23796 2.05 MB/sec warmup 70 sec latency 20.227 ms 2023-10-06T21:23:21.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 24292 2.09 MB/sec warmup 71 sec latency 20.441 ms 2023-10-06T21:23:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:22 smithi146 ceph-mon[102809]: pgmap v75: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 837 KiB/s wr, 92 op/s 2023-10-06T21:23:22.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 24646 2.09 MB/sec warmup 72 sec latency 19.879 ms 2023-10-06T21:23:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:22 smithi142 ceph-mon[102951]: pgmap v75: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 837 KiB/s wr, 92 op/s 2023-10-06T21:23:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:22 smithi031 ceph-mon[98539]: pgmap v75: 97 pgs: 97 active+clean; 34 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 837 KiB/s wr, 92 op/s 2023-10-06T21:23:23.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 24989 2.10 MB/sec warmup 73 sec latency 19.595 ms 2023-10-06T21:23:24.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:24 smithi146 ceph-mon[102809]: pgmap v76: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:23:24.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 25200 2.09 MB/sec warmup 74 sec latency 20.816 ms 2023-10-06T21:23:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:24 smithi142 ceph-mon[102951]: pgmap v76: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:23:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:24 smithi031 ceph-mon[98539]: pgmap v76: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:23:25.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 25378 2.06 MB/sec warmup 75 sec latency 16.165 ms 2023-10-06T21:23:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 -- 172.21.15.31:0/3723762968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508068c90 msgr2=0x7f350810dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 --2- 172.21.15.31:0/3723762968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508068c90 0x7f350810dd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f350000b0a0 tx=0x7f350000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 -- 172.21.15.31:0/3723762968 shutdown_connections 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 --2- 172.21.15.31:0/3723762968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3508068c90 0x7f350810dd00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 --2- 172.21.15.31:0/3723762968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508106f60 0x7f3508107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 --2- 172.21.15.31:0/3723762968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35080682d0 0x7f3508068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 -- 172.21.15.31:0/3723762968 >> 172.21.15.31:0/3723762968 conn(0x7f3508076b70 msgr2=0x7f3508076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 -- 172.21.15.31:0/3723762968 shutdown_connections 2023-10-06T21:23:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.347+0000 7f350fbff700 1 -- 172.21.15.31:0/3723762968 wait complete. 2023-10-06T21:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.348+0000 7f350fbff700 1 Processor -- start 2023-10-06T21:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 -- start start 2023-10-06T21:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 0x7f35081996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3508106f60 0x7f350819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350819e380 con 0x7f35080682d0 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350819e4f0 con 0x7f3508106f60 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350819e6b0 con 0x7f3508068c90 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40878/0 (socket says 172.21.15.31:40878) 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350d99b700 1 -- 172.21.15.31:0/362346277 learned_addr learned my addr 172.21.15.31:0/362346277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350d19a700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 0x7f35081996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.349+0000 7f350e19c700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3508106f60 0x7f350819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 -- 172.21.15.31:0/362346277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 msgr2=0x7f35081996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 0x7f35081996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 -- 172.21.15.31:0/362346277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3508106f60 msgr2=0x7f350819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3508106f60 0x7f350819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 -- 172.21.15.31:0/362346277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009d00 con 0x7f35080682d0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d19a700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 0x7f35081996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350d99b700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9a0 tx=0x7f350400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f34feffd700 1 -- 172.21.15.31:0/362346277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504009790 con 0x7f35080682d0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f34feffd700 1 -- 172.21.15.31:0/362346277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350400cbd0 con 0x7f35080682d0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350819e990 con 0x7f35080682d0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.350+0000 7f34feffd700 1 -- 172.21.15.31:0/362346277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504018680 con 0x7f35080682d0 2023-10-06T21:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.351+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35081a61d0 con 0x7f35080682d0 2023-10-06T21:23:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.356+0000 7f34feffd700 1 -- 172.21.15.31:0/362346277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f350400cd40 con 0x7f35080682d0 2023-10-06T21:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.357+0000 7f3523532b80 1 Processor -- start 2023-10-06T21:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.357+0000 7f3523532b80 1 -- start start 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd25de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd2607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd1bc400 con 0x5626cd25e3d0 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd1adbc0 con 0x5626cd253520 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f3523532b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd1ab4e0 con 0x5626cd252c00 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f34feffd700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 0x7f34f40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f34feffd700 1 -- 172.21.15.31:0/362346277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3504014070 con 0x7f35080682d0 2023-10-06T21:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350e19c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd25de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd2607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350e19c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48754/0 (socket says 172.21.15.31:48754) 2023-10-06T21:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350e19c700 1 -- 172.21.15.31:0/430176494 learned_addr learned my addr 172.21.15.31:0/430176494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.358+0000 7f350e19c700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 0x7f34f40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 -- 172.21.15.31:0/430176494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 msgr2=0x5626cd252fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 -- 172.21.15.31:0/430176494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 msgr2=0x5626cd2607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd2607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 -- 172.21.15.31:0/430176494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ccfd1d80 con 0x5626cd253520 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350e19c700 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350d99b700 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd25de90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f35040ca9c0 tx=0x7f350409a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa9b3e99e2245f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f34f3fff700 1 -- 172.21.15.31:0/430176494 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350409e160 con 0x5626cd253520 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f34f3fff700 1 -- 172.21.15.31:0/430176494 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350409e3a0 con 0x5626cd253520 2023-10-06T21:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f34f3fff700 1 -- 172.21.15.31:0/430176494 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504092150 con 0x5626cd253520 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.359+0000 7f350e19c700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 0x7f34f40a3b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f350000d010 tx=0x7f3500006210 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 -- 172.21.15.31:0/430176494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 msgr2=0x5626cd25de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd25de90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f35040ca9c0 tx=0x7f350409a150 comp rx=0 tx=0).stop 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 -- 172.21.15.31:0/430176494 shutdown_connections 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd2607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd252fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 --2- 172.21.15.31:0/430176494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd25de90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 -- 172.21.15.31:0/430176494 >> 172.21.15.31:0/430176494 conn(0x5626cd1c8200 msgr2=0x5626cd24f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 -- 172.21.15.31:0/430176494 shutdown_connections 2023-10-06T21:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.360+0000 7f3523532b80 1 -- 172.21.15.31:0/430176494 wait complete. 2023-10-06T21:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.361+0000 7f3523532b80 1 -- start start 2023-10-06T21:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.361+0000 7f3523532b80 1 Processor -- start 2023-10-06T21:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd25b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd1a6dd0 con 0x5626cd253520 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd1436b0 con 0x5626cd252c00 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd143350 con 0x5626cd25e3d0 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f3523532b80 10 client.0 _ll_register_callbacks cb 0x5626ccfa07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56654/0 (socket says 172.21.15.31:56654) 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350d99b700 1 -- 172.21.15.31:0/3851611915 learned_addr learned my addr 172.21.15.31:0/3851611915 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350e19c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd25b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.362+0000 7f350d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40894/0 (socket says 172.21.15.31:40894) 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 -- 172.21.15.31:0/3851611915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 msgr2=0x5626cd25b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd25b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 -- 172.21.15.31:0/3851611915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 msgr2=0x5626cd25b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 -- 172.21.15.31:0/3851611915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35081a61d0 con 0x5626cd252c00 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d19a700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f350d99b700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f350400c5b0 tx=0x7f350409a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe20da5c45749b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f34f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040b1070 con 0x5626cd252c00 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cd246d50 con 0x5626cd252c00 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f35040a25e0 con 0x5626cd252c00 2023-10-06T21:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f350409e160 con 0x5626cd252c00 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.363+0000 7f3523532b80 10 client.26126 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626ccfbac00 con 0x5626cd252c00 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5626cd246d50 con 0x5626cd252c00 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f35040a2150 con 0x5626cd252c00 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 20 client.26126 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd26db20 0x5626cd26ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 4 client.26126 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626cd19fdb0 con 0x5626cd26db20 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.364+0000 7f350d19a700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd26db20 0x5626cd26ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.365+0000 7f34f17fa700 10 client.26126 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.365+0000 7f350d19a700 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd26db20 0x5626cd26ff40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0d92fc64266f23 server_cookie=820dac511723935c in_seq=0 out_seq=0 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f34f17fa700 1 -- 172.21.15.31:0/3851611915 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5626cd19fdb0 con 0x5626cd26db20 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f34f17fa700 10 client.26126 handle_command_reply: tid=0 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 client.26126 shutdown 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 msgr2=0x5626cd25de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f350400c5b0 tx=0x7f350409a6e0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 shutdown_connections 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd25b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25de40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd26db20 0x5626cd26ff40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 --2- 172.21.15.31:0/3851611915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd25b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f34f17fa700 0 client.26126 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 >> 172.21.15.31:0/3851611915 conn(0x5626cd1c8200 msgr2=0x5626cd24ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 shutdown_connections 2023-10-06T21:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.366+0000 7f3523532b80 1 -- 172.21.15.31:0/3851611915 wait complete. 2023-10-06T21:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.367+0000 7f3523532b80 20 client.26126 trim_cache size 0 max 16384 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 Processor -- start 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 -- start start 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd264f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd24d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd0278c0 con 0x5626cd253520 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd25c550 con 0x5626cd252c00 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f3523532b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd265480 con 0x5626cd25e3d0 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f350d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f350d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd264f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f350d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56666/0 (socket says 172.21.15.31:56666) 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f350d99b700 1 -- 172.21.15.31:0/1963055138 learned_addr learned my addr 172.21.15.31:0/1963055138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.384+0000 7f350e19c700 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd24d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 -- 172.21.15.31:0/1963055138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 msgr2=0x5626cd24d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd24d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 -- 172.21.15.31:0/1963055138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 msgr2=0x5626cd25a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 -- 172.21.15.31:0/1963055138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ccff81f0 con 0x5626cd253520 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f350d19a700 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd264f10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5626cd2707d0 tx=0x7f34f801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7406686ad18254f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f34fcbff700 1 -- 172.21.15.31:0/1963055138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8020030 con 0x5626cd253520 2023-10-06T21:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f34fcbff700 1 -- 172.21.15.31:0/1963055138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f800be90 con 0x5626cd253520 2023-10-06T21:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.385+0000 7f34fcbff700 1 -- 172.21.15.31:0/1963055138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8017040 con 0x5626cd253520 2023-10-06T21:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 -- 172.21.15.31:0/1963055138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 msgr2=0x5626cd264f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd264f10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5626cd2707d0 tx=0x7f34f801cc90 comp rx=0 tx=0).stop 2023-10-06T21:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 -- 172.21.15.31:0/1963055138 shutdown_connections 2023-10-06T21:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd25e3d0 0x5626cd24d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd252c00 0x5626cd25a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 --2- 172.21.15.31:0/1963055138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd253520 0x5626cd264f10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 -- 172.21.15.31:0/1963055138 >> 172.21.15.31:0/1963055138 conn(0x5626cd1c8200 msgr2=0x5626cd274880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 -- 172.21.15.31:0/1963055138 shutdown_connections 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.386+0000 7f3523532b80 1 -- 172.21.15.31:0/1963055138 wait complete. 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.387+0000 7f3523532b80 1 -- start start 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.387+0000 7f3523532b80 1 Processor -- start 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd258b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd257350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd257890 con 0x5626cd25e3d0 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd257a00 con 0x5626cd253520 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cd257bc0 con 0x5626cd252c00 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f3523532b80 10 client.0 _ll_register_callbacks cb 0x5626ccfa07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd257350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48790/0 (socket says 172.21.15.31:48790) 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 -- 172.21.15.31:0/980607406 learned_addr learned my addr 172.21.15.31:0/980607406 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d19a700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd258b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 -- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 msgr2=0x5626cd258b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd258b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 -- 172.21.15.31:0/980607406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 msgr2=0x5626cd257350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd257350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.388+0000 7f350d99b700 1 -- 172.21.15.31:0/980607406 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8004350 con 0x5626cd252c00 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f350d99b700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35040cafc0 tx=0x7f3504010a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e133cdd19b4f6c32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f34f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040b1020 con 0x5626cd252c00 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f350409e160 con 0x5626cd252c00 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f350409e3a0 con 0x5626cd252c00 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35040a25e0 con 0x5626cd252c00 2023-10-06T21:23:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f3523532b80 10 client.26089 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626ccfd0250 con 0x5626cd252c00 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.389+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350409e3a0 con 0x5626cd252c00 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f35040991b0 con 0x5626cd252c00 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 10 client.26089 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 20 client.26089 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd2c2580 0x5626cd2c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 4 client.26089 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626cd19fad0 con 0x5626cd2c2580 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.390+0000 7f350d19a700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd2c2580 0x5626cd2c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.391+0000 7f350d19a700 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd2c2580 0x5626cd2c49a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80cf2b8b3f54c66 server_cookie=f00c3a04a53cdb2a in_seq=0 out_seq=0 2023-10-06T21:23:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.391+0000 7f34f17fa700 10 client.26089 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f34f17fa700 1 -- 172.21.15.31:0/980607406 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626cd19fad0 con 0x5626cd2c2580 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f34f17fa700 10 client.26089 handle_command_reply: tid=0 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 client.26089 shutdown 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 msgr2=0x5626cd2585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35040cafc0 tx=0x7f3504010a70 comp rx=0 tx=0).stop 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 shutdown_connections 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5626cd2c2580 0x5626cd2c49a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626cd25e3d0 0x5626cd257350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626cd252c00 0x5626cd2585f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f34f17fa700 0 client.26089 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 --2- 172.21.15.31:0/980607406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626cd253520 0x5626cd258b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.412+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 >> 172.21.15.31:0/980607406 conn(0x5626cd1c8200 msgr2=0x5626cd26a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 shutdown_connections 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f3523532b80 1 -- 172.21.15.31:0/980607406 wait complete. 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f3523532b80 20 client.26089 trim_cache size 0 max 16384 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 msgr2=0x7f34f40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 0x7f34f40a3b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f350000d010 tx=0x7f3500006210 comp rx=0 tx=0).stop 2023-10-06T21:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 msgr2=0x7f3508199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.413+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9a0 tx=0x7f350400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 shutdown_connections 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3508106f60 0x7f350819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34f40a1670 0x7f34f40a3b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35080682d0 0x7f3508199190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 --2- 172.21.15.31:0/362346277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3508068c90 0x7f35081996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 >> 172.21.15.31:0/362346277 conn(0x7f3508076b70 msgr2=0x7f350810c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 shutdown_connections 2023-10-06T21:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:25.414+0000 7f350fbff700 1 -- 172.21.15.31:0/362346277 wait complete. 2023-10-06T21:23:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:26 smithi146 ceph-mon[102809]: pgmap v77: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 942 KiB/s wr, 86 op/s 2023-10-06T21:23:26.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 25666 2.04 MB/sec warmup 76 sec latency 21.629 ms 2023-10-06T21:23:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:26 smithi142 ceph-mon[102951]: pgmap v77: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 942 KiB/s wr, 86 op/s 2023-10-06T21:23:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:26 smithi031 ceph-mon[98539]: pgmap v77: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 942 KiB/s wr, 86 op/s 2023-10-06T21:23:27.330 INFO:tasks.workunit.client.0.smithi031.stdout: 1 26018 2.04 MB/sec warmup 77 sec latency 20.966 ms 2023-10-06T21:23:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:28 smithi146 ceph-mon[102809]: pgmap v78: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 732 KiB/s wr, 83 op/s 2023-10-06T21:23:28.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 26407 2.05 MB/sec warmup 78 sec latency 18.907 ms 2023-10-06T21:23:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:28 smithi142 ceph-mon[102951]: pgmap v78: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 732 KiB/s wr, 83 op/s 2023-10-06T21:23:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:28 smithi031 ceph-mon[98539]: pgmap v78: 97 pgs: 97 active+clean; 37 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 732 KiB/s wr, 83 op/s 2023-10-06T21:23:29.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 26793 2.03 MB/sec warmup 79 sec latency 18.030 ms 2023-10-06T21:23:30.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:29 smithi142 ceph-mon[102951]: pgmap v79: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-10-06T21:23:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:29 smithi031 ceph-mon[98539]: pgmap v79: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-10-06T21:23:30.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:29 smithi146 ceph-mon[102809]: pgmap v79: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-10-06T21:23:30.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 27237 2.02 MB/sec warmup 80 sec latency 20.625 ms 2023-10-06T21:23:31.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 27445 2.01 MB/sec warmup 81 sec latency 290.129 ms 2023-10-06T21:23:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:32 smithi146 ceph-mon[102809]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 823 KiB/s wr, 87 op/s 2023-10-06T21:23:32.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 28030 2.05 MB/sec warmup 82 sec latency 18.886 ms 2023-10-06T21:23:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:32 smithi142 ceph-mon[102951]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 823 KiB/s wr, 87 op/s 2023-10-06T21:23:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:32 smithi031 ceph-mon[98539]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 823 KiB/s wr, 87 op/s 2023-10-06T21:23:33.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 28235 2.04 MB/sec warmup 83 sec latency 21.045 ms 2023-10-06T21:23:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:34 smithi146 ceph-mon[102809]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2023-10-06T21:23:34.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 28637 2.07 MB/sec warmup 84 sec latency 18.688 ms 2023-10-06T21:23:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:34 smithi142 ceph-mon[102951]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2023-10-06T21:23:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:34 smithi031 ceph-mon[98539]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2023-10-06T21:23:35.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 28803 2.05 MB/sec warmup 85 sec latency 19.874 ms 2023-10-06T21:23:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:36 smithi146 ceph-mon[102809]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 754 KiB/s wr, 91 op/s 2023-10-06T21:23:36.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 28988 2.02 MB/sec warmup 86 sec latency 18.862 ms 2023-10-06T21:23:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:36 smithi142 ceph-mon[102951]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 754 KiB/s wr, 91 op/s 2023-10-06T21:23:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:36 smithi031 ceph-mon[98539]: pgmap v82: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 754 KiB/s wr, 91 op/s 2023-10-06T21:23:37.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 29307 2.01 MB/sec warmup 87 sec latency 19.857 ms 2023-10-06T21:23:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:38 smithi146 ceph-mon[102809]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 746 KiB/s wr, 90 op/s 2023-10-06T21:23:38.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 29639 2.00 MB/sec warmup 88 sec latency 16.637 ms 2023-10-06T21:23:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:38 smithi142 ceph-mon[102951]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 746 KiB/s wr, 90 op/s 2023-10-06T21:23:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:38 smithi031 ceph-mon[98539]: pgmap v83: 97 pgs: 97 active+clean; 39 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 746 KiB/s wr, 90 op/s 2023-10-06T21:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 -- 172.21.15.31:0/1901750448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 msgr2=0x7fa658101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 --2- 172.21.15.31:0/1901750448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658101300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa654009990 tx=0x7fa654009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 -- 172.21.15.31:0/1901750448 shutdown_connections 2023-10-06T21:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 --2- 172.21.15.31:0/1901750448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa658106040 0x7fa65810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 --2- 172.21.15.31:0/1901750448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658101300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 --2- 172.21.15.31:0/1901750448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6581018d0 0x7fa658105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 -- 172.21.15.31:0/1901750448 >> 172.21.15.31:0/1901750448 conn(0x7fa6580fc7c0 msgr2=0x7fa6580febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 -- 172.21.15.31:0/1901750448 shutdown_connections 2023-10-06T21:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.582+0000 7fa65e59d700 1 -- 172.21.15.31:0/1901750448 wait complete. 2023-10-06T21:23:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.583+0000 7fa65e59d700 1 Processor -- start 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.583+0000 7fa65e59d700 1 -- start start 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 0x7fa658199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa65819d8a0 0x7fa65819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819e320 con 0x7fa658106040 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819e490 con 0x7fa658100f20 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65e59d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819e650 con 0x7fa65819d8a0 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65cd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 0x7fa658199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45644/0 (socket says 172.21.15.31:45644) 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 -- 172.21.15.31:0/895746085 learned_addr learned my addr 172.21.15.31:0/895746085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa65819d8a0 0x7fa65819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 -- 172.21.15.31:0/895746085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa65819d8a0 msgr2=0x7fa65819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa65819d8a0 0x7fa65819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 -- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 msgr2=0x7fa658199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 0x7fa658199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65d59b700 1 -- 172.21.15.31:0/895746085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540095f0 con 0x7fa658100f20 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.584+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 0x7fa658199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa65d59b700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098e0 tx=0x7fa654009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa64e7fc700 1 -- 172.21.15.31:0/895746085 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65401d070 con 0x7fa658100f20 2023-10-06T21:23:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa64e7fc700 1 -- 172.21.15.31:0/895746085 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa654021410 con 0x7fa658100f20 2023-10-06T21:23:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa64e7fc700 1 -- 172.21.15.31:0/895746085 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654021ea0 con 0x7fa658100f20 2023-10-06T21:23:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6581a5b50 con 0x7fa658100f20 2023-10-06T21:23:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.585+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6581a5fd0 con 0x7fa658100f20 2023-10-06T21:23:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.589+0000 7fa64e7fc700 1 -- 172.21.15.31:0/895746085 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa654021580 con 0x7fa658100f20 2023-10-06T21:23:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 Processor -- start 2023-10-06T21:23:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 -- start start 2023-10-06T21:23:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817cc680 0x563d817d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 0x563d817d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d81735400 con 0x563d817cc680 2023-10-06T21:23:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d81726bc0 con 0x563d817d7530 2023-10-06T21:23:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.590+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d817244e0 con 0x563d817cbd60 2023-10-06T21:23:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 0x563d817d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44782/0 (socket says 172.21.15.31:44782) 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 -- 172.21.15.31:0/546645586 learned_addr learned my addr 172.21.15.31:0/546645586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65d59b700 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817cc680 0x563d817d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa64e7fc700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 0x7fa6440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 -- 172.21.15.31:0/546645586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 msgr2=0x563d817d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 0x563d817d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 -- 172.21.15.31:0/546645586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817cc680 msgr2=0x563d817d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817cc680 0x563d817d6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 -- 172.21.15.31:0/546645586 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d8154ad80 con 0x563d817cbd60 2023-10-06T21:23:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa64e7fc700 1 -- 172.21.15.31:0/895746085 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fa6540ce1c0 con 0x7fa658100f20 2023-10-06T21:23:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 0x563d817d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.591+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 0x7fa6440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.592+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa650011790 tx=0x7fa650011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a0edff904f43a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.592+0000 7fa6437fe700 1 -- 172.21.15.31:0/546645586 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000e040 con 0x563d817cbd60 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.592+0000 7fa6437fe700 1 -- 172.21.15.31:0/546645586 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa65000f040 con 0x563d817cbd60 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.592+0000 7fa6437fe700 1 -- 172.21.15.31:0/546645586 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6500156e0 con 0x563d817cbd60 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.592+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 0x7fa6440a3b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa650022000 tx=0x7fa65001f000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 -- 172.21.15.31:0/546645586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 msgr2=0x563d817cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa650011790 tx=0x7fa650011aa0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 -- 172.21.15.31:0/546645586 shutdown_connections 2023-10-06T21:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817d7530 0x563d817d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cc140 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/546645586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817cc680 0x563d817d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 -- 172.21.15.31:0/546645586 >> 172.21.15.31:0/546645586 conn(0x563d81741200 msgr2=0x563d817c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 -- 172.21.15.31:0/546645586 shutdown_connections 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.593+0000 7fa6730bdb80 1 -- 172.21.15.31:0/546645586 wait complete. 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.594+0000 7fa6730bdb80 1 -- start start 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.594+0000 7fa6730bdb80 1 Processor -- start 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8171fdd0 con 0x563d817d7530 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d816bc6b0 con 0x563d817cc680 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d816bc350 con 0x563d817cbd60 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa6730bdb80 10 client.0 _ll_register_callbacks cb 0x563d815197a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45662/0 (socket says 172.21.15.31:45662) 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 -- 172.21.15.31:0/1293624432 learned_addr learned my addr 172.21.15.31:0/1293624432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 -- 172.21.15.31:0/1293624432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 msgr2=0x563d817d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 -- 172.21.15.31:0/1293624432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 msgr2=0x563d817d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.595+0000 7fa65cd9a700 1 -- 172.21.15.31:0/1293624432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa648009c90 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa648011d00 tx=0x7fa64800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58005d41a00865b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa640ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648016070 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d817bfeb0 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fa648004030 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa648004230 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa6730bdb80 10 client.26144 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.596+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa650011380 con 0x563d817cc680 2023-10-06T21:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d817bfeb0 con 0x563d817cc680 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa648022400 con 0x563d817cc680 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 10 client.26144 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 20 client.26144 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d817e7180 0x563d817e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 4 client.26144 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d81718db0 con 0x563d817e7180 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.597+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d817e7180 0x563d817e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.598+0000 7fa640ff9700 10 client.26144 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.598+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d817e7180 0x563d817e95a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454486cf16cc327c server_cookie=23e38ba4503bf184 in_seq=0 out_seq=0 2023-10-06T21:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.598+0000 7fa640ff9700 1 -- 172.21.15.31:0/1293624432 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d81718db0 con 0x563d817e7180 2023-10-06T21:23:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.598+0000 7fa640ff9700 10 client.26144 handle_command_reply: tid=0 2023-10-06T21:23:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.598+0000 7fa6730bdb80 1 client.26144 shutdown 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 msgr2=0x563d817d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa648011d00 tx=0x7fa64800f6a0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 shutdown_connections 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d817e7180 0x563d817e95a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1293624432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817d4760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 >> 172.21.15.31:0/1293624432 conn(0x563d81741200 msgr2=0x563d817c7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa640ff9700 0 client.26144 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 shutdown_connections 2023-10-06T21:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.599+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1293624432 wait complete. 2023-10-06T21:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.600+0000 7fa6730bdb80 20 client.26144 trim_cache size 0 max 16384 2023-10-06T21:23:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.619+0000 7fa6730bdb80 1 Processor -- start 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.619+0000 7fa6730bdb80 1 -- start start 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817de080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d815a08c0 con 0x563d817d7530 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d817de620 con 0x563d817cc680 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d817c6fc0 con 0x563d817cbd60 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa65d59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48644/0 (socket says 172.21.15.31:48644) 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa65dd9c700 1 -- 172.21.15.31:0/3020751476 learned_addr learned my addr 172.21.15.31:0/3020751476 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.620+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817de080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 -- 172.21.15.31:0/3020751476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 msgr2=0x563d817de080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817de080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 -- 172.21.15.31:0/3020751476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 msgr2=0x563d817d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 -- 172.21.15.31:0/3020751476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d817ee190 con 0x563d817cbd60 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65dd9c700 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa65d59b700 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cecd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6540cb8c0 tx=0x7fa65400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c68005cce41cbcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa643fff700 1 -- 172.21.15.31:0/3020751476 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65409afb0 con 0x563d817cbd60 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa643fff700 1 -- 172.21.15.31:0/3020751476 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa65409ad20 con 0x563d817cbd60 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.621+0000 7fa643fff700 1 -- 172.21.15.31:0/3020751476 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65409f3c0 con 0x563d817cbd60 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- 172.21.15.31:0/3020751476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 msgr2=0x563d817cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cecd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6540cb8c0 tx=0x7fa65400b390 comp rx=0 tx=0).stop 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- 172.21.15.31:0/3020751476 shutdown_connections 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d817d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d817cecd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/3020751476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d817de080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- 172.21.15.31:0/3020751476 >> 172.21.15.31:0/3020751476 conn(0x563d81741200 msgr2=0x563d817cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- 172.21.15.31:0/3020751476 shutdown_connections 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- 172.21.15.31:0/3020751476 wait complete. 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.622+0000 7fa6730bdb80 1 -- start start 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 Processor -- start 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d8182f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d81839f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8182fcb0 con 0x563d817d7530 2023-10-06T21:23:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8183a4c0 con 0x563d817cc680 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8183a600 con 0x563d817cbd60 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.623+0000 7fa6730bdb80 10 client.0 _ll_register_callbacks cb 0x7fa6480117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d81839f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44810/0 (socket says 172.21.15.31:44810) 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d8182f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d81839f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48656/0 (socket says 172.21.15.31:48656) 2023-10-06T21:23:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 -- 172.21.15.31:0/1633097432 learned_addr learned my addr 172.21.15.31:0/1633097432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 -- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 msgr2=0x563d8182f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d8182f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 -- 172.21.15.31:0/1633097432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 msgr2=0x563d81839f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d81839f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 -- 172.21.15.31:0/1633097432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa650027070 con 0x563d817cbd60 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d8182f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa65d59b700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c010 tx=0x7fa65409b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22904ff0d69fd3c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.624+0000 7fa640ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65409b8e0 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fa6540a34e0 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa65000f040 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa65409f330 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 10 client.26107 fetch_fsmap learned FSMap version 30 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa6581a5fd0 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa65000f040 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa654097040 con 0x563d817cbd60 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 10 client.26107 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T21:23:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 20 client.26107 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d8183ba30 0x563d8183de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 4 client.26107 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.625+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d81718ad0 con 0x563d8183ba30 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.626+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d8183ba30 0x563d8183de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.626+0000 7fa65cd9a700 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d8183ba30 0x563d8183de50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=205f1526034f337d server_cookie=dbc5635a88e97994 in_seq=0 out_seq=0 2023-10-06T21:23:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.626+0000 7fa640ff9700 10 client.26107 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa640ff9700 1 -- 172.21.15.31:0/1633097432 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d81718ad0 con 0x563d8183ba30 2023-10-06T21:23:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa640ff9700 10 client.26107 handle_command_reply: tid=0 2023-10-06T21:23:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 client.26107 shutdown 2023-10-06T21:23:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 msgr2=0x563d8182ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c010 tx=0x7fa65409b0c0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 shutdown_connections 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563d8183ba30 0x563d8183de50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d817d7530 0x563d81839f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d817cbd60 0x563d8182ef00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa640ff9700 0 client.26107 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 --2- 172.21.15.31:0/1633097432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d817cc680 0x563d8182f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 >> 172.21.15.31:0/1633097432 conn(0x563d81741200 msgr2=0x563d817d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 shutdown_connections 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 1 -- 172.21.15.31:0/1633097432 wait complete. 2023-10-06T21:23:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.650+0000 7fa6730bdb80 20 client.26107 trim_cache size 0 max 16384 2023-10-06T21:23:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:23:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:23:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:23:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 msgr2=0x7fa6440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 0x7fa6440a3b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa650022000 tx=0x7fa65001f000 comp rx=0 tx=0).stop 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 msgr2=0x7fa658198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098e0 tx=0x7fa654009580 comp rx=0 tx=0).stop 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 shutdown_connections 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa65819d8a0 0x7fa65819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa658100f20 0x7fa658198f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa6440a1670 0x7fa6440a3b30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 --2- 172.21.15.31:0/895746085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa658106040 0x7fa658199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 >> 172.21.15.31:0/895746085 conn(0x7fa6580fc7c0 msgr2=0x7fa6581028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 shutdown_connections 2023-10-06T21:23:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:38.651+0000 7fa65e59d700 1 -- 172.21.15.31:0/895746085 wait complete. 2023-10-06T21:23:39.021 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:23:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: ignoring --setuser ceph since I am not root 2023-10-06T21:23:39.021 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:23:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: ignoring --setgroup ceph since I am not root 2023-10-06T21:23:39.021 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:23:38 smithi142 ceph-mds[140278]: -- 172.21.15.142:0/1274663029 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555de79c51e0 con 0x555de799ac00 2023-10-06T21:23:39.021 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:23:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: starting mds.h at 2023-10-06T21:23:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:39 smithi146 ceph-mon[102809]: daemon mds.h restarted 2023-10-06T21:23:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:39 smithi146 ceph-mon[102809]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:23:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:39 smithi146 ceph-mon[102809]: mds.? [v2:172.21.15.142:6836/2494492874,v1:172.21.15.142:6837/2494492874] up:boot 2023-10-06T21:23:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:39 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:23:39.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:39 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:23:39.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 30036 2.01 MB/sec warmup 89 sec latency 26.103 ms 2023-10-06T21:23:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:39 smithi142 ceph-mon[102951]: daemon mds.h restarted 2023-10-06T21:23:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:39 smithi142 ceph-mon[102951]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:23:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:39 smithi142 ceph-mon[102951]: mds.? [v2:172.21.15.142:6836/2494492874,v1:172.21.15.142:6837/2494492874] up:boot 2023-10-06T21:23:39.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:39 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:23:39.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:39 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:23:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:39 smithi031 ceph-mon[98539]: daemon mds.h restarted 2023-10-06T21:23:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:39 smithi031 ceph-mon[98539]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:23:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:39 smithi031 ceph-mon[98539]: mds.? [v2:172.21.15.142:6836/2494492874,v1:172.21.15.142:6837/2494492874] up:boot 2023-10-06T21:23:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:39 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:23:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:39 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:23:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:40 smithi146 ceph-mon[102809]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:40.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 30456 2.00 MB/sec warmup 90 sec latency 21.889 ms 2023-10-06T21:23:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:40 smithi142 ceph-mon[102951]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:40 smithi031 ceph-mon[98539]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:41.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 30826 1.99 MB/sec warmup 91 sec latency 17.159 ms 2023-10-06T21:23:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:42 smithi146 ceph-mon[102809]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:42.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 31292 2.01 MB/sec warmup 92 sec latency 20.980 ms 2023-10-06T21:23:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:42 smithi142 ceph-mon[102951]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:42 smithi031 ceph-mon[98539]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 980 KiB/s wr, 102 op/s 2023-10-06T21:23:43.331 INFO:tasks.workunit.client.0.smithi031.stdout: 1 31721 2.03 MB/sec warmup 93 sec latency 18.090 ms 2023-10-06T21:23:44.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:44 smithi146 ceph-mon[102809]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:44.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 32035 2.04 MB/sec warmup 94 sec latency 19.562 ms 2023-10-06T21:23:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:44 smithi142 ceph-mon[102951]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:44 smithi031 ceph-mon[98539]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:45.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 32266 2.03 MB/sec warmup 95 sec latency 20.677 ms 2023-10-06T21:23:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:46 smithi146 ceph-mon[102809]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:46.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 32440 2.01 MB/sec warmup 96 sec latency 18.030 ms 2023-10-06T21:23:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:46 smithi142 ceph-mon[102951]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:46 smithi031 ceph-mon[98539]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:47.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 32706 1.99 MB/sec warmup 97 sec latency 20.862 ms 2023-10-06T21:23:48.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:48 smithi146 ceph-mon[102809]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:48.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 33067 1.99 MB/sec warmup 98 sec latency 18.084 ms 2023-10-06T21:23:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:48 smithi142 ceph-mon[102951]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:48 smithi031 ceph-mon[98539]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1007 KiB/s wr, 112 op/s 2023-10-06T21:23:49.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 33460 2.00 MB/sec warmup 99 sec latency 18.871 ms 2023-10-06T21:23:50.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:49 smithi142 ceph-mon[102951]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 893 KiB/s wr, 109 op/s 2023-10-06T21:23:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:49 smithi031 ceph-mon[98539]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 893 KiB/s wr, 109 op/s 2023-10-06T21:23:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:49 smithi146 ceph-mon[102809]: pgmap v90: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 893 KiB/s wr, 109 op/s 2023-10-06T21:23:50.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 33832 1.98 MB/sec warmup 100 sec latency 21.134 ms 2023-10-06T21:23:51.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 34269 1.98 MB/sec warmup 101 sec latency 67.224 ms 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.818+0000 7f21023ff700 1 -- 172.21.15.31:0/2674163463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc101a80 msgr2=0x7f20fc105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.818+0000 7f21023ff700 1 --2- 172.21.15.31:0/2674163463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc101a80 0x7f20fc105ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20f00098f0 tx=0x7f20f0009c00 comp rx=0 tx=0).stop 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 -- 172.21.15.31:0/2674163463 shutdown_connections 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 --2- 172.21.15.31:0/2674163463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 0x7f20fc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 --2- 172.21.15.31:0/2674163463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 --2- 172.21.15.31:0/2674163463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc101a80 0x7f20fc105ac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 -- 172.21.15.31:0/2674163463 >> 172.21.15.31:0/2674163463 conn(0x7f20fc0fc950 msgr2=0x7f20fc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 -- 172.21.15.31:0/2674163463 shutdown_connections 2023-10-06T21:23:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.819+0000 7f21023ff700 1 -- 172.21.15.31:0/2674163463 wait complete. 2023-10-06T21:23:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.820+0000 7f21023ff700 1 Processor -- start 2023-10-06T21:23:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.820+0000 7f21023ff700 1 -- start start 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 0x7f20fc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc19bb60 0x7f20fc19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c5e0 con 0x7f20fc1061f0 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c750 con 0x7f20fc1010d0 2023-10-06T21:23:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f21023ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c910 con 0x7f20fc19bb60 2023-10-06T21:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.821+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34494/0 (socket says 172.21.15.31:34494) 2023-10-06T21:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 -- 172.21.15.31:0/1660141364 learned_addr learned my addr 172.21.15.31:0/1660141364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f210099c700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc19bb60 0x7f20fc19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 0x7f20fc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 -- 172.21.15.31:0/1660141364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc19bb60 msgr2=0x7f20fc19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc19bb60 0x7f20fc19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 -- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 msgr2=0x7f20fc1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 0x7f20fc1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.822+0000 7f20fbfff700 1 -- 172.21.15.31:0/1660141364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f4009c40 con 0x7f20fc1010d0 2023-10-06T21:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f20fbfff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009f80 tx=0x7f20ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f20f97fa700 1 -- 172.21.15.31:0/1660141364 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec013be0 con 0x7f20fc1010d0 2023-10-06T21:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f20f97fa700 1 -- 172.21.15.31:0/1660141364 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec00ee50 con 0x7f20fc1010d0 2023-10-06T21:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f20f97fa700 1 -- 172.21.15.31:0/1660141364 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00cc50 con 0x7f20fc1010d0 2023-10-06T21:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f0009520 con 0x7f20fc1010d0 2023-10-06T21:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.823+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc073440 con 0x7f20fc1010d0 2023-10-06T21:23:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.827+0000 7f20f97fa700 1 -- 172.21.15.31:0/1660141364 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f20ec004750 con 0x7f20fc1010d0 2023-10-06T21:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.828+0000 7f2115c4db80 1 Processor -- start 2023-10-06T21:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.828+0000 7f2115c4db80 1 -- start start 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x563029703fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x5630297117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302966d400 con 0x563029703c00 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302965ebc0 con 0x563029704520 2023-10-06T21:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f2115c4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302965c4e0 con 0x56302970f3d0 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f20f97fa700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 0x7f20e40a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.829+0000 7f20f97fa700 1 -- 172.21.15.31:0/1660141364 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f20ec0d10f0 con 0x7f20fc1010d0 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34506/0 (socket says 172.21.15.31:34506) 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 -- 172.21.15.31:0/373624968 learned_addr learned my addr 172.21.15.31:0/373624968 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x5630297117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f210099c700 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x563029703fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f210099c700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 0x7f20e40a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 -- 172.21.15.31:0/373624968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 msgr2=0x5630297117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x5630297117c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 -- 172.21.15.31:0/373624968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 msgr2=0x563029703fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x563029703fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fbfff700 1 -- 172.21.15.31:0/373624968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563029482d80 con 0x563029704520 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.830+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x5630297117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.831+0000 7f20fbfff700 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20ec09a690 tx=0x7f20ec09db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd35866cb545c3d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.831+0000 7f20e27fc700 1 -- 172.21.15.31:0/373624968 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec09e190 con 0x563029704520 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.831+0000 7f20e27fc700 1 -- 172.21.15.31:0/373624968 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec0a22e0 con 0x563029704520 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.831+0000 7f210099c700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 0x7f20e40a3c00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009fd0 tx=0x7f20f400b040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.831+0000 7f20e27fc700 1 -- 172.21.15.31:0/373624968 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec09e3f0 con 0x563029704520 2023-10-06T21:23:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 -- 172.21.15.31:0/373624968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 msgr2=0x56302970ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20ec09a690 tx=0x7f20ec09db70 comp rx=0 tx=0).stop 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 -- 172.21.15.31:0/373624968 shutdown_connections 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x5630297117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x563029703fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 --2- 172.21.15.31:0/373624968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970ee90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 -- 172.21.15.31:0/373624968 >> 172.21.15.31:0/373624968 conn(0x563029679200 msgr2=0x563029700080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 -- 172.21.15.31:0/373624968 shutdown_connections 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.832+0000 7f2115c4db80 1 -- 172.21.15.31:0/373624968 wait complete. 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.833+0000 7f2115c4db80 1 -- start start 2023-10-06T21:23:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.833+0000 7f2115c4db80 1 Processor -- start 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x56302970c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x56302970cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029657dd0 con 0x563029703c00 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630295f46b0 con 0x563029704520 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630295f4350 con 0x56302970f3d0 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f2115c4db80 10 client.0 _ll_register_callbacks cb 0x5630294517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f20fb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f20fb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34518/0 (socket says 172.21.15.31:34518) 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f20fb7fe700 1 -- 172.21.15.31:0/3800392710 learned_addr learned my addr 172.21.15.31:0/3800392710 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.834+0000 7f20fbfff700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x56302970c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f210099c700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x56302970cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 -- 172.21.15.31:0/3800392710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 msgr2=0x56302970cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x56302970cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 -- 172.21.15.31:0/3800392710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 msgr2=0x56302970c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x56302970c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 -- 172.21.15.31:0/3800392710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc073440 con 0x563029704520 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fbfff700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x56302970c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20fc06a480 tx=0x7f20f000f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c656bdc1bbc500 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.835+0000 7f20db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:23:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f001c040 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f20f000be60 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630296f7d50 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f000b910 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f2115c4db80 10 client.26168 fetch_fsmap learned FSMap version 31 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56302946bc00 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.836+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630296f7d50 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f20f001f030 con 0x563029704520 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 10 client.26168 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 20 client.26168 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56302971ef40 0x563029721360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 4 client.26168 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563029650db0 con 0x56302971ef40 2023-10-06T21:23:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.837+0000 7f20fbfff700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56302971ef40 0x563029721360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.838+0000 7f20fbfff700 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56302971ef40 0x563029721360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee454a8655572cb server_cookie=28694d751113ef92 in_seq=0 out_seq=0 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.838+0000 7f20db7fe700 10 client.26168 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.838+0000 7f20db7fe700 1 -- 172.21.15.31:0/3800392710 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563029650db0 con 0x56302971ef40 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.838+0000 7f20db7fe700 10 client.26168 handle_command_reply: tid=0 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.838+0000 7f2115c4db80 1 client.26168 shutdown 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 msgr2=0x56302970c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20fc06a480 tx=0x7f20f000f7c0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 shutdown_connections 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56302970f3d0 0x56302970cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56302971ef40 0x563029721360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563029703c00 0x56302970c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 --2- 172.21.15.31:0/3800392710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302970c690 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f20db7fe700 0 client.26168 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 >> 172.21.15.31:0/3800392710 conn(0x563029679200 msgr2=0x5630296ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 shutdown_connections 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 1 -- 172.21.15.31:0/3800392710 wait complete. 2023-10-06T21:23:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.839+0000 7f2115c4db80 20 client.26168 trim_cache size 0 max 16384 2023-10-06T21:23:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.855+0000 7f2115c4db80 1 Processor -- start 2023-10-06T21:23:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.855+0000 7f2115c4db80 1 -- start start 2023-10-06T21:23:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 0x563029706aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 0x563029715ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630294d88c0 con 0x56302970f3d0 2023-10-06T21:23:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029716400 con 0x563029703c00 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f2115c4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029708000 con 0x563029704520 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f210099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f210099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57982/0 (socket says 172.21.15.31:57982) 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f210099c700 1 -- 172.21.15.31:0/1708424038 learned_addr learned my addr 172.21.15.31:0/1708424038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 0x563029715ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.856+0000 7f20fbfff700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 0x563029706aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 -- 172.21.15.31:0/1708424038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 msgr2=0x563029715ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 0x563029715ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 -- 172.21.15.31:0/1708424038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 msgr2=0x563029706aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 0x563029706aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 -- 172.21.15.31:0/1708424038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630294a91f0 con 0x56302970f3d0 2023-10-06T21:23:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 0x563029715ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f20fbfff700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 0x563029706aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f210099c700 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009500 tx=0x7f20f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ec7387977b7692 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f20e2ffd700 1 -- 172.21.15.31:0/1708424038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400e070 con 0x56302970f3d0 2023-10-06T21:23:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f20e2ffd700 1 -- 172.21.15.31:0/1708424038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f4022040 con 0x56302970f3d0 2023-10-06T21:23:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.857+0000 7f20e2ffd700 1 -- 172.21.15.31:0/1708424038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f401a470 con 0x56302970f3d0 2023-10-06T21:23:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 -- 172.21.15.31:0/1708424038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 msgr2=0x56302971b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009500 tx=0x7f20f4009920 comp rx=0 tx=0).stop 2023-10-06T21:23:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 -- 172.21.15.31:0/1708424038 shutdown_connections 2023-10-06T21:23:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x56302971b0f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029703c00 0x563029706aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 --2- 172.21.15.31:0/1708424038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029704520 0x563029715ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 -- 172.21.15.31:0/1708424038 >> 172.21.15.31:0/1708424038 conn(0x563029679200 msgr2=0x5630296fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 -- 172.21.15.31:0/1708424038 shutdown_connections 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.858+0000 7f2115c4db80 1 -- 172.21.15.31:0/1708424038 wait complete. 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 -- start start 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 Processor -- start 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302976c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630297673c0 con 0x56302970f3d0 2023-10-06T21:23:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029767530 con 0x563029704520 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630297676f0 con 0x563029703c00 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.859+0000 7f2115c4db80 10 client.0 _ll_register_callbacks cb 0x7f20f40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41334/0 (socket says 172.21.15.31:41334) 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20fbfff700 1 -- 172.21.15.31:0/215296358 learned_addr learned my addr 172.21.15.31:0/215296358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57986/0 (socket says 172.21.15.31:57986) 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20fb7fe700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302976c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 -- 172.21.15.31:0/215296358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 msgr2=0x56302976c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 -- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 msgr2=0x56302976c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302976c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 -- 172.21.15.31:0/215296358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f4008720 con 0x56302970f3d0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20fbfff700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f210099c700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009550 tx=0x7f20f4018fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc19567756b8151 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.860+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400e040 con 0x56302970f3d0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ec0a22e0 con 0x56302970f3d0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f20f4004cd0 con 0x56302970f3d0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f4004030 con 0x56302970f3d0 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 10 client.16443 fetch_fsmap learned FSMap version 31 2023-10-06T21:23:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563029481250 con 0x56302970f3d0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f403b080 con 0x56302970f3d0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f20f402d030 con 0x56302970f3d0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 10 client.16443 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 20 client.16443 populate_metadata read hostname 'smithi031' 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563029773ed0 0x5630297762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 4 client.16443 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.861+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563029650ad0 con 0x563029773ed0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.862+0000 7f20fbfff700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563029773ed0 0x5630297762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.862+0000 7f20fbfff700 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563029773ed0 0x5630297762f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d35be599226995 server_cookie=3f30e83bcafa9976 in_seq=0 out_seq=0 2023-10-06T21:23:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.862+0000 7f20db7fe700 10 client.16443 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f20db7fe700 1 -- 172.21.15.31:0/215296358 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563029650ad0 con 0x563029773ed0 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f20db7fe700 10 client.16443 handle_command_reply: tid=0 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 client.16443 shutdown 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 msgr2=0x563029766d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009550 tx=0x7f20f4018fc0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 shutdown_connections 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563029773ed0 0x5630297762f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56302970f3d0 0x563029766d20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563029703c00 0x56302976c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 --2- 172.21.15.31:0/215296358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563029704520 0x56302976c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f20db7fe700 0 client.16443 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 >> 172.21.15.31:0/215296358 conn(0x563029679200 msgr2=0x56302971b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 shutdown_connections 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.880+0000 7f2115c4db80 1 -- 172.21.15.31:0/215296358 wait complete. 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f2115c4db80 20 client.16443 trim_cache size 0 max 16384 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:23:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 msgr2=0x7f20e40a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 0x7f20e40a3c00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009fd0 tx=0x7f20f400b040 comp rx=0 tx=0).stop 2023-10-06T21:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 msgr2=0x7f20fc196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009f80 tx=0x7f20ec00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 shutdown_connections 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20fc19bb60 0x7f20fc19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc1010d0 0x7f20fc196fb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f20e40a1740 0x7f20e40a3c00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 --2- 172.21.15.31:0/1660141364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20fc1061f0 0x7f20fc1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 >> 172.21.15.31:0/1660141364 conn(0x7f20fc0fc950 msgr2=0x7f20fc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.881+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 shutdown_connections 2023-10-06T21:23:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:23:51.882+0000 7f21023ff700 1 -- 172.21.15.31:0/1660141364 wait complete. 2023-10-06T21:23:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:52 smithi146 ceph-mon[102809]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 749 KiB/s wr, 91 op/s 2023-10-06T21:23:52.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 34544 1.97 MB/sec warmup 102 sec latency 94.730 ms 2023-10-06T21:23:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:52 smithi142 ceph-mon[102951]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 749 KiB/s wr, 91 op/s 2023-10-06T21:23:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:52 smithi031 ceph-mon[98539]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 749 KiB/s wr, 91 op/s 2023-10-06T21:23:53.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 35138 2.00 MB/sec warmup 103 sec latency 16.405 ms 2023-10-06T21:23:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:54 smithi146 ceph-mon[102809]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:23:54.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 35338 2.00 MB/sec warmup 104 sec latency 19.561 ms 2023-10-06T21:23:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:54 smithi142 ceph-mon[102951]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:23:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:54 smithi031 ceph-mon[98539]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:23:55.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 35739 2.02 MB/sec warmup 105 sec latency 17.307 ms 2023-10-06T21:23:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:56 smithi146 ceph-mon[102809]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:56.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 35908 2.00 MB/sec warmup 106 sec latency 17.964 ms 2023-10-06T21:23:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:56 smithi142 ceph-mon[102951]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:56 smithi031 ceph-mon[98539]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:57.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 36097 1.98 MB/sec warmup 107 sec latency 15.277 ms 2023-10-06T21:23:58.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:58 smithi146 ceph-mon[102809]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:58.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 36432 1.97 MB/sec warmup 108 sec latency 20.263 ms 2023-10-06T21:23:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:58 smithi142 ceph-mon[102951]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:58 smithi031 ceph-mon[98539]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 812 KiB/s wr, 90 op/s 2023-10-06T21:23:59.332 INFO:tasks.workunit.client.0.smithi031.stdout: 1 36760 1.97 MB/sec warmup 109 sec latency 22.906 ms 2023-10-06T21:24:00.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:23:59 smithi142 ceph-mon[102951]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-06T21:24:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:23:59 smithi031 ceph-mon[98539]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-06T21:24:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:23:59 smithi146 ceph-mon[102809]: pgmap v95: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-06T21:24:00.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 37157 1.98 MB/sec warmup 110 sec latency 21.597 ms 2023-10-06T21:24:01.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 37572 1.96 MB/sec warmup 111 sec latency 17.831 ms 2023-10-06T21:24:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:02 smithi146 ceph-mon[102809]: pgmap v96: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 809 KiB/s wr, 89 op/s 2023-10-06T21:24:02.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 37938 1.96 MB/sec warmup 112 sec latency 18.323 ms 2023-10-06T21:24:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:02 smithi142 ceph-mon[102951]: pgmap v96: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 809 KiB/s wr, 89 op/s 2023-10-06T21:24:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:02 smithi031 ceph-mon[98539]: pgmap v96: 97 pgs: 97 active+clean; 41 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 809 KiB/s wr, 89 op/s 2023-10-06T21:24:03.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 38400 1.98 MB/sec warmup 113 sec latency 20.523 ms 2023-10-06T21:24:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:04 smithi146 ceph-mon[102809]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:24:04.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 38812 1.99 MB/sec warmup 114 sec latency 19.308 ms 2023-10-06T21:24:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:04 smithi142 ceph-mon[102951]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:24:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:04 smithi031 ceph-mon[98539]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 -- 172.21.15.31:0/256619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78681010d0 msgr2=0x7f78681014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 --2- 172.21.15.31:0/256619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78681010d0 0x7f78681014b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f7858009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 -- 172.21.15.31:0/256619 shutdown_connections 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 --2- 172.21.15.31:0/256619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f786810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 --2- 172.21.15.31:0/256619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78681010d0 0x7f78681014b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 --2- 172.21.15.31:0/256619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7868101a80 0x7f7868105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.045+0000 7f786cdff700 1 -- 172.21.15.31:0/256619 >> 172.21.15.31:0/256619 conn(0x7f78680fc930 msgr2=0x7f78680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.046+0000 7f786cdff700 1 -- 172.21.15.31:0/256619 shutdown_connections 2023-10-06T21:24:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.046+0000 7f786cdff700 1 -- 172.21.15.31:0/256619 wait complete. 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.046+0000 7f786cdff700 1 Processor -- start 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 -- start start 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 0x7f7868196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786819bb70 0x7f786819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819c5f0 con 0x7f78681061f0 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819c760 con 0x7f786819bb70 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819c920 con 0x7f78681010d0 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 0x7f7868196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50448/0 (socket says 172.21.15.31:50448) 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f786659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 0x7f7868196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:54574/0 (socket says 172.21.15.31:54574) 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.047+0000 7f7865d9b700 1 -- 172.21.15.31:0/367146107 learned_addr learned my addr 172.21.15.31:0/367146107 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7866d9d700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786819bb70 0x7f786819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 -- 172.21.15.31:0/367146107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 msgr2=0x7f7868196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 0x7f7868196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 -- 172.21.15.31:0/367146107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786819bb70 msgr2=0x7f786819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786819bb70 0x7f786819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 -- 172.21.15.31:0/367146107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580095f0 con 0x7f78681061f0 2023-10-06T21:24:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.048+0000 7f7865d9b700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e970 tx=0x7f785c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.049+0000 7f78577fe700 1 -- 172.21.15.31:0/367146107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c009790 con 0x7f78681061f0 2023-10-06T21:24:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.049+0000 7f78577fe700 1 -- 172.21.15.31:0/367146107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f785c00c7e0 con 0x7f78681061f0 2023-10-06T21:24:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.049+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78681a3e80 con 0x7f78681061f0 2023-10-06T21:24:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.049+0000 7f78577fe700 1 -- 172.21.15.31:0/367146107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c005440 con 0x7f78681061f0 2023-10-06T21:24:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.049+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78681a4300 con 0x7f78681061f0 2023-10-06T21:24:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.053+0000 7f78577fe700 1 -- 172.21.15.31:0/367146107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f785c00c950 con 0x7f78681061f0 2023-10-06T21:24:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.055+0000 7f788077bb80 1 Processor -- start 2023-10-06T21:24:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.055+0000 7f788077bb80 1 -- start start 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.055+0000 7f788077bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 0x56517c506fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.055+0000 7f788077bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c511e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.055+0000 7f788077bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c5147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f788077bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c470400 con 0x56517c507520 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f788077bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c461bc0 con 0x56517c506c00 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f788077bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c45f4e0 con 0x56517c5123d0 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f786659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c511e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f78577fe700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 0x7f78500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f7866d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 0x56517c506fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f7866d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 0x56517c506fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:48380/0 (socket says 172.21.15.31:48380) 2023-10-06T21:24:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f7866d9d700 1 -- 172.21.15.31:0/1885466059 learned_addr learned my addr 172.21.15.31:0/1885466059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f7865d9b700 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c5147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.056+0000 7f78577fe700 1 -- 172.21.15.31:0/367146107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f785c014070 con 0x7f78681061f0 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 -- 172.21.15.31:0/1885466059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 msgr2=0x56517c5147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c5147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f7866d9d700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 0x7f78500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 -- 172.21.15.31:0/1885466059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 msgr2=0x56517c506fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 0x56517c506fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 -- 172.21.15.31:0/1885466059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517c285d80 con 0x56517c507520 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f7865d9b700 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c5147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f786659c700 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c511e90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7858006010 tx=0x7f7858005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaf270c619f7cd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.057+0000 7f7866d9d700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 0x7f78500a3ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f786000fb80 tx=0x7f7860004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.058+0000 7f7854ff9700 1 -- 172.21.15.31:0/1885466059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785801d070 con 0x56517c507520 2023-10-06T21:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.058+0000 7f7854ff9700 1 -- 172.21.15.31:0/1885466059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f785801e040 con 0x56517c507520 2023-10-06T21:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.058+0000 7f7854ff9700 1 -- 172.21.15.31:0/1885466059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858022940 con 0x56517c507520 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.058+0000 7f788077bb80 1 -- 172.21.15.31:0/1885466059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 msgr2=0x56517c511e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.058+0000 7f788077bb80 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c511e90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7858006010 tx=0x7f7858005c90 comp rx=0 tx=0).stop 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 -- 172.21.15.31:0/1885466059 shutdown_connections 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c5147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c506c00 0x56517c506fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 --2- 172.21.15.31:0/1885466059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c511e90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 -- 172.21.15.31:0/1885466059 >> 172.21.15.31:0/1885466059 conn(0x56517c47c200 msgr2=0x56517c503080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 -- 172.21.15.31:0/1885466059 shutdown_connections 2023-10-06T21:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 -- 172.21.15.31:0/1885466059 wait complete. 2023-10-06T21:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.059+0000 7f788077bb80 1 -- start start 2023-10-06T21:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.060+0000 7f788077bb80 1 Processor -- start 2023-10-06T21:24:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.060+0000 7f788077bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c50f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c45add0 con 0x56517c506c00 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c3f76b0 con 0x56517c507520 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c3f7350 con 0x56517c5123d0 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f788077bb80 10 client.0 _ll_register_callbacks cb 0x56517c2547a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f786659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:48396/0 (socket says 172.21.15.31:48396) 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f786659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50452/0 (socket says 172.21.15.31:50452) 2023-10-06T21:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 -- 172.21.15.31:0/2572785549 learned_addr learned my addr 172.21.15.31:0/2572785549 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7866d9d700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c50f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 -- 172.21.15.31:0/2572785549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 msgr2=0x56517c50f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c50f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 -- 172.21.15.31:0/2572785549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 msgr2=0x56517c511da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f7865d9b700 1 -- 172.21.15.31:0/2572785549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786810e5e0 con 0x56517c507520 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.061+0000 7f786659c700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f7865d9b700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f785c00cc00 tx=0x7f785c00ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e097587c5e1543f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0cae70 con 0x56517c507520 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517c4fad50 con 0x56517c507520 2023-10-06T21:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f785c09e3d0 con 0x56517c507520 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c0a2610 con 0x56517c507520 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f788077bb80 10 client.26186 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56517c26ec00 con 0x56517c507520 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.062+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56517c4fad50 con 0x56517c507520 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f785c065020 con 0x56517c507520 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 10 client.26186 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 20 client.26186 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c521df0 0x56517c524210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 4 client.26186 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56517c453db0 con 0x56517c521df0 2023-10-06T21:24:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.063+0000 7f786659c700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c521df0 0x56517c524210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.064+0000 7f786659c700 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c521df0 0x56517c524210 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2995ef703ede3454 server_cookie=7fd8332016aa3447 in_seq=0 out_seq=0 2023-10-06T21:24:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.064+0000 7f784dffb700 10 client.26186 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f784dffb700 1 -- 172.21.15.31:0/2572785549 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56517c453db0 con 0x56517c521df0 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f784dffb700 10 client.26186 handle_command_reply: tid=0 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 client.26186 shutdown 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 msgr2=0x56517c50f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f785c00cc00 tx=0x7f785c00ccd0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 shutdown_connections 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c50f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c521df0 0x56517c524210 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c511da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f784dffb700 0 client.26186 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 --2- 172.21.15.31:0/2572785549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c50f020 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 >> 172.21.15.31:0/2572785549 conn(0x56517c47c200 msgr2=0x56517c502c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 shutdown_connections 2023-10-06T21:24:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 1 -- 172.21.15.31:0/2572785549 wait complete. 2023-10-06T21:24:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.065+0000 7f788077bb80 20 client.26186 trim_cache size 0 max 16384 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.081+0000 7f788077bb80 1 Processor -- start 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.081+0000 7f788077bb80 1 -- start start 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.081+0000 7f788077bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.081+0000 7f788077bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c509aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.081+0000 7f788077bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c518ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f788077bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c2db8c0 con 0x56517c506c00 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f788077bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c51f3d0 con 0x56517c507520 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f788077bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c507e20 con 0x56517c5123d0 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50456/0 (socket says 172.21.15.31:50456) 2023-10-06T21:24:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 -- 172.21.15.31:0/337667138 learned_addr learned my addr 172.21.15.31:0/337667138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f7866d9d700 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c518ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f7865d9b700 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c509aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 -- 172.21.15.31:0/337667138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 msgr2=0x56517c518ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c518ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 -- 172.21.15.31:0/337667138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 msgr2=0x56517c509aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c509aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 -- 172.21.15.31:0/337667138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517c2ac1f0 con 0x56517c506c00 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.082+0000 7f786659c700 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f785800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7eb178dd718afd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.083+0000 7f78557fa700 1 -- 172.21.15.31:0/337667138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785801d040 con 0x56517c506c00 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.083+0000 7f78557fa700 1 -- 172.21.15.31:0/337667138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f785800b710 con 0x56517c506c00 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.083+0000 7f78557fa700 1 -- 172.21.15.31:0/337667138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858022820 con 0x56517c506c00 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.083+0000 7f788077bb80 1 -- 172.21.15.31:0/337667138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 msgr2=0x56517c50efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.083+0000 7f788077bb80 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f785800bf50 comp rx=0 tx=0).stop 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 -- 172.21.15.31:0/337667138 shutdown_connections 2023-10-06T21:24:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c5123d0 0x56517c518ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c506c00 0x56517c50efe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 --2- 172.21.15.31:0/337667138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c507520 0x56517c509aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 -- 172.21.15.31:0/337667138 >> 172.21.15.31:0/337667138 conn(0x56517c47c200 msgr2=0x56517c528b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 -- 172.21.15.31:0/337667138 shutdown_connections 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 -- 172.21.15.31:0/337667138 wait complete. 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 -- start start 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.084+0000 7f788077bb80 1 Processor -- start 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c506c00 0x56517c5190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c5123d0 0x56517c5174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c517b70 con 0x56517c507520 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c517ce0 con 0x56517c5123d0 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517c574c20 con 0x56517c506c00 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f788077bb80 10 client.0 _ll_register_callbacks cb 0x7f785c099dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50468/0 (socket says 172.21.15.31:50468) 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 -- 172.21.15.31:0/197469942 learned_addr learned my addr 172.21.15.31:0/197469942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7866d9d700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c5123d0 0x56517c5174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f786659c700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c506c00 0x56517c5190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 -- 172.21.15.31:0/197469942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c506c00 msgr2=0x56517c5190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c506c00 0x56517c5190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 -- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c5123d0 msgr2=0x56517c5174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c5123d0 0x56517c5174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.085+0000 7f7865d9b700 1 -- 172.21.15.31:0/197469942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56517c285d80 con 0x56517c507520 2023-10-06T21:24:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f7865d9b700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c4e0 tx=0x7f785c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df7b3eaeaaba352 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c09a6e0 con 0x56517c507520 2023-10-06T21:24:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f785c09e3d0 con 0x56517c507520 2023-10-06T21:24:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517c453ad0 con 0x56517c507520 2023-10-06T21:24:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c016540 con 0x56517c507520 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f788077bb80 10 client.16461 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.086+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56517c284250 con 0x56517c507520 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56517c453ad0 con 0x56517c507520 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f785c01f060 con 0x56517c507520 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 20 client.16461 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c576040 0x56517c578460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 4 client.16461 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56517c2a6f10 con 0x56517c576040 2023-10-06T21:24:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.087+0000 7f786659c700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c576040 0x56517c578460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.088+0000 7f786659c700 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c576040 0x56517c578460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbda00125b115152 server_cookie=6f3d76bf1b89ca8f in_seq=0 out_seq=0 2023-10-06T21:24:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.088+0000 7f784dffb700 10 client.16461 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f784dffb700 1 -- 172.21.15.31:0/197469942 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56517c2a6f10 con 0x56517c576040 2023-10-06T21:24:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f784dffb700 10 client.16461 handle_command_reply: tid=0 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 client.16461 shutdown 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 msgr2=0x56517c516ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c4e0 tx=0x7f785c0092d0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 shutdown_connections 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56517c5123d0 0x56517c5174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56517c506c00 0x56517c5190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56517c576040 0x56517c578460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 --2- 172.21.15.31:0/197469942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56517c507520 0x56517c516ea0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f784dffb700 0 client.16461 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.110+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 >> 172.21.15.31:0/197469942 conn(0x56517c47c200 msgr2=0x56517c51e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 shutdown_connections 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f788077bb80 1 -- 172.21.15.31:0/197469942 wait complete. 2023-10-06T21:24:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f788077bb80 20 client.16461 trim_cache size 0 max 16384 2023-10-06T21:24:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:24:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:24:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:24:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 msgr2=0x7f78500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 0x7f78500a3ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f786000fb80 tx=0x7f7860004b40 comp rx=0 tx=0).stop 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 msgr2=0x7f7868197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.111+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e970 tx=0x7f785c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 shutdown_connections 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786819bb70 0x7f786819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78681010d0 0x7f7868196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f78500a15f0 0x7f78500a3ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 --2- 172.21.15.31:0/367146107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78681061f0 0x7f7868197500 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 >> 172.21.15.31:0/367146107 conn(0x7f78680fc930 msgr2=0x7f78680fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 shutdown_connections 2023-10-06T21:24:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:05.112+0000 7f786cdff700 1 -- 172.21.15.31:0/367146107 wait complete. 2023-10-06T21:24:05.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 39113 2.00 MB/sec warmup 115 sec latency 20.487 ms 2023-10-06T21:24:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:06 smithi146 ceph-mon[102809]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:06.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 39356 1.99 MB/sec warmup 116 sec latency 17.672 ms 2023-10-06T21:24:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:06 smithi142 ceph-mon[102951]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:06 smithi031 ceph-mon[98539]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:07.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 39527 1.98 MB/sec warmup 117 sec latency 19.395 ms 2023-10-06T21:24:08.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:08 smithi146 ceph-mon[102809]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:08.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 39788 1.96 MB/sec warmup 118 sec latency 19.013 ms 2023-10-06T21:24:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:08 smithi142 ceph-mon[102951]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:08 smithi031 ceph-mon[98539]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 744 KiB/s wr, 99 op/s 2023-10-06T21:24:09.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:09.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 40128 1.96 MB/sec warmup 119 sec latency 19.180 ms 2023-10-06T21:24:09.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:10 smithi146 ceph-mon[102809]: pgmap v100: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:24:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:10 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:10 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:10 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:24:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:10 smithi142 ceph-mon[102951]: pgmap v100: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:24:10.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:10 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:10.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:10 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:10.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:10 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:24:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:10 smithi031 ceph-mon[98539]: pgmap v100: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:24:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:10 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:10 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:10 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:24:11.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 40918 0.32 MB/sec execute 1 sec latency 18.834 ms 2023-10-06T21:24:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:12 smithi146 ceph-mon[102809]: pgmap v101: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 860 KiB/s wr, 101 op/s 2023-10-06T21:24:12.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 41364 0.87 MB/sec execute 2 sec latency 19.307 ms 2023-10-06T21:24:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:12 smithi142 ceph-mon[102951]: pgmap v101: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 860 KiB/s wr, 101 op/s 2023-10-06T21:24:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:12 smithi031 ceph-mon[98539]: pgmap v101: 97 pgs: 97 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 860 KiB/s wr, 101 op/s 2023-10-06T21:24:13.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 41643 0.97 MB/sec execute 3 sec latency 20.842 ms 2023-10-06T21:24:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:14 smithi146 ceph-mon[102809]: pgmap v102: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-10-06T21:24:14.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 42243 2.05 MB/sec execute 4 sec latency 19.592 ms 2023-10-06T21:24:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:14 smithi142 ceph-mon[102951]: pgmap v102: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-10-06T21:24:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:14 smithi031 ceph-mon[98539]: pgmap v102: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-10-06T21:24:15.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 42444 1.89 MB/sec execute 5 sec latency 20.558 ms 2023-10-06T21:24:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:16 smithi146 ceph-mon[102809]: pgmap v103: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:16.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 42837 2.29 MB/sec execute 6 sec latency 16.272 ms 2023-10-06T21:24:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:16 smithi142 ceph-mon[102951]: pgmap v103: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:16 smithi031 ceph-mon[98539]: pgmap v103: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:17.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 43008 2.00 MB/sec execute 7 sec latency 17.935 ms 2023-10-06T21:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 -- 172.21.15.31:0/2976644364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1010d0 msgr2=0x7fd27c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 --2- 172.21.15.31:0/2976644364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1010d0 0x7fd27c1014b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 -- 172.21.15.31:0/2976644364 shutdown_connections 2023-10-06T21:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 --2- 172.21.15.31:0/2976644364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1061f0 0x7fd27c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 --2- 172.21.15.31:0/2976644364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1010d0 0x7fd27c1014b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 --2- 172.21.15.31:0/2976644364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c101a80 0x7fd27c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 -- 172.21.15.31:0/2976644364 >> 172.21.15.31:0/2976644364 conn(0x7fd27c0fc930 msgr2=0x7fd27c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 -- 172.21.15.31:0/2976644364 shutdown_connections 2023-10-06T21:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.273+0000 7fd2837ff700 1 -- 172.21.15.31:0/2976644364 wait complete. 2023-10-06T21:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.274+0000 7fd2837ff700 1 Processor -- start 2023-10-06T21:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.274+0000 7fd2837ff700 1 -- start start 2023-10-06T21:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1010d0 0x7fd27c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c19dca0 0x7fd27c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19e720 con 0x7fd27c1061f0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19e890 con 0x7fd27c1010d0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd2837ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19ea50 con 0x7fd27c19dca0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34930/0 (socket says 172.21.15.31:34930) 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd28159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1010d0 0x7fd27c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c19dca0 0x7fd27c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 -- 172.21.15.31:0/400758752 learned_addr learned my addr 172.21.15.31:0/400758752 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 -- 172.21.15.31:0/400758752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c19dca0 msgr2=0x7fd27c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c19dca0 0x7fd27c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 -- 172.21.15.31:0/400758752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1010d0 msgr2=0x7fd27c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1010d0 0x7fd27c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.275+0000 7fd280d9a700 1 -- 172.21.15.31:0/400758752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c009520 con 0x7fd27c1061f0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd280d9a700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00ea30 tx=0x7fd26c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd2727fc700 1 -- 172.21.15.31:0/400758752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00cbd0 con 0x7fd27c1061f0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2780095f0 con 0x7fd27c1061f0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd2727fc700 1 -- 172.21.15.31:0/400758752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26c00eed0 con 0x7fd27c1061f0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd2727fc700 1 -- 172.21.15.31:0/400758752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c003c40 con 0x7fd27c1061f0 2023-10-06T21:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.276+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a62f0 con 0x7fd27c1061f0 2023-10-06T21:24:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.280+0000 7fd2727fc700 1 -- 172.21.15.31:0/400758752 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd26c003e00 con 0x7fd27c1061f0 2023-10-06T21:24:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 Processor -- start 2023-10-06T21:24:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 -- start start 2023-10-06T21:24:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 0x5589b1614e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b16177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1573400 con 0x5589b16153d0 2023-10-06T21:24:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1564bc0 con 0x5589b1609c00 2023-10-06T21:24:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.281+0000 7fd297162b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b15624e0 con 0x5589b160a520 2023-10-06T21:24:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b16177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52536/0 (socket says 172.21.15.31:52536) 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 -- 172.21.15.31:0/838674063 learned_addr learned my addr 172.21.15.31:0/838674063 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd28159b700 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 0x5589b1614e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 -- 172.21.15.31:0/838674063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 msgr2=0x5589b1614e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 0x5589b1614e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 -- 172.21.15.31:0/838674063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 msgr2=0x5589b16177c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b16177c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd2727fc700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 0x7fd2680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd281d9c700 1 -- 172.21.15.31:0/838674063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589b1388d80 con 0x5589b1609c00 2023-10-06T21:24:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd28159b700 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 0x5589b1614e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.282+0000 7fd2727fc700 1 -- 172.21.15.31:0/400758752 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd26c014070 con 0x7fd27c1061f0 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.283+0000 7fd281d9c700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 0x7fd2680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.283+0000 7fd281d9c700 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd27400a430 tx=0x7fd274009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d588f017cc751a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.283+0000 7fd2677fe700 1 -- 172.21.15.31:0/838674063 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274010040 con 0x5589b1609c00 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.283+0000 7fd2677fe700 1 -- 172.21.15.31:0/838674063 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd274014e70 con 0x5589b1609c00 2023-10-06T21:24:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.283+0000 7fd2677fe700 1 -- 172.21.15.31:0/838674063 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27401e3e0 con 0x5589b1609c00 2023-10-06T21:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd281d9c700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 0x7fd2680a3ab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd27401b000 tx=0x7fd27401e600 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 -- 172.21.15.31:0/838674063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 msgr2=0x5589b1609fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd27400a430 tx=0x7fd274009650 comp rx=0 tx=0).stop 2023-10-06T21:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 -- 172.21.15.31:0/838674063 shutdown_connections 2023-10-06T21:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b16177c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b1609fe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 --2- 172.21.15.31:0/838674063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b160a520 0x5589b1614e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 -- 172.21.15.31:0/838674063 >> 172.21.15.31:0/838674063 conn(0x5589b157f200 msgr2=0x5589b1606080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 -- 172.21.15.31:0/838674063 shutdown_connections 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.284+0000 7fd297162b80 1 -- 172.21.15.31:0/838674063 wait complete. 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.285+0000 7fd297162b80 1 -- start start 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.285+0000 7fd297162b80 1 Processor -- start 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1614d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 0x5589b1612620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b155ddd0 con 0x5589b160a520 2023-10-06T21:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b14fa6b0 con 0x5589b16153d0 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b14fa350 con 0x5589b1609c00 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd297162b80 10 client.0 _ll_register_callbacks cb 0x5589b13577a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 0x5589b1612620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34954/0 (socket says 172.21.15.31:34954) 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 -- 172.21.15.31:0/2219809561 learned_addr learned my addr 172.21.15.31:0/2219809561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd28159b700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1614d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 -- 172.21.15.31:0/2219809561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 msgr2=0x5589b1614d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1614d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 -- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 msgr2=0x5589b1612620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 0x5589b1612620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd280d9a700 1 -- 172.21.15.31:0/2219809561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c10e5e0 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.286+0000 7fd281d9c700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 0x5589b1612620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd280d9a700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009fd0 tx=0x7fd26c0d0ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cab00c1aad4e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c099e70 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589b15fdd50 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd26c09aa30 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26c099c90 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd297162b80 10 client.16473 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.287+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fd274009c60 con 0x5589b160a520 2023-10-06T21:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5589b15fdd50 con 0x5589b160a520 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd26c09e120 con 0x5589b160a520 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 10 client.16473 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 20 client.16473 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1624ea0 0x5589b16272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 4 client.16473 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589b1556db0 con 0x5589b1624ea0 2023-10-06T21:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.288+0000 7fd28159b700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1624ea0 0x5589b16272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.289+0000 7fd28159b700 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1624ea0 0x5589b16272c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b243f6a64f6669 server_cookie=5f43e52b47e6e964 in_seq=0 out_seq=0 2023-10-06T21:24:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.289+0000 7fd264ff9700 10 client.16473 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd264ff9700 1 -- 172.21.15.31:0/2219809561 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589b1556db0 con 0x5589b1624ea0 2023-10-06T21:24:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd264ff9700 10 client.16473 handle_command_reply: tid=0 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 client.16473 shutdown 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 msgr2=0x5589b1611ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009fd0 tx=0x7fd26c0d0ea0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 shutdown_connections 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b16153d0 0x5589b1612620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1614d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1624ea0 0x5589b16272c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 --2- 172.21.15.31:0/2219809561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b1611ff0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd264ff9700 0 client.16473 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 >> 172.21.15.31:0/2219809561 conn(0x5589b157f200 msgr2=0x5589b1605c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 shutdown_connections 2023-10-06T21:24:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 1 -- 172.21.15.31:0/2219809561 wait complete. 2023-10-06T21:24:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.290+0000 7fd297162b80 20 client.16473 trim_cache size 0 max 16384 2023-10-06T21:24:18.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:18 smithi146 ceph-mon[102809]: pgmap v104: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.307+0000 7fd297162b80 1 Processor -- start 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.307+0000 7fd297162b80 1 -- start start 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b160d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b16153d0 0x5589b160d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b13de8c0 con 0x5589b160a520 2023-10-06T21:24:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1606e30 con 0x5589b1609c00 2023-10-06T21:24:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd297162b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b160ae20 con 0x5589b16153d0 2023-10-06T21:24:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd280d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b160d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52550/0 (socket says 172.21.15.31:52550) 2023-10-06T21:24:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 -- 172.21.15.31:0/514025382 learned_addr learned my addr 172.21.15.31:0/514025382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd281d9c700 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b16153d0 0x5589b160d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 -- 172.21.15.31:0/514025382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b16153d0 msgr2=0x5589b160d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b16153d0 0x5589b160d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 -- 172.21.15.31:0/514025382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 msgr2=0x5589b160d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b160d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.308+0000 7fd28159b700 1 -- 172.21.15.31:0/514025382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589b1371c00 con 0x5589b1609c00 2023-10-06T21:24:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.309+0000 7fd280d9a700 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b160d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.309+0000 7fd28159b700 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10933b80a202474c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.309+0000 7fd267fff700 1 -- 172.21.15.31:0/514025382 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27801d070 con 0x5589b1609c00 2023-10-06T21:24:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.309+0000 7fd267fff700 1 -- 172.21.15.31:0/514025382 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27801e040 con 0x5589b1609c00 2023-10-06T21:24:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.309+0000 7fd267fff700 1 -- 172.21.15.31:0/514025382 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27802a070 con 0x5589b1609c00 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 -- 172.21.15.31:0/514025382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 msgr2=0x5589b161bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278005750 comp rx=0 tx=0).stop 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 -- 172.21.15.31:0/514025382 shutdown_connections 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b16153d0 0x5589b160d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b1609c00 0x5589b161bf10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 --2- 172.21.15.31:0/514025382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b160a520 0x5589b160d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 -- 172.21.15.31:0/514025382 >> 172.21.15.31:0/514025382 conn(0x5589b157f200 msgr2=0x5589b1621150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 -- 172.21.15.31:0/514025382 shutdown_connections 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.310+0000 7fd297162b80 1 -- 172.21.15.31:0/514025382 wait complete. 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.311+0000 7fd297162b80 1 -- start start 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.311+0000 7fd297162b80 1 Processor -- start 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.311+0000 7fd297162b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1670f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.311+0000 7fd297162b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b160a520 0x5589b1671450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd297162b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd297162b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b16184d0 con 0x5589b16153d0 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd297162b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1618640 con 0x5589b160a520 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd297162b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589b1618800 con 0x5589b1609c00 2023-10-06T21:24:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd297162b80 10 client.0 _ll_register_callbacks cb 0x7fd26c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd28159b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1670f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34972/0 (socket says 172.21.15.31:34972) 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 -- 172.21.15.31:0/1871154879 learned_addr learned my addr 172.21.15.31:0/1871154879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd280d9a700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b160a520 0x5589b1671450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 -- 172.21.15.31:0/1871154879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 msgr2=0x5589b1670f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1670f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 -- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b160a520 msgr2=0x5589b1671450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b160a520 0x5589b1671450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.312+0000 7fd281d9c700 1 -- 172.21.15.31:0/1871154879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd274009c60 con 0x5589b16153d0 2023-10-06T21:24:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd281d9c700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd27401e720 tx=0x7fd274022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180dcbb6033c23f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274010070 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd274014e70 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd274006e40 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd27401e3a0 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd297162b80 10 client.16479 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fd2780053c0 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.313+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27403f020 con 0x5589b16153d0 2023-10-06T21:24:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd2740208b0 con 0x5589b16153d0 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 20 client.16479 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1679ff0 0x5589b167c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 4 client.16479 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589b1556ad0 con 0x5589b1679ff0 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.314+0000 7fd28159b700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1679ff0 0x5589b167c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.315+0000 7fd28159b700 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1679ff0 0x5589b167c410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98596f02cdab07af server_cookie=a09f529ff55a817 in_seq=0 out_seq=0 2023-10-06T21:24:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.315+0000 7fd264ff9700 10 client.16479 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:18.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 43194 1.76 MB/sec execute 8 sec latency 19.782 ms 2023-10-06T21:24:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.334+0000 7fd264ff9700 1 -- 172.21.15.31:0/1871154879 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589b1556ad0 con 0x5589b1679ff0 2023-10-06T21:24:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.334+0000 7fd264ff9700 10 client.16479 handle_command_reply: tid=0 2023-10-06T21:24:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 client.16479 shutdown 2023-10-06T21:24:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 msgr2=0x5589b1617ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd27401e720 tx=0x7fd274022040 comp rx=0 tx=0).stop 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 shutdown_connections 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5589b1679ff0 0x5589b167c410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589b16153d0 0x5589b1617ef0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5589b1609c00 0x5589b1670f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd264ff9700 0 client.16479 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 --2- 172.21.15.31:0/1871154879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5589b160a520 0x5589b1671450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 >> 172.21.15.31:0/1871154879 conn(0x5589b157f200 msgr2=0x5589b160d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 shutdown_connections 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 1 -- 172.21.15.31:0/1871154879 wait complete. 2023-10-06T21:24:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.335+0000 7fd297162b80 20 client.16479 trim_cache size 0 max 16384 2023-10-06T21:24:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:24:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:24:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:24:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 msgr2=0x7fd2680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 0x7fd2680a3ab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd27401b000 tx=0x7fd27401e600 comp rx=0 tx=0).stop 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 msgr2=0x7fd27c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00ea30 tx=0x7fd26c00ed40 comp rx=0 tx=0).stop 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 shutdown_connections 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd27c19dca0 0x7fd27c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1010d0 0x7fd27c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2680a15f0 0x7fd2680a3ab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 --2- 172.21.15.31:0/400758752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd27c1061f0 0x7fd27c199630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 >> 172.21.15.31:0/400758752 conn(0x7fd27c0fc930 msgr2=0x7fd27c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 shutdown_connections 2023-10-06T21:24:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:18.336+0000 7fd2837ff700 1 -- 172.21.15.31:0/400758752 wait complete. 2023-10-06T21:24:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:18 smithi142 ceph-mon[102951]: pgmap v104: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:18.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:18 smithi031 ceph-mon[98539]: pgmap v104: 97 pgs: 97 active+clean; 43 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 811 KiB/s wr, 91 op/s 2023-10-06T21:24:19.333 INFO:tasks.workunit.client.0.smithi031.stdout: 1 43538 1.65 MB/sec execute 9 sec latency 18.435 ms 2023-10-06T21:24:20.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:19 smithi142 ceph-mon[102951]: pgmap v105: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:19 smithi031 ceph-mon[98539]: pgmap v105: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:19 smithi146 ceph-mon[102809]: pgmap v105: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:20.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 43878 1.62 MB/sec execute 10 sec latency 18.266 ms 2023-10-06T21:24:21.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 44286 1.75 MB/sec execute 11 sec latency 18.906 ms 2023-10-06T21:24:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:22 smithi146 ceph-mon[102809]: pgmap v106: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 845 KiB/s wr, 88 op/s 2023-10-06T21:24:22.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 44696 1.65 MB/sec execute 12 sec latency 20.354 ms 2023-10-06T21:24:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:22 smithi142 ceph-mon[102951]: pgmap v106: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 845 KiB/s wr, 88 op/s 2023-10-06T21:24:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:22 smithi031 ceph-mon[98539]: pgmap v106: 97 pgs: 97 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 845 KiB/s wr, 88 op/s 2023-10-06T21:24:23.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 45039 1.63 MB/sec execute 13 sec latency 19.878 ms 2023-10-06T21:24:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:24 smithi146 ceph-mon[102809]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:24:24.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 45521 1.90 MB/sec execute 14 sec latency 17.732 ms 2023-10-06T21:24:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:24 smithi142 ceph-mon[102951]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:24:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:24 smithi031 ceph-mon[98539]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:24:25.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 45914 1.90 MB/sec execute 15 sec latency 19.357 ms 2023-10-06T21:24:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:26 smithi146 ceph-mon[102809]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:26.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 46254 1.99 MB/sec execute 16 sec latency 15.804 ms 2023-10-06T21:24:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:26 smithi142 ceph-mon[102951]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:26 smithi031 ceph-mon[98539]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:27.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 46467 1.93 MB/sec execute 17 sec latency 17.999 ms 2023-10-06T21:24:28.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:28 smithi146 ceph-mon[102809]: pgmap v109: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:28.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 46646 1.84 MB/sec execute 18 sec latency 19.454 ms 2023-10-06T21:24:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:28 smithi142 ceph-mon[102951]: pgmap v109: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:28 smithi031 ceph-mon[98539]: pgmap v109: 97 pgs: 97 active+clean; 41 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 847 KiB/s wr, 91 op/s 2023-10-06T21:24:29.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 46928 1.77 MB/sec execute 19 sec latency 19.596 ms 2023-10-06T21:24:30.151 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:29 smithi142 ceph-mon[102951]: pgmap v110: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:29 smithi031 ceph-mon[98539]: pgmap v110: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:29 smithi146 ceph-mon[102809]: pgmap v110: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:24:30.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 47269 1.76 MB/sec execute 20 sec latency 19.297 ms 2023-10-06T21:24:31.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 47655 1.82 MB/sec execute 21 sec latency 18.625 ms 2023-10-06T21:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- 172.21.15.31:0/1121827695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181010d0 msgr2=0x7f77181014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 --2- 172.21.15.31:0/1121827695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181010d0 0x7f77181014b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7714009990 tx=0x7f7714009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- 172.21.15.31:0/1121827695 shutdown_connections 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 --2- 172.21.15.31:0/1121827695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181061f0 0x7f771810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 --2- 172.21.15.31:0/1121827695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181010d0 0x7f77181014b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 --2- 172.21.15.31:0/1121827695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7718101a80 0x7f7718105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- 172.21.15.31:0/1121827695 >> 172.21.15.31:0/1121827695 conn(0x7f77180fc950 msgr2=0x7f77180fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- 172.21.15.31:0/1121827695 shutdown_connections 2023-10-06T21:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- 172.21.15.31:0/1121827695 wait complete. 2023-10-06T21:24:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 Processor -- start 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.500+0000 7f771fbff700 1 -- start start 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 0x7f77180738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181a3120 0x7f77181a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181a5a50 con 0x7f77181061f0 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181a5b90 con 0x7f77181a3120 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181a5d00 con 0x7f77181010d0 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 0x7f77180738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:34024/0 (socket says 172.21.15.31:34024) 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771d99b700 1 -- 172.21.15.31:0/3518757641 learned_addr learned my addr 172.21.15.31:0/3518757641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.501+0000 7f771e19c700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181a3120 0x7f77181a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 -- 172.21.15.31:0/3518757641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 msgr2=0x7f7718073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 -- 172.21.15.31:0/3518757641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181a3120 msgr2=0x7f77181a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181a3120 0x7f77181a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 -- 172.21.15.31:0/3518757641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140095f0 con 0x7f77181061f0 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d99b700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771d19a700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 0x7f77180738d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f770800e9c0 tx=0x7f770800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f770effd700 1 -- 172.21.15.31:0/3518757641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800c9c0 con 0x7f77181061f0 2023-10-06T21:24:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f770effd700 1 -- 172.21.15.31:0/3518757641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7708010430 con 0x7f77181061f0 2023-10-06T21:24:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f770effd700 1 -- 172.21.15.31:0/3518757641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708018680 con 0x7f77181061f0 2023-10-06T21:24:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77181a5fe0 con 0x7f77181061f0 2023-10-06T21:24:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.502+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181a63b0 con 0x7f77181061f0 2023-10-06T21:24:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.506+0000 7f770effd700 1 -- 172.21.15.31:0/3518757641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77080105a0 con 0x7f77181061f0 2023-10-06T21:24:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.507+0000 7f770effd700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 0x7f77040a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.507+0000 7f7733598b80 1 Processor -- start 2023-10-06T21:24:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.507+0000 7f770effd700 1 -- 172.21.15.31:0/3518757641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f7708014070 con 0x7f77181061f0 2023-10-06T21:24:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.507+0000 7f7733598b80 1 -- start start 2023-10-06T21:24:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771d99b700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 0x7f77040a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89db5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89dc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89d1f400 con 0x55df89db5c00 2023-10-06T21:24:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89d10bc0 con 0x55df89dc13d0 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f7733598b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89d0e4e0 con 0x55df89db6520 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89db5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89db5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45622/0 (socket says 172.21.15.31:45622) 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771d99b700 1 -- 172.21.15.31:0/638130580 learned_addr learned my addr 172.21.15.31:0/638130580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771e19c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89dc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.508+0000 7f771d19a700 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 -- 172.21.15.31:0/638130580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 msgr2=0x55df89dc0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 -- 172.21.15.31:0/638130580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 msgr2=0x55df89db5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89db5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 -- 172.21.15.31:0/638130580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df89b34d80 con 0x55df89dc13d0 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771d99b700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 0x7f77040a3b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77140098e0 tx=0x7f771400bc90 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f771e19c700 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89dc37c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7710011580 tx=0x7f7710009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7b078e2e71a0d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f7703fff700 1 -- 172.21.15.31:0/638130580 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000e040 con 0x55df89dc13d0 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f7703fff700 1 -- 172.21.15.31:0/638130580 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771000f040 con 0x55df89dc13d0 2023-10-06T21:24:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.509+0000 7f7703fff700 1 -- 172.21.15.31:0/638130580 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710004290 con 0x55df89dc13d0 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 -- 172.21.15.31:0/638130580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 msgr2=0x55df89dc37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89dc37c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7710011580 tx=0x7f7710009920 comp rx=0 tx=0).stop 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 -- 172.21.15.31:0/638130580 shutdown_connections 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89dc37c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89db5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 --2- 172.21.15.31:0/638130580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 -- 172.21.15.31:0/638130580 >> 172.21.15.31:0/638130580 conn(0x55df89d2b200 msgr2=0x55df89db2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 -- 172.21.15.31:0/638130580 shutdown_connections 2023-10-06T21:24:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.510+0000 7f7733598b80 1 -- 172.21.15.31:0/638130580 wait complete. 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.511+0000 7f7733598b80 1 -- start start 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.511+0000 7f7733598b80 1 Processor -- start 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dbe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 0x55df89dbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89d09dd0 con 0x55df89db5c00 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89ca66b0 con 0x55df89db6520 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89ca6350 con 0x55df89dc13d0 2023-10-06T21:24:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f7733598b80 10 client.0 _ll_register_callbacks cb 0x55df89b037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45636/0 (socket says 172.21.15.31:45636) 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 -- 172.21.15.31:0/1557734856 learned_addr learned my addr 172.21.15.31:0/1557734856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771e19c700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 0x55df89dbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d19a700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dbe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 -- 172.21.15.31:0/1557734856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 msgr2=0x55df89dbe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 0x55df89dbe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 -- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 msgr2=0x55df89dbe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dbe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d99b700 1 -- 172.21.15.31:0/1557734856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7710011230 con 0x55df89db5c00 2023-10-06T21:24:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771e19c700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 0x55df89dbe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.512+0000 7f771d19a700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dbe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f771d99b700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7714005ad0 tx=0x7f7714009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed4d592d2d263f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f77017fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771401d040 con 0x55df89db5c00 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df89da9d50 con 0x55df89db5c00 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f7714027600 con 0x55df89db5c00 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f771400fe60 con 0x55df89db5c00 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f7733598b80 10 client.16491 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.513+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55df89b1dc00 con 0x55df89db5c00 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55df89da9d50 con 0x55df89db5c00 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7714030880 con 0x55df89db5c00 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 20 client.16491 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89dd0df0 0x55df89dd3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 4 client.16491 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df89d02db0 con 0x55df89dd0df0 2023-10-06T21:24:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.515+0000 7f771d19a700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89dd0df0 0x55df89dd3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.516+0000 7f77017fa700 10 client.16491 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.516+0000 7f771d19a700 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89dd0df0 0x55df89dd3210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c806df77566d586 server_cookie=6c20beae030bdf74 in_seq=0 out_seq=0 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f77017fa700 1 -- 172.21.15.31:0/1557734856 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df89d02db0 con 0x55df89dd0df0 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f77017fa700 10 client.16491 handle_command_reply: tid=0 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 client.16491 shutdown 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 msgr2=0x55df89dc0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7714005ad0 tx=0x7f7714009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 shutdown_connections 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89dc13d0 0x55df89dbe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89dd0df0 0x55df89dd3210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc0da0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 --2- 172.21.15.31:0/1557734856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dbe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 >> 172.21.15.31:0/1557734856 conn(0x55df89d2b200 msgr2=0x55df89db1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f77017fa700 0 client.16491 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 shutdown_connections 2023-10-06T21:24:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 1 -- 172.21.15.31:0/1557734856 wait complete. 2023-10-06T21:24:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.517+0000 7f7733598b80 20 client.16491 trim_cache size 0 max 16384 2023-10-06T21:24:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.537+0000 7f7733598b80 1 Processor -- start 2023-10-06T21:24:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.537+0000 7f7733598b80 1 -- start start 2023-10-06T21:24:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db5c00 0x55df89dd2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89dc13d0 0x55df89db9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89b8a8c0 con 0x55df89dc13d0 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89db2dc0 con 0x55df89db6520 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f7733598b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89db9e70 con 0x55df89db5c00 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f771d19a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f771e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89dc13d0 0x55df89db9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f771d99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db5c00 0x55df89dd2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f771d19a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58420/0 (socket says 172.21.15.31:58420) 2023-10-06T21:24:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.538+0000 7f771d19a700 1 -- 172.21.15.31:0/1015923916 learned_addr learned my addr 172.21.15.31:0/1015923916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 -- 172.21.15.31:0/1015923916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db5c00 msgr2=0x55df89dd2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db5c00 0x55df89dd2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 -- 172.21.15.31:0/1015923916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89dc13d0 msgr2=0x55df89db9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89dc13d0 0x55df89db9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 -- 172.21.15.31:0/1015923916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df89b5b1f0 con 0x55df89db6520 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f771d19a700 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7708099e20 tx=0x7f77080cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14166e1a48d3b96b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f770cbff700 1 -- 172.21.15.31:0/1015923916 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708099c80 con 0x55df89db6520 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f770cbff700 1 -- 172.21.15.31:0/1015923916 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f770809a120 con 0x55df89db6520 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.539+0000 7f770cbff700 1 -- 172.21.15.31:0/1015923916 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080b2be0 con 0x55df89db6520 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 -- 172.21.15.31:0/1015923916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 msgr2=0x55df89dcc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7708099e20 tx=0x7f77080cc040 comp rx=0 tx=0).stop 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 -- 172.21.15.31:0/1015923916 shutdown_connections 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89dc13d0 0x55df89db9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db5c00 0x55df89dd2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 --2- 172.21.15.31:0/1015923916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89db6520 0x55df89dcc670 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 -- 172.21.15.31:0/1015923916 >> 172.21.15.31:0/1015923916 conn(0x55df89d2b200 msgr2=0x55df89dcd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 -- 172.21.15.31:0/1015923916 shutdown_connections 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.540+0000 7f7733598b80 1 -- 172.21.15.31:0/1015923916 wait complete. 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 -- start start 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 Processor -- start 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89e23bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89dc4df0 con 0x55df89db5c00 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89e24110 con 0x55df89dc13d0 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df89e242d0 con 0x55df89db6520 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f7733598b80 10 client.0 _ll_register_callbacks cb 0x55df89b037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:34072/0 (socket says 172.21.15.31:34072) 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 -- 172.21.15.31:0/111563249 learned_addr learned my addr 172.21.15.31:0/111563249 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d99b700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771e19c700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89e23bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 -- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 msgr2=0x55df89e23bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89e23bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 -- 172.21.15.31:0/111563249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 msgr2=0x55df89dc40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 -- 172.21.15.31:0/111563249 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181a63b0 con 0x55df89db6520 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771e19c700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89e23bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f771d19a700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7708096620 tx=0x7f77080ca890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ea356faf0f2e4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.541+0000 7f77017fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080a24b0 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f77080b82b0 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f771000f040 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77080b2be0 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f7733598b80 10 client.26182 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55df89b33250 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771000f040 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.543+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f77080a2020 con 0x55df89db6520 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 20 client.26182 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89e256f0 0x55df89e27b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 4 client.26182 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df89d02ad0 con 0x55df89e256f0 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.544+0000 7f771d99b700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89e256f0 0x55df89e27b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.545+0000 7f771d99b700 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89e256f0 0x55df89e27b10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c011a1d00ae98ba1 server_cookie=2c54e5fbc23addf8 in_seq=0 out_seq=0 2023-10-06T21:24:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.545+0000 7f77017fa700 10 client.26182 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f77017fa700 1 -- 172.21.15.31:0/111563249 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df89d02ad0 con 0x55df89e256f0 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f77017fa700 10 client.26182 handle_command_reply: tid=0 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 client.26182 shutdown 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 msgr2=0x55df89dc4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7708096620 tx=0x7f77080ca890 comp rx=0 tx=0).stop 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 shutdown_connections 2023-10-06T21:24:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df89dc13d0 0x55df89e23bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55df89e256f0 0x55df89e27b10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f77017fa700 0 client.26182 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55df89db5c00 0x55df89dc40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.567+0000 7f7733598b80 1 --2- 172.21.15.31:0/111563249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55df89db6520 0x55df89dc4620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.568+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 >> 172.21.15.31:0/111563249 conn(0x55df89d2b200 msgr2=0x55df89db7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.568+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 shutdown_connections 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.568+0000 7f7733598b80 1 -- 172.21.15.31:0/111563249 wait complete. 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.568+0000 7f7733598b80 20 client.26182 trim_cache size 0 max 16384 2023-10-06T21:24:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:24:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:24:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:24:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:24:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 msgr2=0x7f77040a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 0x7f77040a3b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77140098e0 tx=0x7f771400bc90 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 msgr2=0x7f77180738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 0x7f77180738d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f770800e9c0 tx=0x7f770800ecd0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 shutdown_connections 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77181a3120 0x7f77181a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77181010d0 0x7f7718073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77040a1670 0x7f77040a3b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 --2- 172.21.15.31:0/3518757641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77181061f0 0x7f77180738d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 >> 172.21.15.31:0/3518757641 conn(0x7f77180fc950 msgr2=0x7f77180fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 shutdown_connections 2023-10-06T21:24:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:31.569+0000 7f771fbff700 1 -- 172.21.15.31:0/3518757641 wait complete. 2023-10-06T21:24:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:32 smithi146 ceph-mon[102809]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 863 KiB/s wr, 91 op/s 2023-10-06T21:24:32.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 48037 1.75 MB/sec execute 22 sec latency 17.431 ms 2023-10-06T21:24:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:32 smithi142 ceph-mon[102951]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 863 KiB/s wr, 91 op/s 2023-10-06T21:24:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:32 smithi031 ceph-mon[98539]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 863 KiB/s wr, 91 op/s 2023-10-06T21:24:33.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 48483 1.74 MB/sec execute 23 sec latency 16.172 ms 2023-10-06T21:24:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:34 smithi146 ceph-mon[102809]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-10-06T21:24:34.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 48754 1.72 MB/sec execute 24 sec latency 18.644 ms 2023-10-06T21:24:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:34 smithi142 ceph-mon[102951]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-10-06T21:24:34.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:34 smithi031 ceph-mon[98539]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-10-06T21:24:35.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 49408 1.90 MB/sec execute 25 sec latency 16.881 ms 2023-10-06T21:24:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:36 smithi146 ceph-mon[102809]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:36.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 49643 1.94 MB/sec execute 26 sec latency 19.957 ms 2023-10-06T21:24:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:36 smithi142 ceph-mon[102951]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:36 smithi031 ceph-mon[98539]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:37.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 49936 1.92 MB/sec execute 27 sec latency 18.310 ms 2023-10-06T21:24:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:38 smithi146 ceph-mon[102809]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:38.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 50108 1.87 MB/sec execute 28 sec latency 22.623 ms 2023-10-06T21:24:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:38 smithi142 ceph-mon[102951]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:38 smithi031 ceph-mon[98539]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 783 KiB/s wr, 91 op/s 2023-10-06T21:24:39.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 50302 1.80 MB/sec execute 29 sec latency 21.579 ms 2023-10-06T21:24:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:39 smithi031 ceph-mon[98539]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-10-06T21:24:40.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:39 smithi146 ceph-mon[102809]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-10-06T21:24:40.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 50555 1.76 MB/sec execute 30 sec latency 301.573 ms 2023-10-06T21:24:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:39 smithi142 ceph-mon[102951]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-10-06T21:24:41.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 50875 1.75 MB/sec execute 31 sec latency 19.165 ms 2023-10-06T21:24:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:42 smithi146 ceph-mon[102809]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 858 KiB/s wr, 90 op/s 2023-10-06T21:24:42.334 INFO:tasks.workunit.client.0.smithi031.stdout: 1 51285 1.80 MB/sec execute 32 sec latency 18.872 ms 2023-10-06T21:24:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:42 smithi142 ceph-mon[102951]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 858 KiB/s wr, 90 op/s 2023-10-06T21:24:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:42 smithi031 ceph-mon[98539]: pgmap v116: 97 pgs: 97 active+clean; 47 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 858 KiB/s wr, 90 op/s 2023-10-06T21:24:43.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 51692 1.75 MB/sec execute 33 sec latency 20.784 ms 2023-10-06T21:24:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:44 smithi146 ceph-mon[102809]: pgmap v117: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:24:44.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 52083 1.74 MB/sec execute 34 sec latency 17.271 ms 2023-10-06T21:24:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:44 smithi142 ceph-mon[102951]: pgmap v117: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:24:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:44 smithi031 ceph-mon[98539]: pgmap v117: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 -- 172.21.15.31:0/1378984372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c80682d0 msgr2=0x7f88c8068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/1378984372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c80682d0 0x7f88c8068750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009990 tx=0x7f88bc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 -- 172.21.15.31:0/1378984372 shutdown_connections 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/1378984372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8068c90 0x7f88c810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/1378984372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c8106f60 0x7f88c8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/1378984372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c80682d0 0x7f88c8068750 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.732+0000 7f88cd3ff700 1 -- 172.21.15.31:0/1378984372 >> 172.21.15.31:0/1378984372 conn(0x7f88c8076b70 msgr2=0x7f88c8076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.733+0000 7f88cd3ff700 1 -- 172.21.15.31:0/1378984372 shutdown_connections 2023-10-06T21:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.733+0000 7f88cd3ff700 1 -- 172.21.15.31:0/1378984372 wait complete. 2023-10-06T21:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 Processor -- start 2023-10-06T21:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 -- start start 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c80682d0 0x7f88c8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8106f60 0x7f88c819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c819e220 con 0x7f88c8068c90 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c819e390 con 0x7f88c80682d0 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88cd3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c819e550 con 0x7f88c8106f60 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.734+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60830/0 (socket says 172.21.15.31:60830) 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 -- 172.21.15.31:0/3805830321 learned_addr learned my addr 172.21.15.31:0/3805830321 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c80682d0 0x7f88c8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8106f60 0x7f88c819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 -- 172.21.15.31:0/3805830321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8106f60 msgr2=0x7f88c819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8106f60 0x7f88c819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 -- 172.21.15.31:0/3805830321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c80682d0 msgr2=0x7f88c8199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c80682d0 0x7f88c8199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 -- 172.21.15.31:0/3805830321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0095f0 con 0x7f88c8068c90 2023-10-06T21:24:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.735+0000 7f88c67fc700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000c00 tx=0x7f88bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.736+0000 7f88affff700 1 -- 172.21.15.31:0/3805830321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc01d070 con 0x7f88c8068c90 2023-10-06T21:24:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.736+0000 7f88affff700 1 -- 172.21.15.31:0/3805830321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88bc022470 con 0x7f88c8068c90 2023-10-06T21:24:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.736+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c819e7d0 con 0x7f88c8068c90 2023-10-06T21:24:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.736+0000 7f88affff700 1 -- 172.21.15.31:0/3805830321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc017650 con 0x7f88c8068c90 2023-10-06T21:24:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.736+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c81a6070 con 0x7f88c8068c90 2023-10-06T21:24:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.740+0000 7f88affff700 1 -- 172.21.15.31:0/3805830321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f88bc00fac0 con 0x7f88c8068c90 2023-10-06T21:24:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.741+0000 7f88affff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 0x7f88b00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.741+0000 7f88e0ca6b80 1 Processor -- start 2023-10-06T21:24:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.741+0000 7f88affff700 1 -- 172.21.15.31:0/3805830321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f88bc0ce4c0 con 0x7f88c8068c90 2023-10-06T21:24:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 0x7f88b00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 -- start start 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 0x56396e6bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e6ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e626400 con 0x56396e6bd520 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e617bc0 con 0x56396e6bcc00 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.742+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e6154e0 con 0x56396e6c83d0 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 0x7f88b00a3b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c60 tx=0x7f88b8009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60834/0 (socket says 172.21.15.31:60834) 2023-10-06T21:24:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 -- 172.21.15.31:0/1528349245 learned_addr learned my addr 172.21.15.31:0/1528349245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c77fe700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e6ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 0x56396e6bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 -- 172.21.15.31:0/1528349245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 msgr2=0x56396e6ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e6ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 -- 172.21.15.31:0/1528349245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 msgr2=0x56396e6bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 0x56396e6bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 -- 172.21.15.31:0/1528349245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396e43bd80 con 0x56396e6bd520 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 0x56396e6bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0979e0 tx=0x7f88bc0978f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1584020ec57363c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.743+0000 7f88ad7fa700 1 -- 172.21.15.31:0/1528349245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc09ae40 con 0x56396e6bd520 2023-10-06T21:24:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.744+0000 7f88ad7fa700 1 -- 172.21.15.31:0/1528349245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88bc0225e0 con 0x56396e6bd520 2023-10-06T21:24:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.744+0000 7f88ad7fa700 1 -- 172.21.15.31:0/1528349245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0a37e0 con 0x56396e6bd520 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.744+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1528349245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 msgr2=0x56396e6c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.744+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0979e0 tx=0x7f88bc0978f0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1528349245 shutdown_connections 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e6ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bcc00 0x56396e6bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1528349245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c7e90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1528349245 >> 172.21.15.31:0/1528349245 conn(0x56396e632200 msgr2=0x56396e6b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1528349245 shutdown_connections 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1528349245 wait complete. 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 -- start start 2023-10-06T21:24:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.745+0000 7f88e0ca6b80 1 Processor -- start 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bd520 0x56396e6c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e610dd0 con 0x56396e6c83d0 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e5ad6b0 con 0x56396e6bd520 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e5ad350 con 0x56396e6bcc00 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88e0ca6b80 10 client.0 _ll_register_callbacks cb 0x56396e40a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60836/0 (socket says 172.21.15.31:60836) 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bd520 0x56396e6c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.746+0000 7f88c77fe700 1 -- 172.21.15.31:0/2397206514 learned_addr learned my addr 172.21.15.31:0/2397206514 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 -- 172.21.15.31:0/2397206514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 msgr2=0x56396e6c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 -- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bd520 msgr2=0x56396e6c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bd520 0x56396e6c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 -- 172.21.15.31:0/2397206514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c81a6070 con 0x56396e6c83d0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011f10 tx=0x7f88c0014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be4e1ce8fb745a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000e040 con 0x56396e6c83d0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56396e6b0d50 con 0x56396e6c83d0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.747+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f88c0004db0 con 0x56396e6c83d0 2023-10-06T21:24:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c00090d0 con 0x56396e6c83d0 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 10 client.16509 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56396e424c00 con 0x56396e6c83d0 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56396e6b0d50 con 0x56396e6c83d0 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f88c001c030 con 0x56396e6c83d0 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 10 client.16509 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 20 client.16509 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e6d7c00 0x56396e6da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 4 client.16509 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.748+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56396e609db0 con 0x56396e6d7c00 2023-10-06T21:24:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.749+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e6d7c00 0x56396e6da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.749+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e6d7c00 0x56396e6da020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18dc4e35ebf1e2c server_cookie=177b62c36d34f0c0 in_seq=0 out_seq=0 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.749+0000 7f88aaff5700 10 client.16509 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88aaff5700 1 -- 172.21.15.31:0/2397206514 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56396e609db0 con 0x56396e6d7c00 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88aaff5700 10 client.16509 handle_command_reply: tid=0 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 client.16509 shutdown 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 msgr2=0x56396e6c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011f10 tx=0x7f88c0014600 comp rx=0 tx=0).stop 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 shutdown_connections 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e6d7c00 0x56396e6da020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6c83d0 0x56396e6c9230 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2397206514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6bd520 0x56396e6c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88aaff5700 0 client.16509 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.750+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 >> 172.21.15.31:0/2397206514 conn(0x56396e632200 msgr2=0x56396e6b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.751+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 shutdown_connections 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.751+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2397206514 wait complete. 2023-10-06T21:24:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.751+0000 7f88e0ca6b80 20 client.16509 trim_cache size 0 max 16384 2023-10-06T21:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 Processor -- start 2023-10-06T21:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 -- start start 2023-10-06T21:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6c83d0 0x56396e6c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e4918c0 con 0x56396e6bd520 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e6c3350 con 0x56396e6c83d0 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e6bcfe0 con 0x56396e6bcc00 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60850/0 (socket says 172.21.15.31:60850) 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88c67fc700 1 -- 172.21.15.31:0/2243954197 learned_addr learned my addr 172.21.15.31:0/2243954197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.770+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6c83d0 0x56396e6c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 -- 172.21.15.31:0/2243954197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 msgr2=0x56396e6cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 -- 172.21.15.31:0/2243954197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6c83d0 msgr2=0x56396e6c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6c83d0 0x56396e6c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 -- 172.21.15.31:0/2243954197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396e4621f0 con 0x56396e6bd520 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00f740 tx=0x7f88bc0ce450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5221bc1504ab8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88adffb700 1 -- 172.21.15.31:0/2243954197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0092b0 con 0x56396e6bd520 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88adffb700 1 -- 172.21.15.31:0/2243954197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88bc09ae40 con 0x56396e6bd520 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.771+0000 7f88adffb700 1 -- 172.21.15.31:0/2243954197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc09b570 con 0x56396e6bd520 2023-10-06T21:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2243954197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 msgr2=0x56396e6c1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00f740 tx=0x7f88bc0ce450 comp rx=0 tx=0).stop 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2243954197 shutdown_connections 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e6c83d0 0x56396e6c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6bcc00 0x56396e6cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/2243954197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bd520 0x56396e6c1c80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2243954197 >> 172.21.15.31:0/2243954197 conn(0x56396e632200 msgr2=0x56396e6d3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2243954197 shutdown_connections 2023-10-06T21:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.772+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/2243954197 wait complete. 2023-10-06T21:24:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.773+0000 7f88e0ca6b80 1 -- start start 2023-10-06T21:24:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.773+0000 7f88e0ca6b80 1 Processor -- start 2023-10-06T21:24:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.773+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e7200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e728a30 0x56396e72ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e72b360 con 0x56396e6bcc00 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e72b4a0 con 0x56396e728a30 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396e72b5e0 con 0x56396e6c83d0 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88e0ca6b80 10 client.0 _ll_register_callbacks cb 0x7f88c0011840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60866/0 (socket says 172.21.15.31:60866) 2023-10-06T21:24:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 -- 172.21.15.31:0/1033906293 learned_addr learned my addr 172.21.15.31:0/1033906293 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c77fe700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e728a30 0x56396e72ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e7200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 -- 172.21.15.31:0/1033906293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 msgr2=0x56396e7200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e7200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 -- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e728a30 msgr2=0x56396e72ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e728a30 0x56396e72ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.774+0000 7f88c6ffd700 1 -- 172.21.15.31:0/1033906293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56396e43bd80 con 0x56396e6bcc00 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f88b80094f0 tx=0x7f88b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbe1c7a978998e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8023040 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88bc0225e0 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f88b8004830 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b8004570 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88e0ca6b80 10 client.16515 fetch_fsmap learned FSMap version 31 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.775+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56396e43a250 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88bc0225e0 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f88b8003680 con 0x56396e6bcc00 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 20 client.16515 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e72cdc0 0x56396e72f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 4 client.16515 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.776+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56396e609ad0 con 0x56396e72cdc0 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.777+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e72cdc0 0x56396e72f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.777+0000 7f88c67fc700 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e72cdc0 0x56396e72f1e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce581316b73d6c26 server_cookie=b9cca44e6d3e75f7 in_seq=0 out_seq=0 2023-10-06T21:24:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.777+0000 7f88aaff5700 10 client.16515 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88aaff5700 1 -- 172.21.15.31:0/1033906293 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56396e609ad0 con 0x56396e72cdc0 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88aaff5700 10 client.16515 handle_command_reply: tid=0 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 client.16515 shutdown 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 msgr2=0x56396e71fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f88b80094f0 tx=0x7f88b80047c0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 shutdown_connections 2023-10-06T21:24:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56396e728a30 0x56396e72ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56396e72cdc0 0x56396e72f1e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56396e6bcc00 0x56396e71fbb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 --2- 172.21.15.31:0/1033906293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56396e6c83d0 0x56396e7200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88aaff5700 0 client.16515 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.798+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 >> 172.21.15.31:0/1033906293 conn(0x56396e632200 msgr2=0x56396e6c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 shutdown_connections 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88e0ca6b80 1 -- 172.21.15.31:0/1033906293 wait complete. 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88e0ca6b80 20 client.16515 trim_cache size 0 max 16384 2023-10-06T21:24:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:24:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:24:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:24:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:24:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 msgr2=0x7f88b00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 0x7f88b00a3b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c60 tx=0x7f88b8009260 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 msgr2=0x7f88c8199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.799+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000c00 tx=0x7f88bc004730 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 shutdown_connections 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88c8106f60 0x7f88c819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c80682d0 0x7f88c8199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f88b00a1670 0x7f88b00a3b30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 --2- 172.21.15.31:0/3805830321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8068c90 0x7f88c8199570 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 >> 172.21.15.31:0/3805830321 conn(0x7f88c8076b70 msgr2=0x7f88c80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 shutdown_connections 2023-10-06T21:24:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:44.800+0000 7f88cd3ff700 1 -- 172.21.15.31:0/3805830321 wait complete. 2023-10-06T21:24:45.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 52463 1.81 MB/sec execute 35 sec latency 19.957 ms 2023-10-06T21:24:46.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:46 smithi146 ceph-mon[102809]: pgmap v118: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:46.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 52981 1.85 MB/sec execute 36 sec latency 20.070 ms 2023-10-06T21:24:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:46 smithi142 ceph-mon[102951]: pgmap v118: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:46 smithi031 ceph-mon[98539]: pgmap v118: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:47.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 53253 1.89 MB/sec execute 37 sec latency 18.613 ms 2023-10-06T21:24:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:48 smithi146 ceph-mon[102809]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:48.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 53523 1.87 MB/sec execute 38 sec latency 19.501 ms 2023-10-06T21:24:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:48 smithi142 ceph-mon[102951]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:48 smithi031 ceph-mon[98539]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 86 op/s 2023-10-06T21:24:49.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 53695 1.83 MB/sec execute 39 sec latency 20.463 ms 2023-10-06T21:24:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:49 smithi031 ceph-mon[98539]: pgmap v120: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:24:50.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:49 smithi146 ceph-mon[102809]: pgmap v120: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:24:50.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 53929 1.79 MB/sec execute 40 sec latency 23.723 ms 2023-10-06T21:24:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:49 smithi142 ceph-mon[102951]: pgmap v120: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:24:51.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 54292 1.79 MB/sec execute 41 sec latency 18.862 ms 2023-10-06T21:24:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:52 smithi146 ceph-mon[102809]: pgmap v121: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 828 KiB/s wr, 87 op/s 2023-10-06T21:24:52.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 54686 1.82 MB/sec execute 42 sec latency 21.698 ms 2023-10-06T21:24:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:52 smithi142 ceph-mon[102951]: pgmap v121: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 828 KiB/s wr, 87 op/s 2023-10-06T21:24:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:52 smithi031 ceph-mon[98539]: pgmap v121: 97 pgs: 97 active+clean; 49 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 828 KiB/s wr, 87 op/s 2023-10-06T21:24:53.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 55065 1.79 MB/sec execute 43 sec latency 17.967 ms 2023-10-06T21:24:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:54 smithi146 ceph-mon[102809]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-10-06T21:24:54.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 55480 1.76 MB/sec execute 44 sec latency 21.629 ms 2023-10-06T21:24:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:54 smithi142 ceph-mon[102951]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-10-06T21:24:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:54 smithi031 ceph-mon[98539]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-10-06T21:24:55.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:55 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:24:55.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 55811 1.76 MB/sec execute 45 sec latency 18.665 ms 2023-10-06T21:24:55.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:55 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:24:55.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:55 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:24:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:56 smithi146 ceph-mon[102809]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:56.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 56405 1.84 MB/sec execute 46 sec latency 18.855 ms 2023-10-06T21:24:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:56 smithi142 ceph-mon[102951]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:56 smithi031 ceph-mon[98539]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:57.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 56610 1.83 MB/sec execute 47 sec latency 19.732 ms 2023-10-06T21:24:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1562479172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce0101a80 msgr2=0x7f0ce0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1562479172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce0101a80 0x7f0ce0105ac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009990 tx=0x7f0cd4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1562479172 shutdown_connections 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1562479172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01061f0 0x7f0ce010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1562479172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 0x7f0ce01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1562479172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce0101a80 0x7f0ce0105ac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1562479172 >> 172.21.15.31:0/1562479172 conn(0x7f0ce00fc950 msgr2=0x7f0ce00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1562479172 shutdown_connections 2023-10-06T21:24:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.973+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1562479172 wait complete. 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.974+0000 7f0ce57ff700 1 Processor -- start 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.974+0000 7f0ce57ff700 1 -- start start 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 0x7f0ce0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce01061f0 0x7f0ce00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01a59e0 con 0x7f0ce01a30b0 2023-10-06T21:24:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01a5b20 con 0x7f0ce01010d0 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0ce57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01a5c90 con 0x7f0ce01061f0 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49896/0 (socket says 172.21.15.31:49896) 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/1296251687 learned_addr learned my addr 172.21.15.31:0/1296251687 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce01061f0 0x7f0ce00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 0x7f0ce0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/1296251687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce01061f0 msgr2=0x7f0ce00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce01061f0 0x7f0ce00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/1296251687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 msgr2=0x7f0ce0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 0x7f0ce0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.975+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/1296251687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40095f0 con 0x7f0ce01a30b0 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800ef30 tx=0x7f0cd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0cc7fff700 1 -- 172.21.15.31:0/1296251687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8018070 con 0x7f0ce01a30b0 2023-10-06T21:24:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0cc7fff700 1 -- 172.21.15.31:0/1296251687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cd800f040 con 0x7f0ce01a30b0 2023-10-06T21:24:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0cc7fff700 1 -- 172.21.15.31:0/1296251687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800cbc0 con 0x7f0ce01a30b0 2023-10-06T21:24:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01a5f10 con 0x7f0ce01a30b0 2023-10-06T21:24:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.976+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce01a62e0 con 0x7f0ce01a30b0 2023-10-06T21:24:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.980+0000 7f0cc7fff700 1 -- 172.21.15.31:0/1296251687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0cd8007500 con 0x7f0ce01a30b0 2023-10-06T21:24:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.982+0000 7f0cc7fff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 0x7f0cc80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.982+0000 7f0cc7fff700 1 -- 172.21.15.31:0/1296251687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0cd80ccbb0 con 0x7f0ce01a30b0 2023-10-06T21:24:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.982+0000 7f0cf91b0b80 1 Processor -- start 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.982+0000 7f0cf91b0b80 1 -- start start 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.982+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 0x7f0cc80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x561758081e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 0x5617580847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757fe0400 con 0x561758077520 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757fd1bc0 con 0x561758076c00 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757fcf4e0 con 0x5617580823d0 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x561758081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49020/0 (socket says 172.21.15.31:49020) 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3147819819 learned_addr learned my addr 172.21.15.31:0/3147819819 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 0x5617580847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 0x7f0cc80a3ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009cb0 tx=0x7f0cd0009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3147819819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 msgr2=0x5617580847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 0x5617580847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3147819819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 msgr2=0x561758081e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x561758081e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3147819819 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561757df5d80 con 0x561758076c00 2023-10-06T21:24:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.983+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 0x5617580847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.984+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x561758081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.984+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff174e5b29d950bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.984+0000 7f0cc57fa700 1 -- 172.21.15.31:0/3147819819 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001ce30 con 0x561758076c00 2023-10-06T21:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.984+0000 7f0cc57fa700 1 -- 172.21.15.31:0/3147819819 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cd0004cc0 con 0x561758076c00 2023-10-06T21:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.984+0000 7f0cc57fa700 1 -- 172.21.15.31:0/3147819819 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0024420 con 0x561758076c00 2023-10-06T21:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3147819819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 msgr2=0x561758076fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd001c000 comp rx=0 tx=0).stop 2023-10-06T21:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3147819819 shutdown_connections 2023-10-06T21:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617580823d0 0x5617580847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561758076c00 0x561758076fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3147819819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x561758081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3147819819 >> 172.21.15.31:0/3147819819 conn(0x561757fec200 msgr2=0x561758073080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3147819819 shutdown_connections 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.985+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3147819819 wait complete. 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.986+0000 7f0cf91b0b80 1 -- start start 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.986+0000 7f0cf91b0b80 1 Processor -- start 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x561758081de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757fcadd0 con 0x561758076c00 2023-10-06T21:24:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757f676b0 con 0x5617580823d0 2023-10-06T21:24:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757f67350 con 0x561758077520 2023-10-06T21:24:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.987+0000 7f0cf91b0b80 10 client.0 _ll_register_callbacks cb 0x561757dc47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x561758081de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49028/0 (socket says 172.21.15.31:49028) 2023-10-06T21:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2701897185 learned_addr learned my addr 172.21.15.31:0/2701897185 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2701897185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 msgr2=0x56175807f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2701897185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 msgr2=0x561758081de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x561758081de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2701897185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce010e5e0 con 0x5617580823d0 2023-10-06T21:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80ca2b0 tx=0x7f0cd8012400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e5605be29d04077 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.988+0000 7f0cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd809aea0 con 0x5617580823d0 2023-10-06T21:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0cd8099b70 con 0x5617580823d0 2023-10-06T21:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56175806ad50 con 0x5617580823d0 2023-10-06T21:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd809dcc0 con 0x5617580823d0 2023-10-06T21:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 10 client.26255 fetch_fsmap learned FSMap version 32 2023-10-06T21:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0ce01a62e0 con 0x5617580823d0 2023-10-06T21:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56175806ad50 con 0x5617580823d0 2023-10-06T21:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cd8099060 con 0x5617580823d0 2023-10-06T21:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T21:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 20 client.26255 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561758091e80 0x5617580942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 4 client.26255 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.989+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561757fc3db0 con 0x561758091e80 2023-10-06T21:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.990+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561758091e80 0x5617580942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.991+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561758091e80 0x5617580942a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad249843c6818e8 server_cookie=b03c50a0ceb47381 in_seq=0 out_seq=0 2023-10-06T21:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.991+0000 7f0cc2ff5700 10 client.26255 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/2701897185 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561757fc3db0 con 0x561758091e80 2023-10-06T21:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cc2ff5700 10 client.26255 handle_command_reply: tid=0 2023-10-06T21:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 client.26255 shutdown 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 msgr2=0x56175807f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80ca2b0 tx=0x7f0cd8012400 comp rx=0 tx=0).stop 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 shutdown_connections 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807f690 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561758091e80 0x5617580942a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x561758081de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/2701897185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cc2ff5700 0 client.26255 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 >> 172.21.15.31:0/2701897185 conn(0x561757fec200 msgr2=0x561758072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.992+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 shutdown_connections 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.993+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/2701897185 wait complete. 2023-10-06T21:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:57.993+0000 7f0cf91b0b80 20 client.26255 trim_cache size 0 max 16384 2023-10-06T21:24:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 Processor -- start 2023-10-06T21:24:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 -- start start 2023-10-06T21:24:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x561758088f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561757e4b8c0 con 0x561758076c00 2023-10-06T21:24:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175807eeb0 con 0x5617580823d0 2023-10-06T21:24:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561758089500 con 0x561758077520 2023-10-06T21:24:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49924/0 (socket says 172.21.15.31:49924) 2023-10-06T21:24:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.012+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3062021762 learned_addr learned my addr 172.21.15.31:0/3062021762 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x561758088f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3062021762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 msgr2=0x56175807e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3062021762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 msgr2=0x561758088f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x561758088f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 -- 172.21.15.31:0/3062021762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0017050 con 0x561758076c00 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805dd8ef2125514d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.013+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3062021762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0005d60 con 0x561758076c00 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3062021762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cd00092d0 con 0x561758076c00 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3062021762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0003c30 con 0x561758076c00 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3062021762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 msgr2=0x56175807f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd0009480 comp rx=0 tx=0).stop 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3062021762 shutdown_connections 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x561758088f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758076c00 0x56175807f610 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/3062021762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758077520 0x56175807e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.014+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3062021762 >> 172.21.15.31:0/3062021762 conn(0x561757fec200 msgr2=0x561758098bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.015+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3062021762 shutdown_connections 2023-10-06T21:24:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.015+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/3062021762 wait complete. 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.015+0000 7f0cf91b0b80 1 -- start start 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.015+0000 7f0cf91b0b80 1 Processor -- start 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 0x56175807c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175807b820 con 0x561758077520 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175807b990 con 0x5617580823d0 2023-10-06T21:24:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175807bb50 con 0x561758076c00 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cf91b0b80 10 client.0 _ll_register_callbacks cb 0x7f0cd80994d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 0x56175807c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49938/0 (socket says 172.21.15.31:49938) 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 -- 172.21.15.31:0/1000166543 learned_addr learned my addr 172.21.15.31:0/1000166543 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:24:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 -- 172.21.15.31:0/1000166543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 msgr2=0x56175807c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 0x56175807c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 -- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 msgr2=0x56175807b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cde7fc700 1 -- 172.21.15.31:0/1000166543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561757df5d80 con 0x561758077520 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.016+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 0x56175807c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cde7fc700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006a480 tx=0x7f0cd400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec58c9d7c11ac6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd401d030 con 0x561758077520 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd0004cc0 con 0x561758077520 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f0cd4004700 con 0x561758077520 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.017+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd40048c0 con 0x561758077520 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 10 client.16533 fetch_fsmap learned FSMap version 32 2023-10-06T21:24:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561757ddec00 con 0x561758077520 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cd403a080 con 0x561758077520 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cd4034030 con 0x561758077520 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 20 client.16533 populate_metadata read hostname 'smithi031' 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617580e6c20 0x5617580e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 4 client.16533 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.018+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561757fc3ad0 con 0x5617580e6c20 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.019+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617580e6c20 0x5617580e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.019+0000 7f0cdeffd700 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617580e6c20 0x5617580e9040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cf06c77c515d73 server_cookie=b46c588b98a3c115 in_seq=0 out_seq=0 2023-10-06T21:24:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.019+0000 7f0cc2ff5700 10 client.16533 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cc2ff5700 1 -- 172.21.15.31:0/1000166543 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561757fc3ad0 con 0x5617580e6c20 2023-10-06T21:24:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cc2ff5700 10 client.16533 handle_command_reply: tid=0 2023-10-06T21:24:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 client.16533 shutdown 2023-10-06T21:24:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 msgr2=0x56175807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006a480 tx=0x7f0cd400b8e0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 shutdown_connections 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617580823d0 0x56175807b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561758076c00 0x56175807c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617580e6c20 0x5617580e9040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 --2- 172.21.15.31:0/1000166543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561758077520 0x56175807cb70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 >> 172.21.15.31:0/1000166543 conn(0x561757fec200 msgr2=0x56175807f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cc2ff5700 0 client.16533 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.043+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 shutdown_connections 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.044+0000 7f0cf91b0b80 1 -- 172.21.15.31:0/1000166543 wait complete. 2023-10-06T21:24:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.044+0000 7f0cf91b0b80 20 client.16533 trim_cache size 0 max 16384 2023-10-06T21:24:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:24:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:24:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:24:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:24:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 msgr2=0x7f0cc80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 0x7f0cc80a3ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009cb0 tx=0x7f0cd0009260 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 msgr2=0x7f0ce01a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800ef30 tx=0x7f0cd800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 shutdown_connections 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0cc80a15f0 0x7f0cc80a3ab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce01a30b0 0x7f0ce01a54a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce01010d0 0x7f0ce0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 --2- 172.21.15.31:0/1296251687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ce01061f0 0x7f0ce00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 >> 172.21.15.31:0/1296251687 conn(0x7f0ce00fc950 msgr2=0x7f0ce00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 shutdown_connections 2023-10-06T21:24:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:24:58.045+0000 7f0ce57ff700 1 -- 172.21.15.31:0/1296251687 wait complete. 2023-10-06T21:24:58.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:24:58 smithi146 ceph-mon[102809]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:58.335 INFO:tasks.workunit.client.0.smithi031.stdout: 1 57005 1.88 MB/sec execute 48 sec latency 27.812 ms 2023-10-06T21:24:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:24:58 smithi142 ceph-mon[102951]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:24:58 smithi031 ceph-mon[98539]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 864 KiB/s wr, 91 op/s 2023-10-06T21:24:59.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 57141 1.85 MB/sec execute 49 sec latency 245.956 ms 2023-10-06T21:25:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:00 smithi146 ceph-mon[102809]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:25:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:00 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:00.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 57325 1.81 MB/sec execute 50 sec latency 16.198 ms 2023-10-06T21:25:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:00 smithi142 ceph-mon[102951]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:25:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:00 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:00 smithi031 ceph-mon[98539]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:25:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:00 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:01.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 57644 1.79 MB/sec execute 51 sec latency 16.026 ms 2023-10-06T21:25:02.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 57962 1.78 MB/sec execute 52 sec latency 16.984 ms 2023-10-06T21:25:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:02 smithi142 ceph-mon[102951]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 775 KiB/s wr, 88 op/s 2023-10-06T21:25:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:02 smithi031 ceph-mon[98539]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 775 KiB/s wr, 88 op/s 2023-10-06T21:25:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:02 smithi146 ceph-mon[102809]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 775 KiB/s wr, 88 op/s 2023-10-06T21:25:03.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 58371 1.81 MB/sec execute 53 sec latency 19.457 ms 2023-10-06T21:25:04.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 58764 1.78 MB/sec execute 54 sec latency 18.714 ms 2023-10-06T21:25:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:04 smithi142 ceph-mon[102951]: pgmap v127: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2023-10-06T21:25:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:04 smithi031 ceph-mon[98539]: pgmap v127: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2023-10-06T21:25:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:04 smithi146 ceph-mon[102809]: pgmap v127: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2023-10-06T21:25:05.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 59158 1.77 MB/sec execute 55 sec latency 18.761 ms 2023-10-06T21:25:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:05 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:05 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:05 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:25:06.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 59541 1.81 MB/sec execute 56 sec latency 20.268 ms 2023-10-06T21:25:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:06 smithi142 ceph-mon[102951]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:06 smithi031 ceph-mon[98539]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:06 smithi146 ceph-mon[102809]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:07.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 60068 1.84 MB/sec execute 57 sec latency 18.798 ms 2023-10-06T21:25:08.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 60331 1.86 MB/sec execute 58 sec latency 19.982 ms 2023-10-06T21:25:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:08 smithi142 ceph-mon[102951]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:08 smithi031 ceph-mon[98539]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:08.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:08 smithi146 ceph-mon[102809]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 897 KiB/s wr, 103 op/s 2023-10-06T21:25:09.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 60607 1.85 MB/sec execute 59 sec latency 17.611 ms 2023-10-06T21:25:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:09 smithi031 ceph-mon[98539]: pgmap v130: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2023-10-06T21:25:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:09 smithi146 ceph-mon[102809]: pgmap v130: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2023-10-06T21:25:10.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 60778 1.83 MB/sec execute 60 sec latency 18.319 ms 2023-10-06T21:25:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:09 smithi142 ceph-mon[102951]: pgmap v130: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2023-10-06T21:25:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 -- 172.21.15.31:0/1857064581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0101ab0 msgr2=0x7fd6e0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/1857064581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0101ab0 0x7fd6e0105af0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009990 tx=0x7fd6cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 -- 172.21.15.31:0/1857064581 shutdown_connections 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/1857064581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106220 0x7fd6e010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/1857064581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/1857064581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0101ab0 0x7fd6e0105af0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 -- 172.21.15.31:0/1857064581 >> 172.21.15.31:0/1857064581 conn(0x7fd6e00fc9a0 msgr2=0x7fd6e00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 -- 172.21.15.31:0/1857064581 shutdown_connections 2023-10-06T21:25:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.216+0000 7fd6e6dff700 1 -- 172.21.15.31:0/1857064581 wait complete. 2023-10-06T21:25:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.217+0000 7fd6e6dff700 1 Processor -- start 2023-10-06T21:25:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 -- start start 2023-10-06T21:25:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 0x7fd6e0197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e019bc80 0x7fd6e019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e019c700 con 0x7fd6e0106220 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e019c870 con 0x7fd6e019bc80 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e019ca30 con 0x7fd6e0101100 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 0x7fd6e0197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39918/0 (socket says 172.21.15.31:39918) 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3113471377 learned_addr learned my addr 172.21.15.31:0/3113471377 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.218+0000 7fd6e539c700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e019bc80 0x7fd6e019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3113471377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e019bc80 msgr2=0x7fd6e019c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e019bc80 0x7fd6e019c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 msgr2=0x7fd6e0197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 0x7fd6e0197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3113471377 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0095f0 con 0x7fd6e0101100 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6dffff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 0x7fd6e0197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9a0 tx=0x7fd6d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6ddffb700 1 -- 172.21.15.31:0/3113471377 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4009790 con 0x7fd6e0101100 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6ddffb700 1 -- 172.21.15.31:0/3113471377 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6d4010430 con 0x7fd6e0101100 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e01a3f90 con 0x7fd6e0101100 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.219+0000 7fd6ddffb700 1 -- 172.21.15.31:0/3113471377 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400cea0 con 0x7fd6e0101100 2023-10-06T21:25:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.220+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e01a43b0 con 0x7fd6e0101100 2023-10-06T21:25:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.223+0000 7fd6ddffb700 1 -- 172.21.15.31:0/3113471377 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd6d40105a0 con 0x7fd6e0101100 2023-10-06T21:25:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.225+0000 7fd6fa70cb80 1 Processor -- start 2023-10-06T21:25:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6ddffb700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 0x7fd6d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 -- start start 2023-10-06T21:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6ddffb700 1 -- 172.21.15.31:0/3113471377 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd6d4014070 con 0x7fd6e0101100 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6e539c700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 0x7fd6d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a75140 0x561800a755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 0x561800a8ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618009de400 con 0x561800a88a70 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618009cfbc0 con 0x561800a75140 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.226+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618009cd4e0 con 0x561800a76790 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39928/0 (socket says 172.21.15.31:39928) 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2696609776 learned_addr learned my addr 172.21.15.31:0/2696609776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e539c700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 0x561800a8ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6dffff700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a75140 0x561800a755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e539c700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 0x7fd6d00a3ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8003ee0 tx=0x7fd6d8009ba0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2696609776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a75140 msgr2=0x561800a755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a75140 0x561800a755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2696609776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 msgr2=0x561800a8ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 0x561800a8ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2696609776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618007f3d80 con 0x561800a76790 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e539c700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 0x561800a8ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.227+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4096740 tx=0x7fd6d4099e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ccc028e9880208 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.228+0000 7fd6caffd700 1 -- 172.21.15.31:0/2696609776 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d409e390 con 0x561800a76790 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.228+0000 7fd6caffd700 1 -- 172.21.15.31:0/2696609776 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6d40a25d0 con 0x561800a76790 2023-10-06T21:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.228+0000 7fd6caffd700 1 -- 172.21.15.31:0/2696609776 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4099970 con 0x561800a76790 2023-10-06T21:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.228+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2696609776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 msgr2=0x561800a74c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.228+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4096740 tx=0x7fd6d4099e90 comp rx=0 tx=0).stop 2023-10-06T21:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2696609776 shutdown_connections 2023-10-06T21:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a88a70 0x561800a8ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a74c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2696609776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a75140 0x561800a755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2696609776 >> 172.21.15.31:0/2696609776 conn(0x5618009ea200 msgr2=0x561800a71080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2696609776 shutdown_connections 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2696609776 wait complete. 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.229+0000 7fd6fa70cb80 1 -- start start 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 Processor -- start 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618009c8dd0 con 0x561800a75140 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618009656b0 con 0x561800a88a70 2023-10-06T21:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.230+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800965350 con 0x561800a76790 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6fa70cb80 10 client.0 _ll_register_callbacks cb 0x5618007c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a78c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39938/0 (socket says 172.21.15.31:39938) 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6dffff700 1 -- 172.21.15.31:0/2924744766 learned_addr learned my addr 172.21.15.31:0/2924744766 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e539c700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2924744766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 msgr2=0x561800a78c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a78c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 msgr2=0x561800a79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.231+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/2924744766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d80042d0 con 0x561800a75140 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x561800a8b420 tx=0x7fd6d4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ca5aa522e0fbb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6bffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b84c0 con 0x561800a75140 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd6d409e390 con 0x561800a75140 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561800a68d50 con 0x561800a75140 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6d409a4a0 con 0x561800a75140 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6fa70cb80 10 client.16545 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fd6e01a43b0 con 0x561800a75140 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.232+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561800a68d50 con 0x561800a75140 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6d4065030 con 0x561800a75140 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 10 client.16545 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 20 client.16545 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800a98300 0x561800a9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 4 client.16545 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618009c1db0 con 0x561800a98300 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.233+0000 7fd6dffff700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800a98300 0x561800a9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.234+0000 7fd6dffff700 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800a98300 0x561800a9a720 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eafaef80d51c0175 server_cookie=83eab08bba30bdae in_seq=0 out_seq=0 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.234+0000 7fd6bffff700 10 client.16545 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6bffff700 1 -- 172.21.15.31:0/2924744766 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5618009c1db0 con 0x561800a98300 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6bffff700 10 client.16545 handle_command_reply: tid=0 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 client.16545 shutdown 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 msgr2=0x561800a7b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x561800a8b420 tx=0x7fd6d4011ef0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 shutdown_connections 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800a98300 0x561800a9a720 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/2924744766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a78c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6bffff700 0 client.16545 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 >> 172.21.15.31:0/2924744766 conn(0x5618009ea200 msgr2=0x561800a70c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 shutdown_connections 2023-10-06T21:25:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.235+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/2924744766 wait complete. 2023-10-06T21:25:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.236+0000 7fd6fa70cb80 20 client.16545 trim_cache size 0 max 16384 2023-10-06T21:25:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.251+0000 7fd6fa70cb80 1 Processor -- start 2023-10-06T21:25:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.251+0000 7fd6fa70cb80 1 -- start start 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618008498c0 con 0x561800a75140 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800a7a1c0 con 0x561800a88a70 2023-10-06T21:25:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800a79950 con 0x561800a76790 2023-10-06T21:25:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52668/0 (socket says 172.21.15.31:52668) 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3737601699 learned_addr learned my addr 172.21.15.31:0/3737601699 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e539c700 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3737601699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 msgr2=0x561800a88030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a88030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3737601699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 msgr2=0x561800a86b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a86b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3737601699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618007dcc00 con 0x561800a75140 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.252+0000 7fd6dffff700 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a88030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.253+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d40987f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e263c8dc80b9238 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.253+0000 7fd6cb7fe700 1 -- 172.21.15.31:0/3737601699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b84c0 con 0x561800a75140 2023-10-06T21:25:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.253+0000 7fd6cb7fe700 1 -- 172.21.15.31:0/3737601699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6d4010880 con 0x561800a75140 2023-10-06T21:25:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.253+0000 7fd6cb7fe700 1 -- 172.21.15.31:0/3737601699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a2380 con 0x561800a75140 2023-10-06T21:25:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3737601699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 msgr2=0x561800a7b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d40987f0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3737601699 shutdown_connections 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a86b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a7b530 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3737601699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a88030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3737601699 >> 172.21.15.31:0/3737601699 conn(0x5618009ea200 msgr2=0x561800a93ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3737601699 shutdown_connections 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3737601699 wait complete. 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.254+0000 7fd6fa70cb80 1 -- start start 2023-10-06T21:25:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 Processor -- start 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800a8de90 con 0x561800a75140 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800a8e000 con 0x561800a88a70 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561800a8e1c0 con 0x561800a76790 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6fa70cb80 10 client.0 _ll_register_callbacks cb 0x7fd6d8009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6e4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52670/0 (socket says 172.21.15.31:52670) 2023-10-06T21:25:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.255+0000 7fd6e4b9b700 1 -- 172.21.15.31:0/3742597219 learned_addr learned my addr 172.21.15.31:0/3742597219 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6e539c700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 -- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 msgr2=0x561800a8d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a8d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 -- 172.21.15.31:0/3742597219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 msgr2=0x561800a8f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 -- 172.21.15.31:0/3742597219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e010e610 con 0x561800a76790 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6dffff700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a8f670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b2e0 tx=0x7fd6cc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf0e9c765456bfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.256+0000 7fd6bffff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:25:11.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:10 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc01d040 con 0x561800a76790 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd6cc028040 con 0x561800a76790 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6d40a25d0 con 0x561800a76790 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6cc028200 con 0x561800a76790 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6fa70cb80 10 client.26236 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fd6e01a43b0 con 0x561800a76790 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.257+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6d40a25d0 con 0x561800a76790 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6cc02e020 con 0x561800a76790 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 10 client.26236 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 20 client.26236 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800aed110 0x561800aef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 4 client.26236 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618009c1ad0 con 0x561800aed110 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800aed110 0x561800aef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6e4b9b700 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800aed110 0x561800aef530 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb2a7c40b7ecfea server_cookie=8f0acc5cdfb50827 in_seq=0 out_seq=0 2023-10-06T21:25:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.258+0000 7fd6bffff700 10 client.26236 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.284+0000 7fd6bffff700 1 -- 172.21.15.31:0/3742597219 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618009c1ad0 con 0x561800aed110 2023-10-06T21:25:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.284+0000 7fd6bffff700 10 client.26236 handle_command_reply: tid=0 2023-10-06T21:25:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.284+0000 7fd6fa70cb80 1 client.26236 shutdown 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 msgr2=0x561800a8f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a8f670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b2e0 tx=0x7fd6cc009310 comp rx=0 tx=0).stop 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 shutdown_connections 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561800a88a70 0x561800a8d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561800aed110 0x561800aef530 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561800a75140 0x561800a8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 --2- 172.21.15.31:0/3742597219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561800a76790 0x561800a8f670 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6bffff700 0 client.26236 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 >> 172.21.15.31:0/3742597219 conn(0x5618009ea200 msgr2=0x561800a88030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 shutdown_connections 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 1 -- 172.21.15.31:0/3742597219 wait complete. 2023-10-06T21:25:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.285+0000 7fd6fa70cb80 20 client.26236 trim_cache size 0 max 16384 2023-10-06T21:25:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:25:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:25:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:25:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 msgr2=0x7fd6d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 0x7fd6d00a3ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8003ee0 tx=0x7fd6d8009ba0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 msgr2=0x7fd6e01970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9a0 tx=0x7fd6d400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 shutdown_connections 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e019bc80 0x7fd6e019c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6e0101100 0x7fd6e01970d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6d00a15f0 0x7fd6d00a3ab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 --2- 172.21.15.31:0/3113471377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0106220 0x7fd6e0197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 >> 172.21.15.31:0/3113471377 conn(0x7fd6e00fc9a0 msgr2=0x7fd6e00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 shutdown_connections 2023-10-06T21:25:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:11.286+0000 7fd6e6dff700 1 -- 172.21.15.31:0/3113471377 wait complete. 2023-10-06T21:25:11.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:10 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:11.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 60990 1.80 MB/sec execute 61 sec latency 28.706 ms 2023-10-06T21:25:11.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:10 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:11 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:25:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:11 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:25:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:11 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:25:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:11 smithi031 ceph-mon[98539]: pgmap v131: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-06T21:25:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:11 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:25:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:11 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:25:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:11 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:25:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:11 smithi146 ceph-mon[102809]: pgmap v131: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-06T21:25:12.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 61333 1.78 MB/sec execute 62 sec latency 16.316 ms 2023-10-06T21:25:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:11 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:25:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:11 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:25:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:11 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:25:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:11 smithi142 ceph-mon[102951]: pgmap v131: 97 pgs: 97 active+clean; 51 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-06T21:25:13.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 61739 1.82 MB/sec execute 63 sec latency 19.195 ms 2023-10-06T21:25:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:14 smithi146 ceph-mon[102809]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-06T21:25:14.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 62126 1.80 MB/sec execute 64 sec latency 19.742 ms 2023-10-06T21:25:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:14 smithi142 ceph-mon[102951]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-06T21:25:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:14 smithi031 ceph-mon[98539]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-06T21:25:15.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 62473 1.77 MB/sec execute 65 sec latency 158.911 ms 2023-10-06T21:25:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:16 smithi146 ceph-mon[102809]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:16.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 62785 1.77 MB/sec execute 66 sec latency 196.742 ms 2023-10-06T21:25:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:16 smithi142 ceph-mon[102951]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:16 smithi031 ceph-mon[98539]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:17.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 63299 1.82 MB/sec execute 67 sec latency 15.984 ms 2023-10-06T21:25:18.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:18 smithi146 ceph-mon[102809]: pgmap v134: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:18.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 63651 1.82 MB/sec execute 68 sec latency 21.012 ms 2023-10-06T21:25:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:18 smithi142 ceph-mon[102951]: pgmap v134: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:18 smithi031 ceph-mon[98539]: pgmap v134: 97 pgs: 97 active+clean; 54 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 891 KiB/s wr, 88 op/s 2023-10-06T21:25:19.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 64001 1.85 MB/sec execute 69 sec latency 18.790 ms 2023-10-06T21:25:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:19 smithi031 ceph-mon[98539]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:25:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:19 smithi146 ceph-mon[102809]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:25:20.336 INFO:tasks.workunit.client.0.smithi031.stdout: 1 64221 1.84 MB/sec execute 70 sec latency 17.841 ms 2023-10-06T21:25:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:19 smithi142 ceph-mon[102951]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:25:21.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 64402 1.81 MB/sec execute 71 sec latency 19.455 ms 2023-10-06T21:25:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:22 smithi146 ceph-mon[102809]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 815 KiB/s wr, 91 op/s 2023-10-06T21:25:22.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 64692 1.80 MB/sec execute 72 sec latency 16.007 ms 2023-10-06T21:25:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:22 smithi142 ceph-mon[102951]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 815 KiB/s wr, 91 op/s 2023-10-06T21:25:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:22 smithi031 ceph-mon[98539]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 815 KiB/s wr, 91 op/s 2023-10-06T21:25:23.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 65025 1.79 MB/sec execute 73 sec latency 21.915 ms 2023-10-06T21:25:24.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:24 smithi146 ceph-mon[102809]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:25:24.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 65403 1.81 MB/sec execute 74 sec latency 19.019 ms 2023-10-06T21:25:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:24 smithi142 ceph-mon[102951]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:25:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 -- 172.21.15.31:0/4128761511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103ad0 msgr2=0x7fca28107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/4128761511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103ad0 0x7fca28107b10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fca18009990 tx=0x7fca18009ca0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 -- 172.21.15.31:0/4128761511 shutdown_connections 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/4128761511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca28108240 0x7fca2810fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/4128761511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28103120 0x7fca28103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/4128761511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103ad0 0x7fca28107b10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 -- 172.21.15.31:0/4128761511 >> 172.21.15.31:0/4128761511 conn(0x7fca280fe9c0 msgr2=0x7fca28100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 -- 172.21.15.31:0/4128761511 shutdown_connections 2023-10-06T21:25:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.451+0000 7fca2ef9d700 1 -- 172.21.15.31:0/4128761511 wait complete. 2023-10-06T21:25:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.452+0000 7fca2ef9d700 1 Processor -- start 2023-10-06T21:25:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.452+0000 7fca2ef9d700 1 -- start start 2023-10-06T21:25:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 0x7fca28199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca2819d9b0 0x7fca2819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819e430 con 0x7fca28103120 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819e5a0 con 0x7fca28108240 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2ef9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819e760 con 0x7fca2819d9b0 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59766/0 (socket says 172.21.15.31:59766) 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2df9b700 1 -- 172.21.15.31:0/3690114271 learned_addr learned my addr 172.21.15.31:0/3690114271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 0x7fca28199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.453+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca2819d9b0 0x7fca2819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 -- 172.21.15.31:0/3690114271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca2819d9b0 msgr2=0x7fca2819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca2819d9b0 0x7fca2819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 -- 172.21.15.31:0/3690114271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 msgr2=0x7fca28199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 0x7fca28199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 -- 172.21.15.31:0/3690114271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180095f0 con 0x7fca28103120 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 0x7fca28199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9a0 tx=0x7fca2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca1effd700 1 -- 172.21.15.31:0/3690114271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24009790 con 0x7fca28103120 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca1effd700 1 -- 172.21.15.31:0/3690114271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca24010430 con 0x7fca28103120 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca281a5cc0 con 0x7fca28103120 2023-10-06T21:25:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.454+0000 7fca1effd700 1 -- 172.21.15.31:0/3690114271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cea0 con 0x7fca28103120 2023-10-06T21:25:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.455+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a61a0 con 0x7fca28103120 2023-10-06T21:25:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.458+0000 7fca1effd700 1 -- 172.21.15.31:0/3690114271 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fca240105a0 con 0x7fca28103120 2023-10-06T21:25:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.459+0000 7fca1effd700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 0x7fca140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.459+0000 7fca1effd700 1 -- 172.21.15.31:0/3690114271 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fca24014070 con 0x7fca28103120 2023-10-06T21:25:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.459+0000 7fca43b2cb80 1 Processor -- start 2023-10-06T21:25:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 0x7fca140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 -- start start 2023-10-06T21:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84decff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84ded530 0x562a84def920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d4b400 con 0x562a84ded530 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d3cbc0 con 0x562a84de2680 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.460+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d3a4e0 con 0x562a84de1d60 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2d79a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84decff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:34218/0 (socket says 172.21.15.31:34218) 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 -- 172.21.15.31:0/1326014146 learned_addr learned my addr 172.21.15.31:0/1326014146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2e79c700 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84ded530 0x562a84def920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 0x7fca140a3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca18006010 tx=0x7fca1800b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 -- 172.21.15.31:0/1326014146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 msgr2=0x562a84decff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84decff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 -- 172.21.15.31:0/1326014146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84ded530 msgr2=0x562a84def920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84ded530 0x562a84def920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 -- 172.21.15.31:0/1326014146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a84b60d80 con 0x562a84de1d60 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.461+0000 7fca2df9b700 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca24096740 tx=0x7fca24099e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41663d08b8ca47a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.462+0000 7fca13fff700 1 -- 172.21.15.31:0/1326014146 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2409e160 con 0x562a84de1d60 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.462+0000 7fca13fff700 1 -- 172.21.15.31:0/1326014146 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca2409e390 con 0x562a84de1d60 2023-10-06T21:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.462+0000 7fca13fff700 1 -- 172.21.15.31:0/1326014146 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24099930 con 0x562a84de1d60 2023-10-06T21:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.462+0000 7fca43b2cb80 1 -- 172.21.15.31:0/1326014146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 msgr2=0x562a84de2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.462+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca24096740 tx=0x7fca24099e90 comp rx=0 tx=0).stop 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 -- 172.21.15.31:0/1326014146 shutdown_connections 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84ded530 0x562a84def920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84de2140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/1326014146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84decff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 -- 172.21.15.31:0/1326014146 >> 172.21.15.31:0/1326014146 conn(0x562a84d57200 msgr2=0x562a84dde1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 -- 172.21.15.31:0/1326014146 shutdown_connections 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 -- 172.21.15.31:0/1326014146 wait complete. 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 -- start start 2023-10-06T21:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.463+0000 7fca43b2cb80 1 Processor -- start 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 0x562a84dea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84deacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d35dd0 con 0x562a84de2680 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84cd26b0 con 0x562a84de1d60 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84cd2350 con 0x562a84ded530 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca43b2cb80 10 client.0 _ll_register_callbacks cb 0x562a84b2f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59778/0 (socket says 172.21.15.31:59778) 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca2d79a700 1 -- 172.21.15.31:0/3724304829 learned_addr learned my addr 172.21.15.31:0/3724304829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 0x562a84dea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.464+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84deacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 -- 172.21.15.31:0/3724304829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 msgr2=0x562a84deacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84deacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 -- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 msgr2=0x562a84dea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 0x562a84dea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 -- 172.21.15.31:0/3724304829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a61a0 con 0x562a84de2680 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 0x562a84dea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fca18009990 tx=0x7fca1801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610a64df5ea0f13e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca117fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18028030 con 0x562a84de2680 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a84dd5eb0 con 0x562a84de2680 2023-10-06T21:25:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.465+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fca180199f0 con 0x562a84de2680 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca18019bb0 con 0x562a84de2680 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 10 client.16563 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562a84b49c00 con 0x562a84de2680 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a84dd5eb0 con 0x562a84de2680 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca1802fb00 con 0x562a84de2680 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 20 client.16563 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84dfd050 0x562a84dff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 4 client.16563 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.466+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a84d2edb0 con 0x562a84dfd050 2023-10-06T21:25:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.467+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84dfd050 0x562a84dff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.467+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84dfd050 0x562a84dff470 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6b7f1b0520eeed server_cookie=7688efab563bcbe8 in_seq=0 out_seq=0 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.467+0000 7fca117fa700 10 client.16563 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca117fa700 1 -- 172.21.15.31:0/3724304829 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a84d2edb0 con 0x562a84dfd050 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca117fa700 10 client.16563 handle_command_reply: tid=0 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 client.16563 shutdown 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 msgr2=0x562a84dea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fca18009990 tx=0x7fca1801a050 comp rx=0 tx=0).stop 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 shutdown_connections 2023-10-06T21:25:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84deacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de1d60 0x562a84dea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84dfd050 0x562a84dff470 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3724304829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84dea780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca117fa700 0 client.16563 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 >> 172.21.15.31:0/3724304829 conn(0x562a84d57200 msgr2=0x562a84dddda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.468+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 shutdown_connections 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.469+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3724304829 wait complete. 2023-10-06T21:25:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.469+0000 7fca43b2cb80 20 client.16563 trim_cache size 0 max 16384 2023-10-06T21:25:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.487+0000 7fca43b2cb80 1 Processor -- start 2023-10-06T21:25:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 -- start start 2023-10-06T21:25:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84df4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84ded530 0x562a84de74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84bb68c0 con 0x562a84de2680 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84df45b0 con 0x562a84ded530 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84de87c0 con 0x562a84de1d60 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59784/0 (socket says 172.21.15.31:59784) 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca2d79a700 1 -- 172.21.15.31:0/2007861342 learned_addr learned my addr 172.21.15.31:0/2007861342 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca2df9b700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84df4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.488+0000 7fca2e79c700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84ded530 0x562a84de74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 -- 172.21.15.31:0/2007861342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 msgr2=0x562a84df4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84df4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 -- 172.21.15.31:0/2007861342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84ded530 msgr2=0x562a84de74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84ded530 0x562a84de74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 -- 172.21.15.31:0/2007861342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a84e040a0 con 0x562a84de2680 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2df9b700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84df4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca2d79a700 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fca18005930 tx=0x7fca18005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f4f933b278f0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca1cff9700 1 -- 172.21.15.31:0/2007861342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18028070 con 0x562a84de2680 2023-10-06T21:25:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca1cff9700 1 -- 172.21.15.31:0/2007861342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca1801e040 con 0x562a84de2680 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.489+0000 7fca1cff9700 1 -- 172.21.15.31:0/2007861342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18019930 con 0x562a84de2680 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 -- 172.21.15.31:0/2007861342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 msgr2=0x562a84deaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fca18005930 tx=0x7fca18005800 comp rx=0 tx=0).stop 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 -- 172.21.15.31:0/2007861342 shutdown_connections 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84ded530 0x562a84de74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84de1d60 0x562a84df4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/2007861342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de2680 0x562a84deaac0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 -- 172.21.15.31:0/2007861342 >> 172.21.15.31:0/2007861342 conn(0x562a84d57200 msgr2=0x562a84de5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 -- 172.21.15.31:0/2007861342 shutdown_connections 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.490+0000 7fca43b2cb80 1 -- 172.21.15.31:0/2007861342 wait complete. 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.491+0000 7fca43b2cb80 1 -- start start 2023-10-06T21:25:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.491+0000 7fca43b2cb80 1 Processor -- start 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 0x562a84df2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84df25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84df2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84e4ffc0 con 0x562a84de1d60 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84e50130 con 0x562a84de2680 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84e502f0 con 0x562a84ded530 2023-10-06T21:25:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca43b2cb80 10 client.0 _ll_register_callbacks cb 0x7fca1800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 0x562a84df2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 0x562a84df2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59794/0 (socket says 172.21.15.31:59794) 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2df9b700 1 -- 172.21.15.31:0/3674838168 learned_addr learned my addr 172.21.15.31:0/3674838168 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84df25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84df2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 -- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 msgr2=0x562a84df25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84df25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 -- 172.21.15.31:0/3674838168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 msgr2=0x562a84df2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 0x562a84df2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.492+0000 7fca2e79c700 1 -- 172.21.15.31:0/3674838168 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a84b60d80 con 0x562a84ded530 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca2d79a700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84df25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca2e79c700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84df2ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562a84ded430 tx=0x7fca20011f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8161ddfc7ae4b0bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca117fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2001c070 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca2409e390 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fca20007bc0 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca2000e040 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca43b2cb80 10 client.26254 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.493+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562a84b49c00 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca2002c020 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca20020a50 con 0x562a84ded530 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 10 client.26254 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 20 client.26254 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84e51b10 0x562a84e53f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 4 client.26254 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a84d2ead0 con 0x562a84e51b10 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.494+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84e51b10 0x562a84e53f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.496+0000 7fca2df9b700 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84e51b10 0x562a84e53f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab59f1960ea25450 server_cookie=ea0ede11c3f2e7c7 in_seq=0 out_seq=0 2023-10-06T21:25:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.496+0000 7fca117fa700 10 client.26254 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:24 smithi031 ceph-mon[98539]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2023-10-06T21:25:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.515+0000 7fca117fa700 1 -- 172.21.15.31:0/3674838168 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a84d2ead0 con 0x562a84e51b10 2023-10-06T21:25:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.515+0000 7fca117fa700 10 client.26254 handle_command_reply: tid=0 2023-10-06T21:25:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 client.26254 shutdown 2023-10-06T21:25:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 msgr2=0x562a84df2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84df2ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562a84ded430 tx=0x7fca20011f30 comp rx=0 tx=0).stop 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 shutdown_connections 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562a84ded530 0x562a84df2ae0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562a84e51b10 0x562a84e53f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a84de1d60 0x562a84df2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 --2- 172.21.15.31:0/3674838168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a84de2680 0x562a84df25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca117fa700 0 client.26254 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 >> 172.21.15.31:0/3674838168 conn(0x562a84d57200 msgr2=0x562a84de8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 shutdown_connections 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 1 -- 172.21.15.31:0/3674838168 wait complete. 2023-10-06T21:25:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.516+0000 7fca43b2cb80 20 client.26254 trim_cache size 0 max 16384 2023-10-06T21:25:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:25:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:25:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:25:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 msgr2=0x7fca140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 0x7fca140a3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca18006010 tx=0x7fca1800b260 comp rx=0 tx=0).stop 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 msgr2=0x7fca28199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9a0 tx=0x7fca2400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 shutdown_connections 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca2819d9b0 0x7fca2819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca140a1670 0x7fca140a3b30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca28103120 0x7fca28199010 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 --2- 172.21.15.31:0/3690114271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca28108240 0x7fca28199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 >> 172.21.15.31:0/3690114271 conn(0x7fca280fe9c0 msgr2=0x7fca280ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.517+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 shutdown_connections 2023-10-06T21:25:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:24.518+0000 7fca2ef9d700 1 -- 172.21.15.31:0/3690114271 wait complete. 2023-10-06T21:25:25.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 65788 1.79 MB/sec execute 75 sec latency 17.916 ms 2023-10-06T21:25:26.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:26 smithi146 ceph-mon[102809]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:26.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 66225 1.79 MB/sec execute 76 sec latency 17.872 ms 2023-10-06T21:25:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:26 smithi142 ceph-mon[102951]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:26 smithi031 ceph-mon[98539]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:27.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 66583 1.81 MB/sec execute 77 sec latency 15.227 ms 2023-10-06T21:25:28.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:28 smithi146 ceph-mon[102809]: pgmap v139: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:28.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 67143 1.84 MB/sec execute 78 sec latency 20.128 ms 2023-10-06T21:25:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:28 smithi142 ceph-mon[102951]: pgmap v139: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:28 smithi031 ceph-mon[98539]: pgmap v139: 97 pgs: 97 active+clean; 56 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 798 KiB/s wr, 86 op/s 2023-10-06T21:25:29.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 67386 1.85 MB/sec execute 79 sec latency 21.233 ms 2023-10-06T21:25:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:29 smithi031 ceph-mon[98539]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-10-06T21:25:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:29 smithi146 ceph-mon[102809]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-10-06T21:25:30.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 67668 1.84 MB/sec execute 80 sec latency 19.855 ms 2023-10-06T21:25:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:29 smithi142 ceph-mon[102951]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-10-06T21:25:31.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 67795 1.83 MB/sec execute 81 sec latency 324.720 ms 2023-10-06T21:25:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:32 smithi146 ceph-mon[102809]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 872 KiB/s wr, 91 op/s 2023-10-06T21:25:32.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 67969 1.80 MB/sec execute 82 sec latency 18.324 ms 2023-10-06T21:25:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:32 smithi142 ceph-mon[102951]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 872 KiB/s wr, 91 op/s 2023-10-06T21:25:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:32 smithi031 ceph-mon[98539]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 872 KiB/s wr, 91 op/s 2023-10-06T21:25:33.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 68279 1.79 MB/sec execute 83 sec latency 19.357 ms 2023-10-06T21:25:34.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:34 smithi146 ceph-mon[102809]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:34.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 68596 1.79 MB/sec execute 84 sec latency 19.632 ms 2023-10-06T21:25:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:34 smithi142 ceph-mon[102951]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:34.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:34 smithi031 ceph-mon[98539]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:35.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 68917 1.80 MB/sec execute 85 sec latency 174.831 ms 2023-10-06T21:25:36.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:36 smithi146 ceph-mon[102809]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:36.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 69294 1.78 MB/sec execute 86 sec latency 16.194 ms 2023-10-06T21:25:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:36 smithi142 ceph-mon[102951]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:36 smithi031 ceph-mon[98539]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:37.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 69728 1.78 MB/sec execute 87 sec latency 19.209 ms 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 -- 172.21.15.31:0/3455547365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34900682d0 msgr2=0x7f3490068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 --2- 172.21.15.31:0/3455547365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34900682d0 0x7f3490068750 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f34840099c0 tx=0x7f3484009cd0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 -- 172.21.15.31:0/3455547365 shutdown_connections 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 --2- 172.21.15.31:0/3455547365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f349010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 --2- 172.21.15.31:0/3455547365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3490106d30 0x7f3490107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 --2- 172.21.15.31:0/3455547365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34900682d0 0x7f3490068750 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 -- 172.21.15.31:0/3455547365 >> 172.21.15.31:0/3455547365 conn(0x7f34900756e0 msgr2=0x7f3490075af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 -- 172.21.15.31:0/3455547365 shutdown_connections 2023-10-06T21:25:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.690+0000 7f349519d700 1 -- 172.21.15.31:0/3455547365 wait complete. 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 Processor -- start 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 -- start start 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34900682d0 0x7f3490196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f34901971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 0x7f349019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349019aba0 con 0x7f3490106d30 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349019ad10 con 0x7f3490068c90 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.692+0000 7f349519d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349019aed0 con 0x7f34900682d0 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f349499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 0x7f349019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f34901971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34900682d0 0x7f3490196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f349499c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 0x7f349019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39588/0 (socket says 172.21.15.31:39588) 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f349499c700 1 -- 172.21.15.31:0/2873190 learned_addr learned my addr 172.21.15.31:0/2873190 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 -- 172.21.15.31:0/2873190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34900682d0 msgr2=0x7f3490196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34900682d0 0x7f3490196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 -- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 msgr2=0x7f349019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 0x7f349019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 -- 172.21.15.31:0/2873190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840095f0 con 0x7f3490068c90 2023-10-06T21:25:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.693+0000 7f348f7fe700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f34901971f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f34840098e0 tx=0x7f348400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.694+0000 7f348d7fa700 1 -- 172.21.15.31:0/2873190 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348401d070 con 0x7f3490068c90 2023-10-06T21:25:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.694+0000 7f348d7fa700 1 -- 172.21.15.31:0/2873190 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3484005050 con 0x7f3490068c90 2023-10-06T21:25:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.694+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349019b150 con 0x7f3490068c90 2023-10-06T21:25:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.694+0000 7f348d7fa700 1 -- 172.21.15.31:0/2873190 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484004970 con 0x7f3490068c90 2023-10-06T21:25:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.694+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349019b420 con 0x7f3490068c90 2023-10-06T21:25:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.698+0000 7f348d7fa700 1 -- 172.21.15.31:0/2873190 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3484005430 con 0x7f3490068c90 2023-10-06T21:25:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.699+0000 7f34a9c4eb80 1 Processor -- start 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f34a9c4eb80 1 -- start start 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f348d7fa700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 0x7f34780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f348d7fa700 1 -- 172.21.15.31:0/2873190 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f34840cdcc0 con 0x7f3490068c90 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f349499c700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 0x7f34780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.700+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 0x5633d002dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0030920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff8c400 con 0x5633d0023680 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff7dbc0 con 0x5633d0022d60 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff7b4e0 con 0x5633d002e530 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58980/0 (socket says 172.21.15.31:58980) 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 -- 172.21.15.31:0/2387579415 learned_addr learned my addr 172.21.15.31:0/2387579415 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348f7fe700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 0x5633d002dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f349499c700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0030920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 -- 172.21.15.31:0/2387579415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 msgr2=0x5633d0030920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0030920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 -- 172.21.15.31:0/2387579415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 msgr2=0x5633d002dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 0x5633d002dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 -- 172.21.15.31:0/2387579415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633cfda1d80 con 0x5633d0022d60 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348f7fe700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 0x5633d002dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f349499c700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 0x7f34780a3ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3488009c10 tx=0x7f3488012040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.701+0000 7f348ffff700 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3480009d00 tx=0x7f3480011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41d6d55ff7858fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.702+0000 7f34767fc700 1 -- 172.21.15.31:0/2387579415 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480019630 con 0x5633d0022d60 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.702+0000 7f34767fc700 1 -- 172.21.15.31:0/2387579415 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3480019c20 con 0x5633d0022d60 2023-10-06T21:25:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.702+0000 7f34767fc700 1 -- 172.21.15.31:0/2387579415 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000ea20 con 0x5633d0022d60 2023-10-06T21:25:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/2387579415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 msgr2=0x5633d0023140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3480009d00 tx=0x7f3480011720 comp rx=0 tx=0).stop 2023-10-06T21:25:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/2387579415 shutdown_connections 2023-10-06T21:25:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0030920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0022d60 0x5633d0023140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/2387579415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0023680 0x5633d002dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/2387579415 >> 172.21.15.31:0/2387579415 conn(0x5633cff98200 msgr2=0x5633d001f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/2387579415 shutdown_connections 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/2387579415 wait complete. 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.703+0000 7f34a9c4eb80 1 -- start start 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 Processor -- start 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d002df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff76dd0 con 0x5633d0022d60 2023-10-06T21:25:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff136b0 con 0x5633d0023680 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cff13350 con 0x5633d002e530 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.704+0000 7f34a9c4eb80 10 client.0 _ll_register_callbacks cb 0x5633cfd707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d002df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d002df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39606/0 (socket says 172.21.15.31:39606) 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348ffff700 1 -- 172.21.15.31:0/1773060007 learned_addr learned my addr 172.21.15.31:0/1773060007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58982/0 (socket says 172.21.15.31:58982) 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f349499c700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 -- 172.21.15.31:0/1773060007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 msgr2=0x5633d002b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 -- 172.21.15.31:0/1773060007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 msgr2=0x5633d002df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d002df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 -- 172.21.15.31:0/1773060007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349019b420 con 0x5633d0023680 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34840cb3c0 tx=0x7f348409b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50dee94d720b183 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f346f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840b2070 con 0x5633d0023680 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f348409ee30 con 0x5633d0023680 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.705+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633d0016eb0 con 0x5633d0023680 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34840a2fe0 con 0x5633d0023680 2023-10-06T21:25:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 10 client.26309 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f3480009960 con 0x5633d0023680 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3484064020 con 0x5633d0023680 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34840b3260 con 0x5633d0023680 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 10 client.26309 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 10 client.26309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 20 client.26309 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d003dbe0 0x5633d0040000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 4 client.26309 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.706+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633cff6fdb0 con 0x5633d003dbe0 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.707+0000 7f348ffff700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d003dbe0 0x5633d0040000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.707+0000 7f348ffff700 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d003dbe0 0x5633d0040000 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a3501ad932f975e server_cookie=f71db95c2475b9db in_seq=0 out_seq=0 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.707+0000 7f346f7fe700 10 client.26309 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f346f7fe700 1 -- 172.21.15.31:0/1773060007 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633cff6fdb0 con 0x5633d003dbe0 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f346f7fe700 10 client.26309 handle_command_reply: tid=0 2023-10-06T21:25:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 client.26309 shutdown 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 msgr2=0x5633d002b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34840cb3c0 tx=0x7f348409b4d0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 shutdown_connections 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d003dbe0 0x5633d0040000 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d002df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1773060007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002b1a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f346f7fe700 0 client.26309 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.708+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 >> 172.21.15.31:0/1773060007 conn(0x5633cff98200 msgr2=0x5633d001eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.709+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 shutdown_connections 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.709+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1773060007 wait complete. 2023-10-06T21:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.709+0000 7f34a9c4eb80 20 client.26309 trim_cache size 0 max 16384 2023-10-06T21:25:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.725+0000 7f34a9c4eb80 1 Processor -- start 2023-10-06T21:25:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.725+0000 7f34a9c4eb80 1 -- start start 2023-10-06T21:25:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cfdf78c0 con 0x5633d0022d60 2023-10-06T21:25:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d0027030 con 0x5633d0023680 2023-10-06T21:25:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d0040660 con 0x5633d002e530 2023-10-06T21:25:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39618/0 (socket says 172.21.15.31:39618) 2023-10-06T21:25:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348ffff700 1 -- 172.21.15.31:0/3422115917 learned_addr learned my addr 172.21.15.31:0/3422115917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f349499c700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 -- 172.21.15.31:0/3422115917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 msgr2=0x5633d002dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 -- 172.21.15.31:0/3422115917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 msgr2=0x5633d0035070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348f7fe700 1 -- 172.21.15.31:0/3422115917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633cfd8ac00 con 0x5633d0023680 2023-10-06T21:25:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f349499c700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:25:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.726+0000 7f348ffff700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.727+0000 7f348f7fe700 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002a980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34840cb9c0 tx=0x7f348409ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63aba1afa35b39b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.727+0000 7f3476ffd700 1 -- 172.21.15.31:0/3422115917 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348409ee30 con 0x5633d0023680 2023-10-06T21:25:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.727+0000 7f3476ffd700 1 -- 172.21.15.31:0/3422115917 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3484064070 con 0x5633d0023680 2023-10-06T21:25:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.727+0000 7f3476ffd700 1 -- 172.21.15.31:0/3422115917 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840662b0 con 0x5633d0023680 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/3422115917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 msgr2=0x5633d002a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002a980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34840cb9c0 tx=0x7f348409ae60 comp rx=0 tx=0).stop 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/3422115917 shutdown_connections 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d002dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0035070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/3422115917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d002a980 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/3422115917 >> 172.21.15.31:0/3422115917 conn(0x5633cff98200 msgr2=0x5633d001d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/3422115917 shutdown_connections 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/3422115917 wait complete. 2023-10-06T21:25:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.728+0000 7f34a9c4eb80 1 -- start start 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 Processor -- start 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d0028b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0027440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d00292b0 con 0x5633d0022d60 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d0027980 con 0x5633d0023680 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d0027b40 con 0x5633d002e530 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f34a9c4eb80 10 client.0 _ll_register_callbacks cb 0x5633cfd707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f348ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39632/0 (socket says 172.21.15.31:39632) 2023-10-06T21:25:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d0028b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.729+0000 7f348ffff700 1 -- 172.21.15.31:0/1751520768 learned_addr learned my addr 172.21.15.31:0/1751520768 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f349499c700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0027440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 -- 172.21.15.31:0/1751520768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 msgr2=0x5633d0027440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0027440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 -- 172.21.15.31:0/1751520768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 msgr2=0x5633d0028610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 -- 172.21.15.31:0/1751520768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633cfda1d80 con 0x5633d0023680 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348ffff700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f348f7fe700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d0028b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3484009fd0 tx=0x7f3484099290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dd5a124515f166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f346f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840b2070 con 0x5633d0023680 2023-10-06T21:25:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f3484065080 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.730+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3480019c20 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f348409b3e0 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 10 client.26315 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f3480009960 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f348407d080 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34840cbb10 con 0x5633d0023680 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 10 client.26315 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 10 client.26315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 20 client.26315 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d0092340 0x5633d0094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 4 client.26315 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.731+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633cff6fad0 con 0x5633d0092340 2023-10-06T21:25:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.732+0000 7f348ffff700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d0092340 0x5633d0094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.732+0000 7f346f7fe700 10 client.26315 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.732+0000 7f348ffff700 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d0092340 0x5633d0094730 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82dc29107ce3135 server_cookie=7086e14b8c487dfe in_seq=0 out_seq=0 2023-10-06T21:25:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f346f7fe700 1 -- 172.21.15.31:0/1751520768 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633cff6fad0 con 0x5633d0092340 2023-10-06T21:25:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f346f7fe700 10 client.26315 handle_command_reply: tid=0 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 client.26315 shutdown 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 msgr2=0x5633d0028b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d0028b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3484009fd0 tx=0x7f3484099290 comp rx=0 tx=0).stop 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 shutdown_connections 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633d002e530 0x5633d0027440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5633d0092340 0x5633d0094730 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5633d0022d60 0x5633d0028610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 --2- 172.21.15.31:0/1751520768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d0023680 0x5633d0028b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f346f7fe700 0 client.26315 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 >> 172.21.15.31:0/1751520768 conn(0x5633cff98200 msgr2=0x5633d00264f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 shutdown_connections 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 1 -- 172.21.15.31:0/1751520768 wait complete. 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.755+0000 7f34a9c4eb80 20 client.26315 trim_cache size 0 max 16384 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:25:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 msgr2=0x7f34780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 0x7f34780a3ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3488009c10 tx=0x7f3488012040 comp rx=0 tx=0).stop 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 msgr2=0x7f34901971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f34901971f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f34840098e0 tx=0x7f348400f690 comp rx=0 tx=0).stop 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 shutdown_connections 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f34780a15f0 0x7f34780a3ab0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3490106d30 0x7f349019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34900682d0 0x7f3490196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 --2- 172.21.15.31:0/2873190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3490068c90 0x7f34901971f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 >> 172.21.15.31:0/2873190 conn(0x7f34900756e0 msgr2=0x7f34900feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 shutdown_connections 2023-10-06T21:25:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:37.756+0000 7f349519d700 1 -- 172.21.15.31:0/2873190 wait complete. 2023-10-06T21:25:38.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:38 smithi146 ceph-mon[102809]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:38.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 70002 1.77 MB/sec execute 88 sec latency 19.786 ms 2023-10-06T21:25:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:38 smithi142 ceph-mon[102951]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:38 smithi031 ceph-mon[98539]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 839 KiB/s wr, 85 op/s 2023-10-06T21:25:39.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 70590 1.81 MB/sec execute 89 sec latency 18.367 ms 2023-10-06T21:25:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:39 smithi031 ceph-mon[98539]: pgmap v145: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:40.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:39 smithi146 ceph-mon[102809]: pgmap v145: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:40.337 INFO:tasks.workunit.client.0.smithi031.stdout: 1 70790 1.80 MB/sec execute 90 sec latency 20.884 ms 2023-10-06T21:25:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:39 smithi142 ceph-mon[102951]: pgmap v145: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:25:41.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 71190 1.83 MB/sec execute 91 sec latency 18.182 ms 2023-10-06T21:25:42.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:42 smithi146 ceph-mon[102809]: pgmap v146: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 709 KiB/s wr, 86 op/s 2023-10-06T21:25:42.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 71355 1.82 MB/sec execute 92 sec latency 17.800 ms 2023-10-06T21:25:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:42 smithi142 ceph-mon[102951]: pgmap v146: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 709 KiB/s wr, 86 op/s 2023-10-06T21:25:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:42 smithi031 ceph-mon[98539]: pgmap v146: 97 pgs: 97 active+clean; 53 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 709 KiB/s wr, 86 op/s 2023-10-06T21:25:43.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 71538 1.80 MB/sec execute 93 sec latency 18.705 ms 2023-10-06T21:25:44.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:44 smithi146 ceph-mon[102809]: pgmap v147: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:25:44.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 71867 1.78 MB/sec execute 94 sec latency 19.978 ms 2023-10-06T21:25:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:44 smithi142 ceph-mon[102951]: pgmap v147: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:25:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:44 smithi031 ceph-mon[98539]: pgmap v147: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:25:45.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 72200 1.78 MB/sec execute 95 sec latency 19.425 ms 2023-10-06T21:25:46.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:46 smithi146 ceph-mon[102809]: pgmap v148: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:46.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 72588 1.79 MB/sec execute 96 sec latency 18.665 ms 2023-10-06T21:25:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:46 smithi142 ceph-mon[102951]: pgmap v148: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:46 smithi031 ceph-mon[98539]: pgmap v148: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:47.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 73033 1.78 MB/sec execute 97 sec latency 17.859 ms 2023-10-06T21:25:48.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:48 smithi146 ceph-mon[102809]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:48.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 73410 1.78 MB/sec execute 98 sec latency 18.696 ms 2023-10-06T21:25:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:48 smithi142 ceph-mon[102951]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:48 smithi031 ceph-mon[98539]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 869 KiB/s wr, 88 op/s 2023-10-06T21:25:49.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 73889 1.81 MB/sec execute 99 sec latency 17.975 ms 2023-10-06T21:25:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:49 smithi031 ceph-mon[98539]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2023-10-06T21:25:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:49 smithi146 ceph-mon[102809]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2023-10-06T21:25:50.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 74301 1.81 MB/sec execute 100 sec latency 20.110 ms 2023-10-06T21:25:50.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:49 smithi142 ceph-mon[102951]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 -- 172.21.15.31:0/4028475678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8100f40 msgr2=0x7ff3f8101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/4028475678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8100f40 0x7ff3f8101320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009990 tx=0x7ff3e8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 -- 172.21.15.31:0/4028475678 shutdown_connections 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/4028475678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8106060 0x7ff3f810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/4028475678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8100f40 0x7ff3f8101320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/4028475678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f81018f0 0x7ff3f8105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 -- 172.21.15.31:0/4028475678 >> 172.21.15.31:0/4028475678 conn(0x7ff3f80fc7c0 msgr2=0x7ff3f80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.922+0000 7ff3f7fff700 1 -- 172.21.15.31:0/4028475678 shutdown_connections 2023-10-06T21:25:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.923+0000 7ff3f7fff700 1 -- 172.21.15.31:0/4028475678 wait complete. 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 Processor -- start 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 -- start start 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 0x7ff3f8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f81975c0 con 0x7ff3f8100f40 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8197700 con 0x7ff3f8106060 2023-10-06T21:25:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f7fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f81978c0 con 0x7ff3f8196b40 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.924+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52674/0 (socket says 172.21.15.31:52674) 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:51428/0 (socket says 172.21.15.31:51428) 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/731878906 learned_addr learned my addr 172.21.15.31:0/731878906 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 0x7ff3f8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/731878906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 msgr2=0x7ff3f8196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 0x7ff3f8196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/731878906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 msgr2=0x7ff3f819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/731878906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec009520 con 0x7ff3f8100f40 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 0x7ff3f8196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.925+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005740 tx=0x7ff3e8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.926+0000 7ff3dffff700 1 -- 172.21.15.31:0/731878906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e801d070 con 0x7ff3f8100f40 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.926+0000 7ff3dffff700 1 -- 172.21.15.31:0/731878906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3e8022470 con 0x7ff3f8100f40 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.926+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80095f0 con 0x7ff3f8100f40 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.926+0000 7ff3dffff700 1 -- 172.21.15.31:0/731878906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8017650 con 0x7ff3f8100f40 2023-10-06T21:25:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.926+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f81a3c30 con 0x7ff3f8100f40 2023-10-06T21:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.929+0000 7ff3dffff700 1 -- 172.21.15.31:0/731878906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff3e800fb00 con 0x7ff3f8100f40 2023-10-06T21:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 Processor -- start 2023-10-06T21:25:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 -- start start 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff3dffff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 0x7ff3e00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33904920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33860400 con 0x55ec338f7680 2023-10-06T21:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33851bc0 con 0x55ec338f6d60 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.931+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3384f4e0 con 0x55ec33902530 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3dffff700 1 -- 172.21.15.31:0/731878906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff3e80ce6e0 con 0x7ff3f8100f40 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 0x7ff3e00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52686/0 (socket says 172.21.15.31:52686) 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 -- 172.21.15.31:0/160673306 learned_addr learned my addr 172.21.15.31:0/160673306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33904920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 -- 172.21.15.31:0/160673306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 msgr2=0x55ec33904920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33904920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 -- 172.21.15.31:0/160673306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 msgr2=0x55ec338f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.932+0000 7ff3f67fc700 1 -- 172.21.15.31:0/160673306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec33675d80 con 0x55ec338f7680 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 0x7ff3e00a3ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0007a80 tx=0x7ff3f0007e40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec011810 tx=0x7ff3ec011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1afde54bb9f3b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3dd7fa700 1 -- 172.21.15.31:0/160673306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec004830 con 0x55ec338f7680 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3dd7fa700 1 -- 172.21.15.31:0/160673306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3ec004e20 con 0x55ec338f7680 2023-10-06T21:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.933+0000 7ff3dd7fa700 1 -- 172.21.15.31:0/160673306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec00f540 con 0x55ec338f7680 2023-10-06T21:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 -- 172.21.15.31:0/160673306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 msgr2=0x55ec33901ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec011810 tx=0x7ff3ec011b20 comp rx=0 tx=0).stop 2023-10-06T21:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 -- 172.21.15.31:0/160673306 shutdown_connections 2023-10-06T21:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33904920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/160673306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901ff0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 -- 172.21.15.31:0/160673306 >> 172.21.15.31:0/160673306 conn(0x55ec3386c200 msgr2=0x55ec338f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 -- 172.21.15.31:0/160673306 shutdown_connections 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.934+0000 7ff4110b4b80 1 -- 172.21.15.31:0/160673306 wait complete. 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.935+0000 7ff4110b4b80 1 -- start start 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.935+0000 7ff4110b4b80 1 Processor -- start 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.935+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec33901fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec338f7680 0x55ec338ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3384add0 con 0x55ec33902530 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec337e76b0 con 0x55ec338f6d60 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec337e7350 con 0x55ec338f7680 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff4110b4b80 10 client.0 _ll_register_callbacks cb 0x55ec336447a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec33901fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52696/0 (socket says 172.21.15.31:52696) 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 -- 172.21.15.31:0/746944782 learned_addr learned my addr 172.21.15.31:0/746944782 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec338f7680 0x55ec338ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 -- 172.21.15.31:0/746944782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec338f7680 msgr2=0x55ec338ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec338f7680 0x55ec338ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 -- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 msgr2=0x55ec33901fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec33901fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.936+0000 7ff3f77fe700 1 -- 172.21.15.31:0/746944782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f81a3c30 con 0x55ec33902530 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0027b80 tx=0x7ff3f0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f935ac7280c6425c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff3daff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004230 con 0x55ec33902530 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec338eaeb0 con 0x55ec33902530 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff3f0017e80 con 0x55ec33902530 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3f0027e90 con 0x55ec33902530 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff4110b4b80 10 client.16599 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.937+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7ff3ec011440 con 0x55ec33902530 2023-10-06T21:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ec338eaeb0 con 0x55ec33902530 2023-10-06T21:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3f001a9d0 con 0x55ec33902530 2023-10-06T21:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 20 client.16599 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33911f80 0x55ec339143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 4 client.16599 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec33843db0 con 0x55ec33911f80 2023-10-06T21:25:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.938+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33911f80 0x55ec339143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.939+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33911f80 0x55ec339143a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e86dd755651929e server_cookie=257e04e252edb915 in_seq=0 out_seq=0 2023-10-06T21:25:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.939+0000 7ff3daff5700 10 client.16599 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff3daff5700 1 -- 172.21.15.31:0/746944782 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec33843db0 con 0x55ec33911f80 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff3daff5700 10 client.16599 handle_command_reply: tid=0 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 client.16599 shutdown 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 msgr2=0x55ec338ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0027b80 tx=0x7ff3f0004b40 comp rx=0 tx=0).stop 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 shutdown_connections 2023-10-06T21:25:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33911f80 0x55ec339143a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec33902530 0x55ec338ff920 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec33901fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/746944782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec338f7680 0x55ec338ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff3daff5700 0 client.16599 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 >> 172.21.15.31:0/746944782 conn(0x55ec3386c200 msgr2=0x55ec338f2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 shutdown_connections 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 1 -- 172.21.15.31:0/746944782 wait complete. 2023-10-06T21:25:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.940+0000 7ff4110b4b80 20 client.16599 trim_cache size 0 max 16384 2023-10-06T21:25:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 Processor -- start 2023-10-06T21:25:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 -- start start 2023-10-06T21:25:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec338fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec336cb8c0 con 0x55ec338f7680 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec338f47e0 con 0x55ec338f6d60 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.957+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec338ed420 con 0x55ec33902530 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:51454/0 (socket says 172.21.15.31:51454) 2023-10-06T21:25:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/806963368 learned_addr learned my addr 172.21.15.31:0/806963368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec338fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/806963368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 msgr2=0x55ec338fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec338fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/806963368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 msgr2=0x55ec33901b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/806963368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3365ec00 con 0x55ec338f6d60 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec338fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.958+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80cbde0 tx=0x7ff3e8086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1820e41fce42847 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff3ddffb700 1 -- 172.21.15.31:0/806963368 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8099ef0 con 0x55ec338f6d60 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff3ddffb700 1 -- 172.21.15.31:0/806963368 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3e8099d00 con 0x55ec338f6d60 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff3ddffb700 1 -- 172.21.15.31:0/806963368 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e809f7f0 con 0x55ec338f6d60 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 -- 172.21.15.31:0/806963368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 msgr2=0x55ec338f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80cbde0 tx=0x7ff3e8086040 comp rx=0 tx=0).stop 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 -- 172.21.15.31:0/806963368 shutdown_connections 2023-10-06T21:25:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec338fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f6d60 0x55ec338f9c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/806963368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f7680 0x55ec33901b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 -- 172.21.15.31:0/806963368 >> 172.21.15.31:0/806963368 conn(0x55ec3386c200 msgr2=0x55ec3390e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.959+0000 7ff4110b4b80 1 -- 172.21.15.31:0/806963368 shutdown_connections 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.960+0000 7ff4110b4b80 1 -- 172.21.15.31:0/806963368 wait complete. 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.960+0000 7ff4110b4b80 1 -- start start 2023-10-06T21:25:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.960+0000 7ff4110b4b80 1 Processor -- start 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f7680 0x55ec33907be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33908120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33909600 con 0x55ec338f6d60 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33965dc0 con 0x55ec338f7680 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec33965f50 con 0x55ec33902530 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff4110b4b80 10 client.0 _ll_register_callbacks cb 0x7ff3f0003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f7680 0x55ec33907be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52712/0 (socket says 172.21.15.31:52712) 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/356522062 learned_addr learned my addr 172.21.15.31:0/356522062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33908120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/356522062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 msgr2=0x55ec33908120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33908120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f7680 msgr2=0x55ec33907be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f7680 0x55ec33907be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.961+0000 7ff3f6ffd700 1 -- 172.21.15.31:0/356522062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e80095f0 con 0x55ec338f6d60 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3f77fe700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3f6ffd700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8099e70 tx=0x7ff3e80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9da2beafd8d5278 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3daff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80a39a0 con 0x55ec338f6d60 2023-10-06T21:25:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7ff3e809f7f0 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3ec004e20 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3e809b6a0 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff4110b4b80 10 client.16605 fetch_fsmap learned FSMap version 34 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.962+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec3369c1f0 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3e807e020 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3e8098d90 con 0x55ec338f6d60 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 20 client.16605 populate_metadata read hostname 'smithi031' 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33967240 0x55ec33969660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 4 client.16605 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec33843ad0 con 0x55ec33967240 2023-10-06T21:25:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.963+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33967240 0x55ec33969660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.964+0000 7ff3f67fc700 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33967240 0x55ec33969660 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4453a7468ec8ba90 server_cookie=adb3f8dbce6142f6 in_seq=0 out_seq=0 2023-10-06T21:25:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.964+0000 7ff3daff5700 10 client.16605 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff3daff5700 1 -- 172.21.15.31:0/356522062 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec33843ad0 con 0x55ec33967240 2023-10-06T21:25:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff3daff5700 10 client.16605 handle_command_reply: tid=0 2023-10-06T21:25:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 client.16605 shutdown 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 msgr2=0x55ec339076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8099e70 tx=0x7ff3e80057e0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 shutdown_connections 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec33902530 0x55ec33908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ec33967240 0x55ec33969660 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ec338f6d60 0x55ec339076a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 --2- 172.21.15.31:0/356522062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec338f7680 0x55ec33907be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 >> 172.21.15.31:0/356522062 conn(0x55ec3386c200 msgr2=0x55ec338fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff3daff5700 0 client.16605 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 shutdown_connections 2023-10-06T21:25:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.985+0000 7ff4110b4b80 1 -- 172.21.15.31:0/356522062 wait complete. 2023-10-06T21:25:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.986+0000 7ff4110b4b80 20 client.16605 trim_cache size 0 max 16384 2023-10-06T21:25:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:25:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:25:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:25:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.986+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 msgr2=0x7ff3e00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.986+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 0x7ff3e00a3ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0007a80 tx=0x7ff3f0007e40 comp rx=0 tx=0).stop 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.986+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 msgr2=0x7ff3f819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.986+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005740 tx=0x7ff3e8005770 comp rx=0 tx=0).stop 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 shutdown_connections 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3f8196b40 0x7ff3f8196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3e00a15f0 0x7ff3e00a3ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3f8100f40 0x7ff3f819cad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 --2- 172.21.15.31:0/731878906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f8106060 0x7ff3f819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 >> 172.21.15.31:0/731878906 conn(0x7ff3f80fc7c0 msgr2=0x7ff3f81049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 shutdown_connections 2023-10-06T21:25:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:25:50.987+0000 7ff3f7fff700 1 -- 172.21.15.31:0/731878906 wait complete. 2023-10-06T21:25:51.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 74614 1.83 MB/sec execute 101 sec latency 19.420 ms 2023-10-06T21:25:52.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:52 smithi146 ceph-mon[102809]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 788 KiB/s wr, 86 op/s 2023-10-06T21:25:52.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 74850 1.82 MB/sec execute 102 sec latency 16.547 ms 2023-10-06T21:25:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:52 smithi142 ceph-mon[102951]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 788 KiB/s wr, 86 op/s 2023-10-06T21:25:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:52 smithi031 ceph-mon[98539]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 788 KiB/s wr, 86 op/s 2023-10-06T21:25:53.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 75018 1.81 MB/sec execute 103 sec latency 18.099 ms 2023-10-06T21:25:54.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:54 smithi146 ceph-mon[102809]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-10-06T21:25:54.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 75271 1.79 MB/sec execute 104 sec latency 19.881 ms 2023-10-06T21:25:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:54 smithi142 ceph-mon[102951]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-10-06T21:25:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:54 smithi031 ceph-mon[98539]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-10-06T21:25:55.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 75636 1.79 MB/sec execute 105 sec latency 16.080 ms 2023-10-06T21:25:56.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:56 smithi146 ceph-mon[102809]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:56.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 76016 1.80 MB/sec execute 106 sec latency 19.098 ms 2023-10-06T21:25:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:56 smithi142 ceph-mon[102951]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:56 smithi031 ceph-mon[98539]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:57.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 76408 1.79 MB/sec execute 107 sec latency 16.664 ms 2023-10-06T21:25:58.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 76851 1.79 MB/sec execute 108 sec latency 18.576 ms 2023-10-06T21:25:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:58 smithi142 ceph-mon[102951]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:58 smithi031 ceph-mon[98539]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:58.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:58 smithi146 ceph-mon[102809]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 896 KiB/s wr, 91 op/s 2023-10-06T21:25:59.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 77139 1.78 MB/sec execute 109 sec latency 16.828 ms 2023-10-06T21:26:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:25:59 smithi031 ceph-mon[98539]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:26:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:25:59 smithi146 ceph-mon[102809]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:26:00.338 INFO:tasks.workunit.client.0.smithi031.stdout: 1 77739 1.81 MB/sec execute 110 sec latency 27.583 ms 2023-10-06T21:26:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:25:59 smithi142 ceph-mon[102951]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-10-06T21:26:01.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 77934 1.81 MB/sec execute 111 sec latency 24.137 ms 2023-10-06T21:26:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:02 smithi146 ceph-mon[102809]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 897 KiB/s wr, 91 op/s 2023-10-06T21:26:02.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 78322 1.83 MB/sec execute 112 sec latency 21.249 ms 2023-10-06T21:26:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:02 smithi142 ceph-mon[102951]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 897 KiB/s wr, 91 op/s 2023-10-06T21:26:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:02 smithi031 ceph-mon[98539]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 897 KiB/s wr, 91 op/s 2023-10-06T21:26:03.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 78489 1.82 MB/sec execute 113 sec latency 19.229 ms 2023-10-06T21:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 -- 172.21.15.31:0/2767335610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 msgr2=0x7f771c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 --2- 172.21.15.31:0/2767335610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c0736c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7718009990 tx=0x7f7718009ca0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 -- 172.21.15.31:0/2767335610 shutdown_connections 2023-10-06T21:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 --2- 172.21.15.31:0/2767335610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c10d6a0 0x7f771c10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 --2- 172.21.15.31:0/2767335610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c0736c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 --2- 172.21.15.31:0/2767335610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c073c00 0x7f771c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 -- 172.21.15.31:0/2767335610 >> 172.21.15.31:0/2767335610 conn(0x7f771c0fc930 msgr2=0x7f771c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 -- 172.21.15.31:0/2767335610 shutdown_connections 2023-10-06T21:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.148+0000 7f77239ff700 1 -- 172.21.15.31:0/2767335610 wait complete. 2023-10-06T21:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.149+0000 7f77239ff700 1 Processor -- start 2023-10-06T21:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.149+0000 7f77239ff700 1 -- start start 2023-10-06T21:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c073c00 0x7f771c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 0x7f771c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c19e2e0 con 0x7f771c0732e0 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c19e450 con 0x7f771c10d6a0 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f77239ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c19e610 con 0x7f771c073c00 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50916/0 (socket says 172.21.15.31:50916) 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f772179b700 1 -- 172.21.15.31:0/2418396575 learned_addr learned my addr 172.21.15.31:0/2418396575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 0x7f771c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f7720f9a700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c073c00 0x7f771c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f7721f9c700 1 -- 172.21.15.31:0/2418396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c073c00 msgr2=0x7f771c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.150+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c073c00 0x7f771c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f7721f9c700 1 -- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 msgr2=0x7f771c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f7721f9c700 1 -- 172.21.15.31:0/2418396575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180095f0 con 0x7f771c10d6a0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f772179b700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 0x7f771c19dca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f771400efc0 tx=0x7f771400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f77127fc700 1 -- 172.21.15.31:0/2418396575 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714018070 con 0x7f771c10d6a0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c19e890 con 0x7f771c10d6a0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f77127fc700 1 -- 172.21.15.31:0/2418396575 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771400f040 con 0x7f771c10d6a0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f77127fc700 1 -- 172.21.15.31:0/2418396575 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400cba0 con 0x7f771c10d6a0 2023-10-06T21:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.151+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c1a6130 con 0x7f771c10d6a0 2023-10-06T21:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.155+0000 7f77127fc700 1 -- 172.21.15.31:0/2418396575 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7714003a00 con 0x7f771c10d6a0 2023-10-06T21:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 Processor -- start 2023-10-06T21:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 -- start start 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e94400 con 0x563b00f2b520 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e85bc0 con 0x563b00f363d0 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.157+0000 7f773723fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e834e0 con 0x563b00f2ac00 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f77127fc700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 0x7f77080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57766/0 (socket says 172.21.15.31:57766) 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50930/0 (socket says 172.21.15.31:50930) 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 -- 172.21.15.31:0/436683511 learned_addr learned my addr 172.21.15.31:0/436683511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f77127fc700 1 -- 172.21.15.31:0/2418396575 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f77140ccb90 con 0x7f771c10d6a0 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 0x7f77080a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 -- 172.21.15.31:0/436683511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 msgr2=0x563b00f387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 -- 172.21.15.31:0/436683511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 msgr2=0x563b00f35e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f7720f9a700 1 -- 172.21.15.31:0/436683511 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b00ca9d80 con 0x563b00f2ac00 2023-10-06T21:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.158+0000 7f772179b700 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.159+0000 7f7720f9a700 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f770c0136b0 tx=0x7f770c009a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf80624e0d910d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.159+0000 7f7721f9c700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 0x7f77080a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fd0 tx=0x7f7714009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.159+0000 7f77077fe700 1 -- 172.21.15.31:0/436683511 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c004050 con 0x563b00f2ac00 2023-10-06T21:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.159+0000 7f77077fe700 1 -- 172.21.15.31:0/436683511 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f770c013e90 con 0x563b00f2ac00 2023-10-06T21:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.159+0000 7f77077fe700 1 -- 172.21.15.31:0/436683511 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c010430 con 0x563b00f2ac00 2023-10-06T21:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 -- 172.21.15.31:0/436683511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 msgr2=0x563b00f2afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f770c0136b0 tx=0x7f770c009a90 comp rx=0 tx=0).stop 2023-10-06T21:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 -- 172.21.15.31:0/436683511 shutdown_connections 2023-10-06T21:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f2afe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 --2- 172.21.15.31:0/436683511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f35e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 -- 172.21.15.31:0/436683511 >> 172.21.15.31:0/436683511 conn(0x563b00ea0200 msgr2=0x563b00f27080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 -- 172.21.15.31:0/436683511 shutdown_connections 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.160+0000 7f773723fb80 1 -- 172.21.15.31:0/436683511 wait complete. 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 -- start start 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 Processor -- start 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2b520 0x563b00f33660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e7edd0 con 0x563b00f2ac00 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e1b6b0 con 0x563b00f363d0 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.161+0000 7f773723fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00e1b350 con 0x563b00f2b520 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f773723fb80 10 client.0 _ll_register_callbacks cb 0x563b00c787a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50942/0 (socket says 172.21.15.31:50942) 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 -- 172.21.15.31:0/2584591379 learned_addr learned my addr 172.21.15.31:0/2584591379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f772179b700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2b520 0x563b00f33660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 -- 172.21.15.31:0/2584591379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2b520 msgr2=0x563b00f33660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2b520 0x563b00f33660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 -- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 msgr2=0x563b00f33ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f33ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.162+0000 7f7720f9a700 1 -- 172.21.15.31:0/2584591379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c1107d0 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7720f9a700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f770c009830 tx=0x7f770c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c945be4ade91e2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7704ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c023c00 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b00f1ed50 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f770c023d60 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f770c010430 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f773723fb80 10 client.16617 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f770c009e50 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.163+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b00f1ed50 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f770c010ce0 con 0x563b00f2ac00 2023-10-06T21:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 10 client.16617 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 20 client.16617 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f45fc0 0x563b00f483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 4 client.16617 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b00e77db0 con 0x563b00f45fc0 2023-10-06T21:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.164+0000 7f772179b700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f45fc0 0x563b00f483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.165+0000 7f772179b700 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f45fc0 0x563b00f483e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6befc8dce9ea2e server_cookie=d424b26099e65b7d in_seq=0 out_seq=0 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.165+0000 7f7704ff9700 10 client.16617 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.165+0000 7f7704ff9700 1 -- 172.21.15.31:0/2584591379 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563b00e77db0 con 0x563b00f45fc0 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.165+0000 7f7704ff9700 10 client.16617 handle_command_reply: tid=0 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 client.16617 shutdown 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 msgr2=0x563b00f33120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f770c009830 tx=0x7f770c005760 comp rx=0 tx=0).stop 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 shutdown_connections 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f33ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f45fc0 0x563b00f483e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2ac00 0x563b00f33120 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 --2- 172.21.15.31:0/2584591379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2b520 0x563b00f33660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 >> 172.21.15.31:0/2584591379 conn(0x563b00ea0200 msgr2=0x563b00f26c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f7704ff9700 0 client.16617 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 shutdown_connections 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 1 -- 172.21.15.31:0/2584591379 wait complete. 2023-10-06T21:26:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.166+0000 7f773723fb80 20 client.16617 trim_cache size 0 max 16384 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 Processor -- start 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 -- start start 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00cff8c0 con 0x563b00f2b520 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00f3d450 con 0x563b00f363d0 2023-10-06T21:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f773723fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00f2df50 con 0x563b00f2ac00 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50950/0 (socket says 172.21.15.31:50950) 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f772179b700 1 -- 172.21.15.31:0/3017180756 learned_addr learned my addr 172.21.15.31:0/3017180756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.186+0000 7f7720f9a700 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 -- 172.21.15.31:0/3017180756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 msgr2=0x563b00f323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 -- 172.21.15.31:0/3017180756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 msgr2=0x563b00f303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 -- 172.21.15.31:0/3017180756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b00c92c00 con 0x563b00f2b520 2023-10-06T21:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f772179b700 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f771800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74ed0109144039c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f7707fff700 1 -- 172.21.15.31:0/3017180756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771801d040 con 0x563b00f2b520 2023-10-06T21:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f7707fff700 1 -- 172.21.15.31:0/3017180756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771801e040 con 0x563b00f2b520 2023-10-06T21:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.187+0000 7f7707fff700 1 -- 172.21.15.31:0/3017180756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718005500 con 0x563b00f2b520 2023-10-06T21:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 -- 172.21.15.31:0/3017180756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 msgr2=0x563b00f3cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f771800b260 comp rx=0 tx=0).stop 2023-10-06T21:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 -- 172.21.15.31:0/3017180756 shutdown_connections 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f363d0 0x563b00f303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f2ac00 0x563b00f323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 --2- 172.21.15.31:0/3017180756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cf10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 -- 172.21.15.31:0/3017180756 >> 172.21.15.31:0/3017180756 conn(0x563b00ea0200 msgr2=0x563b00f2e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 -- 172.21.15.31:0/3017180756 shutdown_connections 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.188+0000 7f773723fb80 1 -- 172.21.15.31:0/3017180756 wait complete. 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.189+0000 7f773723fb80 1 -- start start 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.189+0000 7f773723fb80 1 Processor -- start 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 0x563b00f3c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f363d0 0x563b00f3af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00f3b510 con 0x563b00f2b520 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00f3b680 con 0x563b00f2ac00 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b00f3b840 con 0x563b00f363d0 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f773723fb80 10 client.0 _ll_register_callbacks cb 0x7f770c023750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f7720f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 0x563b00f3c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f772179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50958/0 (socket says 172.21.15.31:50958) 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f772179b700 1 -- 172.21.15.31:0/3606826394 learned_addr learned my addr 172.21.15.31:0/3606826394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.190+0000 7f7720f9a700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f363d0 0x563b00f3af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 -- 172.21.15.31:0/3606826394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f363d0 msgr2=0x563b00f3af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f363d0 0x563b00f3af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 -- 172.21.15.31:0/3606826394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 msgr2=0x563b00f3cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 -- 172.21.15.31:0/3606826394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771800bd60 con 0x563b00f2ac00 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f772179b700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7720f9a700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 0x563b00f3c720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f770c0058b0 tx=0x7f770c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281bf475e62db88d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7704ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0038b0 con 0x563b00f2ac00 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f770c013e90 con 0x563b00f2ac00 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.191+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f770c000e40 con 0x563b00f2ac00 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f770c023df0 con 0x563b00f2ac00 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 10 client.26351 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563b00cd01f0 con 0x563b00f2ac00 2023-10-06T21:26:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f770c013e90 con 0x563b00f2ac00 2023-10-06T21:26:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f770c0050f0 con 0x563b00f2ac00 2023-10-06T21:26:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 10 client.26351 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 20 client.26351 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f9ab20 0x563b00f9cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 4 client.26351 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.192+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b00e77ad0 con 0x563b00f9ab20 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.193+0000 7f772179b700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f9ab20 0x563b00f9cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.193+0000 7f772179b700 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f9ab20 0x563b00f9cf10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6aa4fe09458079 server_cookie=b1d1dac1499d0406 in_seq=0 out_seq=0 2023-10-06T21:26:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.193+0000 7f7704ff9700 10 client.26351 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f7704ff9700 1 -- 172.21.15.31:0/3606826394 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b00e77ad0 con 0x563b00f9ab20 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f7704ff9700 10 client.26351 handle_command_reply: tid=0 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 client.26351 shutdown 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 msgr2=0x563b00f3c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 0x563b00f3c720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f770c0058b0 tx=0x7f770c005760 comp rx=0 tx=0).stop 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 shutdown_connections 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563b00f363d0 0x563b00f3af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b00f2ac00 0x563b00f3c720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563b00f9ab20 0x563b00f9cf10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 --2- 172.21.15.31:0/3606826394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b00f2b520 0x563b00f3cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f7704ff9700 0 client.26351 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.218+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 >> 172.21.15.31:0/3606826394 conn(0x563b00ea0200 msgr2=0x563b00f33070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 shutdown_connections 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f773723fb80 1 -- 172.21.15.31:0/3606826394 wait complete. 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f773723fb80 20 client.26351 trim_cache size 0 max 16384 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:26:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 msgr2=0x7f77080a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 0x7f77080a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fd0 tx=0x7f7714009040 comp rx=0 tx=0).stop 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 msgr2=0x7f771c19dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 0x7f771c19dca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f771400efc0 tx=0x7f771400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 shutdown_connections 2023-10-06T21:26:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.219+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f771c10d6a0 0x7f771c19dca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77080a1670 0x7f77080a3b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f771c0732e0 0x7f771c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 --2- 172.21.15.31:0/2418396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f771c073c00 0x7f771c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 >> 172.21.15.31:0/2418396575 conn(0x7f771c0fc930 msgr2=0x7f771c10f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 shutdown_connections 2023-10-06T21:26:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:04.220+0000 7f77239ff700 1 -- 172.21.15.31:0/2418396575 wait complete. 2023-10-06T21:26:04.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:04 smithi146 ceph-mon[102809]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-10-06T21:26:04.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 78675 1.80 MB/sec execute 114 sec latency 21.929 ms 2023-10-06T21:26:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:04 smithi142 ceph-mon[102951]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-10-06T21:26:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:04 smithi031 ceph-mon[98539]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-10-06T21:26:05.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 79025 1.79 MB/sec execute 115 sec latency 16.892 ms 2023-10-06T21:26:06.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:06 smithi146 ceph-mon[102809]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:06.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 79352 1.79 MB/sec execute 116 sec latency 16.727 ms 2023-10-06T21:26:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:06 smithi142 ceph-mon[102951]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:06.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:06 smithi031 ceph-mon[98539]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:07.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 79779 1.80 MB/sec execute 117 sec latency 15.451 ms 2023-10-06T21:26:08.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 80183 1.79 MB/sec execute 118 sec latency 19.290 ms 2023-10-06T21:26:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:08 smithi142 ceph-mon[102951]: pgmap v159: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:08 smithi031 ceph-mon[98539]: pgmap v159: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:08 smithi146 ceph-mon[102809]: pgmap v159: 97 pgs: 97 active+clean; 60 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 909 KiB/s wr, 99 op/s 2023-10-06T21:26:09.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 80529 1.79 MB/sec execute 119 sec latency 16.980 ms 2023-10-06T21:26:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:09 smithi031 ceph-mon[98539]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2023-10-06T21:26:10.301 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:09 smithi146 ceph-mon[102809]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2023-10-06T21:26:10.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 81010 1.82 MB/sec execute 120 sec latency 19.461 ms 2023-10-06T21:26:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:09 smithi142 ceph-mon[102951]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2023-10-06T21:26:11.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 81401 1.82 MB/sec execute 121 sec latency 18.168 ms 2023-10-06T21:26:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:12 smithi146 ceph-mon[102809]: pgmap v161: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 948 KiB/s wr, 105 op/s 2023-10-06T21:26:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:12 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:26:12.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 81726 1.83 MB/sec execute 122 sec latency 17.802 ms 2023-10-06T21:26:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:12 smithi142 ceph-mon[102951]: pgmap v161: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 948 KiB/s wr, 105 op/s 2023-10-06T21:26:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:12 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:26:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:12 smithi031 ceph-mon[98539]: pgmap v161: 97 pgs: 97 active+clean; 61 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 948 KiB/s wr, 105 op/s 2023-10-06T21:26:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:12 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:26:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:13.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:26:13.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 81945 1.82 MB/sec execute 123 sec latency 19.195 ms 2023-10-06T21:26:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:13.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:26:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:26:14.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:14 smithi146 ceph-mon[102809]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-10-06T21:26:14.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 82113 1.81 MB/sec execute 124 sec latency 18.185 ms 2023-10-06T21:26:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:14 smithi142 ceph-mon[102951]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-10-06T21:26:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:14 smithi031 ceph-mon[98539]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-10-06T21:26:15.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 82391 1.80 MB/sec execute 125 sec latency 24.387 ms 2023-10-06T21:26:16.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 82750 1.80 MB/sec execute 126 sec latency 21.410 ms 2023-10-06T21:26:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:16 smithi142 ceph-mon[102951]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:16 smithi031 ceph-mon[98539]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:16 smithi146 ceph-mon[102809]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:17.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 83118 1.81 MB/sec execute 127 sec latency 24.511 ms 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 -- 172.21.15.31:0/3342750665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073ba0 msgr2=0x7f2c7c10d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 --2- 172.21.15.31:0/3342750665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c10d1f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009990 tx=0x7f2c70009ca0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 -- 172.21.15.31:0/3342750665 shutdown_connections 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 --2- 172.21.15.31:0/3342750665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c10d730 0x7f2c7c10fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 --2- 172.21.15.31:0/3342750665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c073280 0x7f2c7c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 --2- 172.21.15.31:0/3342750665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c10d1f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.393+0000 7f2c81bff700 1 -- 172.21.15.31:0/3342750665 >> 172.21.15.31:0/3342750665 conn(0x7f2c7c0fc9a0 msgr2=0x7f2c7c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.394+0000 7f2c81bff700 1 -- 172.21.15.31:0/3342750665 shutdown_connections 2023-10-06T21:26:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.394+0000 7f2c81bff700 1 -- 172.21.15.31:0/3342750665 wait complete. 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 Processor -- start 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 -- start start 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c10d730 0x7f2c7c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19e350 con 0x7f2c7c073280 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19e4c0 con 0x7f2c7c10d730 2023-10-06T21:26:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c81bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19e680 con 0x7f2c7c073ba0 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.395+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35560/0 (socket says 172.21.15.31:35560) 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/4103640997 learned_addr learned my addr 172.21.15.31:0/4103640997 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7bfff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c10d730 0x7f2c7c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7affd700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/4103640997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 msgr2=0x7f2c7c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/4103640997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c10d730 msgr2=0x7f2c7c19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c10d730 0x7f2c7c19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/4103640997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700095f0 con 0x7f2c7c073280 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7affd700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00d980 tx=0x7f2c6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c78ff9700 1 -- 172.21.15.31:0/4103640997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c004220 con 0x7f2c7c073280 2023-10-06T21:26:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c78ff9700 1 -- 172.21.15.31:0/4103640997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c6c007640 con 0x7f2c7c073280 2023-10-06T21:26:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.396+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c19e960 con 0x7f2c7c073280 2023-10-06T21:26:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.397+0000 7f2c78ff9700 1 -- 172.21.15.31:0/4103640997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0055e0 con 0x7f2c7c073280 2023-10-06T21:26:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.397+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c1a61a0 con 0x7f2c7c073280 2023-10-06T21:26:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.401+0000 7f2c78ff9700 1 -- 172.21.15.31:0/4103640997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2c6c004750 con 0x7f2c7c073280 2023-10-06T21:26:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 Processor -- start 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 -- start start 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c93e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9bf2400 con 0x55c2e9c89520 2023-10-06T21:26:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9be3bc0 con 0x55c2e9c943d0 2023-10-06T21:26:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.402+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9be14e0 con 0x55c2e9c88c00 2023-10-06T21:26:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c88fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51908/0 (socket says 172.21.15.31:51908) 2023-10-06T21:26:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7bfff700 1 -- 172.21.15.31:0/2502481961 learned_addr learned my addr 172.21.15.31:0/2502481961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c93e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c78ff9700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 0x7f2c640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 -- 172.21.15.31:0/2502481961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 msgr2=0x55c2e9c88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c88fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 -- 172.21.15.31:0/2502481961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 msgr2=0x55c2e9c93e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c93e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7affd700 1 -- 172.21.15.31:0/2502481961 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e9a07d80 con 0x55c2e9c943d0 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c78ff9700 1 -- 172.21.15.31:0/4103640997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2c6c0ccfc0 con 0x7f2c7c073280 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.403+0000 7f2c7bfff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 0x7f2c640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.404+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c967c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c700036c0 tx=0x7f2c70005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a982fabf065672 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.404+0000 7f2c61ffb700 1 -- 172.21.15.31:0/2502481961 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70009ee0 con 0x55c2e9c943d0 2023-10-06T21:26:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.404+0000 7f2c61ffb700 1 -- 172.21.15.31:0/2502481961 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c7001e040 con 0x55c2e9c943d0 2023-10-06T21:26:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.404+0000 7f2c61ffb700 1 -- 172.21.15.31:0/2502481961 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70004760 con 0x55c2e9c943d0 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c7bfff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 0x7f2c640a3ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c740099a0 tx=0x7f2c74004ad0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2502481961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 msgr2=0x55c2e9c967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c967c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c700036c0 tx=0x7f2c70005e80 comp rx=0 tx=0).stop 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2502481961 shutdown_connections 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c967c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2502481961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2502481961 >> 172.21.15.31:0/2502481961 conn(0x55c2e9bfe200 msgr2=0x55c2e9c85080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2502481961 shutdown_connections 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.405+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2502481961 wait complete. 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.406+0000 7f2c954eeb80 1 -- start start 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.406+0000 7f2c954eeb80 1 Processor -- start 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c93dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9bdcdd0 con 0x55c2e9c89520 2023-10-06T21:26:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9b796b0 con 0x55c2e9c943d0 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9b79350 con 0x55c2e9c88c00 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c954eeb80 10 client.0 _ll_register_callbacks cb 0x55c2e99d67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c93dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35570/0 (socket says 172.21.15.31:35570) 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7affd700 1 -- 172.21.15.31:0/251266392 learned_addr learned my addr 172.21.15.31:0/251266392 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7bfff700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 msgr2=0x55c2e9c915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/251266392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 msgr2=0x55c2e9c90fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/251266392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c110870 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.407+0000 7f2c7affd700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c93dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0ca660 tx=0x7f2c6c099ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=173d0af374401e60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c5affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c09a350 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2c6c004380 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2e9c7cd50 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c6c09a5f0 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c954eeb80 10 client.26314 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c2e99f0c00 con 0x55c2e9c88c00 2023-10-06T21:26:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.408+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2e9c7cd50 con 0x55c2e9c88c00 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c6c0993e0 con 0x55c2e9c88c00 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 10 client.26314 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 20 client.26314 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9ca3e80 0x55c2e9ca62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 4 client.26314 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2e9bd5db0 con 0x55c2e9ca3e80 2023-10-06T21:26:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.409+0000 7f2c7affd700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9ca3e80 0x55c2e9ca62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.410+0000 7f2c5affd700 10 client.26314 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.410+0000 7f2c7affd700 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9ca3e80 0x55c2e9ca62a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10332bfb5e4cbb78 server_cookie=336a31a33419bbcd in_seq=0 out_seq=0 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c5affd700 1 -- 172.21.15.31:0/251266392 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2e9bd5db0 con 0x55c2e9ca3e80 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c5affd700 10 client.26314 handle_command_reply: tid=0 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 client.26314 shutdown 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 msgr2=0x55c2e9c93dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c93dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0ca660 tx=0x7f2c6c099ab0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 shutdown_connections 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c93dd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9ca3e80 0x55c2e9ca62a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/251266392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c5affd700 0 client.26314 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 >> 172.21.15.31:0/251266392 conn(0x55c2e9bfe200 msgr2=0x55c2e9c84c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 shutdown_connections 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.411+0000 7f2c954eeb80 1 -- 172.21.15.31:0/251266392 wait complete. 2023-10-06T21:26:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.412+0000 7f2c954eeb80 20 client.26314 trim_cache size 0 max 16384 2023-10-06T21:26:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.428+0000 7f2c954eeb80 1 Processor -- start 2023-10-06T21:26:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.428+0000 7f2c954eeb80 1 -- start start 2023-10-06T21:26:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.428+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9a5d8c0 con 0x55c2e9c89520 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9c8ce70 con 0x55c2e9c943d0 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9ca64c0 con 0x55c2e9c88c00 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51916/0 (socket says 172.21.15.31:51916) 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/1229934490 learned_addr learned my addr 172.21.15.31:0/1229934490 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7affd700 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/1229934490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 msgr2=0x55c2e9c9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c9f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/1229934490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 msgr2=0x55c2e9c90fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.429+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/1229934490 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e9a2e1f0 con 0x55c2e9c88c00 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.430+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00d950 tx=0x7f2c6c0984a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d2ef9567952b0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.430+0000 7f2c627fc700 1 -- 172.21.15.31:0/1229934490 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c004380 con 0x55c2e9c88c00 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.430+0000 7f2c627fc700 1 -- 172.21.15.31:0/1229934490 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c6c09a5f0 con 0x55c2e9c88c00 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.430+0000 7f2c627fc700 1 -- 172.21.15.31:0/1229934490 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c09a150 con 0x55c2e9c88c00 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- 172.21.15.31:0/1229934490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 msgr2=0x55c2e9c967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00d950 tx=0x7f2c6c0984a0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- 172.21.15.31:0/1229934490 shutdown_connections 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c88c00 0x55c2e9c967c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/1229934490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c89520 0x55c2e9c90fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- 172.21.15.31:0/1229934490 >> 172.21.15.31:0/1229934490 conn(0x55c2e9bfe200 msgr2=0x55c2e9caabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- 172.21.15.31:0/1229934490 shutdown_connections 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- 172.21.15.31:0/1229934490 wait complete. 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 -- start start 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.431+0000 7f2c954eeb80 1 Processor -- start 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 0x55c2e9c96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 0x55c2e9c97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c97970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9c9b4a0 con 0x55c2e9c88c00 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9cf6c80 con 0x55c2e9c943d0 2023-10-06T21:26:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e9cf6dc0 con 0x55c2e9c89520 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c954eeb80 10 client.0 _ll_register_callbacks cb 0x55c2e99d67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 0x55c2e9c96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c7affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 0x55c2e9c97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c7b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 0x55c2e9c96ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35590/0 (socket says 172.21.15.31:35590) 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c7b7fe700 1 -- 172.21.15.31:0/2804606390 learned_addr learned my addr 172.21.15.31:0/2804606390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.432+0000 7f2c7bfff700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c97970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 -- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 msgr2=0x55c2e9c97970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c97970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 -- 172.21.15.31:0/2804606390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 msgr2=0x55c2e9c96ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 0x55c2e9c96ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 -- 172.21.15.31:0/2804606390 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2e99f0c00 con 0x55c2e9c89520 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c7affd700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 0x55c2e9c97430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55c2e9c92690 tx=0x7f2c70005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5acef9b429d411e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c5affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70035060 con 0x55c2e9c89520 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c7001e040 con 0x55c2e9c89520 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.433+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2c70031d40 con 0x55c2e9c89520 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c70005340 con 0x55c2e9c89520 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 10 client.26320 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c2e9a06250 con 0x55c2e9c89520 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c7003d080 con 0x55c2e9c89520 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c70021070 con 0x55c2e9c89520 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 10 client.26320 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 20 client.26320 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9cf8140 0x55c2e9cfa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 4 client.26320 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.434+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2e9bd5ad0 con 0x55c2e9cf8140 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.435+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9cf8140 0x55c2e9cfa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.435+0000 7f2c7b7fe700 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9cf8140 0x55c2e9cfa560 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c00132941d53be0 server_cookie=1cf86ecb808495f in_seq=0 out_seq=0 2023-10-06T21:26:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.435+0000 7f2c5affd700 10 client.26320 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c5affd700 1 -- 172.21.15.31:0/2804606390 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2e9bd5ad0 con 0x55c2e9cf8140 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c5affd700 10 client.26320 handle_command_reply: tid=0 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 client.26320 shutdown 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 msgr2=0x55c2e9c97430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 0x55c2e9c97430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55c2e9c92690 tx=0x7f2c70005c00 comp rx=0 tx=0).stop 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 shutdown_connections 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2e9c943d0 0x55c2e9c97970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c2e9cf8140 0x55c2e9cfa560 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c2e9c88c00 0x55c2e9c96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 --2- 172.21.15.31:0/2804606390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e9c89520 0x55c2e9c97430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c5affd700 0 client.26320 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.458+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 >> 172.21.15.31:0/2804606390 conn(0x55c2e9bfe200 msgr2=0x55c2e9c9f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 shutdown_connections 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c954eeb80 1 -- 172.21.15.31:0/2804606390 wait complete. 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c954eeb80 20 client.26320 trim_cache size 0 max 16384 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:26:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 msgr2=0x7f2c640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 0x7f2c640a3ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c740099a0 tx=0x7f2c74004ad0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 msgr2=0x7f2c7c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.459+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00d980 tx=0x7f2c6c00dc90 comp rx=0 tx=0).stop 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 shutdown_connections 2023-10-06T21:26:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c7c10d730 0x7f2c7c19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2c640a15f0 0x7f2c640a3ab0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c073280 0x7f2c7c199160 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 --2- 172.21.15.31:0/4103640997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7c073ba0 0x7f2c7c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 >> 172.21.15.31:0/4103640997 conn(0x7f2c7c0fc9a0 msgr2=0x7f2c7c10f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 shutdown_connections 2023-10-06T21:26:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:17.460+0000 7f2c81bff700 1 -- 172.21.15.31:0/4103640997 wait complete. 2023-10-06T21:26:18.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 83446 1.80 MB/sec execute 128 sec latency 26.705 ms 2023-10-06T21:26:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:18 smithi142 ceph-mon[102951]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:18 smithi031 ceph-mon[98539]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:18 smithi146 ceph-mon[102809]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 912 KiB/s wr, 93 op/s 2023-10-06T21:26:19.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 83815 1.78 MB/sec execute 129 sec latency 29.387 ms 2023-10-06T21:26:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:19 smithi031 ceph-mon[98539]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:26:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:19 smithi146 ceph-mon[102809]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:26:20.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 84114 1.79 MB/sec execute 130 sec latency 21.114 ms 2023-10-06T21:26:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:19 smithi142 ceph-mon[102951]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2023-10-06T21:26:21.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 84607 1.81 MB/sec execute 131 sec latency 16.131 ms 2023-10-06T21:26:22.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:22 smithi146 ceph-mon[102809]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 941 KiB/s wr, 94 op/s 2023-10-06T21:26:22.339 INFO:tasks.workunit.client.0.smithi031.stdout: 1 84961 1.81 MB/sec execute 132 sec latency 16.894 ms 2023-10-06T21:26:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:22 smithi142 ceph-mon[102951]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 941 KiB/s wr, 94 op/s 2023-10-06T21:26:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:22 smithi031 ceph-mon[98539]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 941 KiB/s wr, 94 op/s 2023-10-06T21:26:23.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 85192 1.82 MB/sec execute 133 sec latency 364.071 ms 2023-10-06T21:26:24.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 85461 1.82 MB/sec execute 134 sec latency 19.318 ms 2023-10-06T21:26:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:24 smithi142 ceph-mon[102951]: pgmap v167: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:26:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:24 smithi031 ceph-mon[98539]: pgmap v167: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:26:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:24 smithi146 ceph-mon[102809]: pgmap v167: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:26:25.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 85629 1.80 MB/sec execute 135 sec latency 18.549 ms 2023-10-06T21:26:26.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 85844 1.79 MB/sec execute 136 sec latency 24.508 ms 2023-10-06T21:26:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:26 smithi142 ceph-mon[102951]: pgmap v168: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:26 smithi031 ceph-mon[98539]: pgmap v168: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:26 smithi146 ceph-mon[102809]: pgmap v168: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:27.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 86232 1.79 MB/sec execute 137 sec latency 15.273 ms 2023-10-06T21:26:28.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 86604 1.80 MB/sec execute 138 sec latency 21.474 ms 2023-10-06T21:26:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:28 smithi142 ceph-mon[102951]: pgmap v169: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:28 smithi031 ceph-mon[98539]: pgmap v169: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:28 smithi146 ceph-mon[102809]: pgmap v169: 97 pgs: 97 active+clean; 64 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 818 KiB/s wr, 85 op/s 2023-10-06T21:26:29.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 86901 1.79 MB/sec execute 139 sec latency 27.144 ms 2023-10-06T21:26:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:29 smithi031 ceph-mon[98539]: pgmap v170: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-10-06T21:26:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:29 smithi146 ceph-mon[102809]: pgmap v170: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-10-06T21:26:30.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 87277 1.78 MB/sec execute 140 sec latency 27.396 ms 2023-10-06T21:26:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:29 smithi142 ceph-mon[102951]: pgmap v170: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 -- 172.21.15.31:0/3851922590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580073b70 msgr2=0x7fd580111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 --2- 172.21.15.31:0/3851922590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580073b70 0x7fd580111930 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009990 tx=0x7fd56c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 -- 172.21.15.31:0/3851922590 shutdown_connections 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 --2- 172.21.15.31:0/3851922590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580111e70 0x7fd580114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 --2- 172.21.15.31:0/3851922590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073250 0x7fd580073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 --2- 172.21.15.31:0/3851922590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580073b70 0x7fd580111930 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 -- 172.21.15.31:0/3851922590 >> 172.21.15.31:0/3851922590 conn(0x7fd5800fc950 msgr2=0x7fd5800fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 -- 172.21.15.31:0/3851922590 shutdown_connections 2023-10-06T21:26:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.616+0000 7fd586dff700 1 -- 172.21.15.31:0/3851922590 wait complete. 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.617+0000 7fd586dff700 1 Processor -- start 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 -- start start 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580073250 0x7fd5801a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073b70 0x7fd5801a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019da00 con 0x7fd580111e70 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019db70 con 0x7fd580073b70 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.618+0000 7fd586dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019dd30 con 0x7fd580073250 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd584b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580073250 0x7fd5801a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43364/0 (socket says 172.21.15.31:43364) 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 -- 172.21.15.31:0/3731660578 learned_addr learned my addr 172.21.15.31:0/3731660578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073b70 0x7fd5801a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 -- 172.21.15.31:0/3731660578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580073250 msgr2=0x7fd5801a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580073250 0x7fd5801a3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 -- 172.21.15.31:0/3731660578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073b70 msgr2=0x7fd5801a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073b70 0x7fd5801a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 -- 172.21.15.31:0/3731660578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574009520 con 0x7fd580111e70 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.619+0000 7fd58539c700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd578007ec0 tx=0x7fd57800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.620+0000 7fd57dffb700 1 -- 172.21.15.31:0/3731660578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578017070 con 0x7fd580111e70 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.620+0000 7fd57dffb700 1 -- 172.21.15.31:0/3731660578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd57800f040 con 0x7fd580111e70 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.620+0000 7fd57dffb700 1 -- 172.21.15.31:0/3731660578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578004280 con 0x7fd580111e70 2023-10-06T21:26:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.620+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c0095f0 con 0x7fd580111e70 2023-10-06T21:26:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.620+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5801aa340 con 0x7fd580111e70 2023-10-06T21:26:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.625+0000 7fd57dffb700 1 -- 172.21.15.31:0/3731660578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd578003a00 con 0x7fd580111e70 2023-10-06T21:26:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 Processor -- start 2023-10-06T21:26:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 -- start start 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fcffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fdae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd57dffb700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 0x7fd5700a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622f39400 con 0x558622fdb3d0 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622f2abc0 con 0x558622fcfc00 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622f284e0 con 0x558622fd0520 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.626+0000 7fd57dffb700 1 -- 172.21.15.31:0/3731660578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd5780cc720 con 0x7fd580111e70 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd57ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fcffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 0x7fd5700a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43368/0 (socket says 172.21.15.31:43368) 2023-10-06T21:26:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 -- 172.21.15.31:0/2224292455 learned_addr learned my addr 172.21.15.31:0/2224292455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd584b9b700 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fdae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 -- 172.21.15.31:0/2224292455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 msgr2=0x558622fdae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fdae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 -- 172.21.15.31:0/2224292455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 msgr2=0x558622fcffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fcffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.627+0000 7fd58539c700 1 -- 172.21.15.31:0/2224292455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558622d4ed80 con 0x558622fdb3d0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd58539c700 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd5780090c0 tx=0x7fd5780995a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cb64bfd55c7b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 0x7fd5700a3ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005b40 tx=0x7fd56c005a90 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd56affd700 1 -- 172.21.15.31:0/2224292455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780641b0 con 0x558622fdb3d0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd56affd700 1 -- 172.21.15.31:0/2224292455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd578003cb0 con 0x558622fdb3d0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd56affd700 1 -- 172.21.15.31:0/2224292455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57809d830 con 0x558622fdb3d0 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.628+0000 7fd59a6edb80 1 -- 172.21.15.31:0/2224292455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 msgr2=0x558622fdd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd5780090c0 tx=0x7fd5780995a0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 -- 172.21.15.31:0/2224292455 shutdown_connections 2023-10-06T21:26:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fdd7c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fcffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/2224292455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fdae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 -- 172.21.15.31:0/2224292455 >> 172.21.15.31:0/2224292455 conn(0x558622f45200 msgr2=0x558622fcc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 -- 172.21.15.31:0/2224292455 shutdown_connections 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 -- 172.21.15.31:0/2224292455 wait complete. 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.629+0000 7fd59a6edb80 1 -- start start 2023-10-06T21:26:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 Processor -- start 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fdae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fd8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622f23dd0 con 0x558622fdb3d0 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622ec06b0 con 0x558622fcfc00 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622ec0350 con 0x558622fd0520 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.630+0000 7fd59a6edb80 10 client.0 _ll_register_callbacks cb 0x558622d1d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd57ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fd8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43382/0 (socket says 172.21.15.31:43382) 2023-10-06T21:26:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 -- 172.21.15.31:0/978335928 learned_addr learned my addr 172.21.15.31:0/978335928 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd584b9b700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fdae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 -- 172.21.15.31:0/978335928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 msgr2=0x558622fd8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fd8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 -- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 msgr2=0x558622fdae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fdae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 -- 172.21.15.31:0/978335928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5801aa340 con 0x558622fdb3d0 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd584b9b700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fdae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd58539c700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd578003e20 tx=0x7fd578004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11103f95bf3c7684 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.631+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57809d830 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd5780996e0 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558622fc3d50 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd578099940 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 10 client.16653 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558622d37c00 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558622fc3d50 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5780a1550 con 0x558622fdb3d0 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 10 client.16653 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 20 client.16653 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x558622fea9d0 0x558622fecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 4 client.16653 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.632+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558622f1cdb0 con 0x558622fea9d0 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.633+0000 7fd584b9b700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x558622fea9d0 0x558622fecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.633+0000 7fd584b9b700 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x558622fea9d0 0x558622fecdc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a757213bf54992b server_cookie=4c59d9390c224a64 in_seq=0 out_seq=0 2023-10-06T21:26:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.633+0000 7fd55ffff700 10 client.16653 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.634+0000 7fd55ffff700 1 -- 172.21.15.31:0/978335928 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558622f1cdb0 con 0x558622fea9d0 2023-10-06T21:26:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.634+0000 7fd55ffff700 10 client.16653 handle_command_reply: tid=0 2023-10-06T21:26:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.634+0000 7fd59a6edb80 1 client.16653 shutdown 2023-10-06T21:26:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.634+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 msgr2=0x558622fd87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.634+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd578003e20 tx=0x7fd578004ad0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 shutdown_connections 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x558622fea9d0 0x558622fecdc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fdb3d0 0x558622fd87d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fcfc00 0x558622fdae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd55ffff700 0 client.16653 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/978335928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fd8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 >> 172.21.15.31:0/978335928 conn(0x558622f45200 msgr2=0x558622fcbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 shutdown_connections 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 1 -- 172.21.15.31:0/978335928 wait complete. 2023-10-06T21:26:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.635+0000 7fd59a6edb80 20 client.16653 trim_cache size 0 max 16384 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 Processor -- start 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 -- start start 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fe1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fdb3d0 0x558622fe7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622da48c0 con 0x558622fcfc00 2023-10-06T21:26:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622fc8860 con 0x558622fdb3d0 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.654+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622fccdf0 con 0x558622fd0520 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43390/0 (socket says 172.21.15.31:43390) 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 -- 172.21.15.31:0/3249018903 learned_addr learned my addr 172.21.15.31:0/3249018903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fe1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd58539c700 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fdb3d0 0x558622fe7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 -- 172.21.15.31:0/3249018903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 msgr2=0x558622fe1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fe1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 -- 172.21.15.31:0/3249018903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fdb3d0 msgr2=0x558622fe7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fdb3d0 0x558622fe7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 -- 172.21.15.31:0/3249018903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558622d751f0 con 0x558622fcfc00 2023-10-06T21:26:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.655+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x558622fed6c0 tx=0x7fd574012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173a6643193631d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.656+0000 7fd56b7fe700 1 -- 172.21.15.31:0/3249018903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740122d0 con 0x558622fcfc00 2023-10-06T21:26:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.656+0000 7fd56b7fe700 1 -- 172.21.15.31:0/3249018903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd57401ee40 con 0x558622fcfc00 2023-10-06T21:26:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.656+0000 7fd56b7fe700 1 -- 172.21.15.31:0/3249018903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574004bb0 con 0x558622fcfc00 2023-10-06T21:26:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3249018903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 msgr2=0x558622fd2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x558622fed6c0 tx=0x7fd574012730 comp rx=0 tx=0).stop 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3249018903 shutdown_connections 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fdb3d0 0x558622fe7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd2b70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3249018903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fd0520 0x558622fe1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3249018903 >> 172.21.15.31:0/3249018903 conn(0x558622f45200 msgr2=0x558622fca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3249018903 shutdown_connections 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3249018903 wait complete. 2023-10-06T21:26:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.657+0000 7fd59a6edb80 1 -- start start 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.658+0000 7fd59a6edb80 1 Processor -- start 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.658+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.658+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fd0520 0x558622fd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.658+0000 7fd59a6edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fdb3d0 0x558622fd4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622fd4890 con 0x558622fcfc00 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622fd4a00 con 0x558622fd0520 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd59a6edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622fd4bc0 con 0x558622fdb3d0 2023-10-06T21:26:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd59a6edb80 10 client.0 _ll_register_callbacks cb 0x7fd56c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43402/0 (socket says 172.21.15.31:43402) 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 -- 172.21.15.31:0/3618485421 learned_addr learned my addr 172.21.15.31:0/3618485421 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fd0520 0x558622fd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd58539c700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fdb3d0 0x558622fd4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 -- 172.21.15.31:0/3618485421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fdb3d0 msgr2=0x558622fd4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fdb3d0 0x558622fd4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 -- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fd0520 msgr2=0x558622fd5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fd0520 0x558622fd5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.659+0000 7fd584b9b700 1 -- 172.21.15.31:0/3618485421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57401eab0 con 0x558622fcfc00 2023-10-06T21:26:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd584b9b700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd5740094f0 tx=0x7fd57401e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d606487f1505fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:26:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574012b90 con 0x558622fcfc00 2023-10-06T21:26:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd578003cb0 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd574012cf0 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd574012eb0 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd59a6edb80 10 client.16659 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558622d4d250 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.660+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd578003cb0 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd574004b70 con 0x558622fcfc00 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 10 client.16659 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 20 client.16659 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55862303f640 0x558623041a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 4 client.16659 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:26:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558622f1cad0 con 0x55862303f640 2023-10-06T21:26:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.661+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55862303f640 0x558623041a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.662+0000 7fd57ffff700 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55862303f640 0x558623041a60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d105ff2a7d51e8ae server_cookie=78fab97ac56552e5 in_seq=0 out_seq=0 2023-10-06T21:26:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.662+0000 7fd55ffff700 10 client.16659 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd55ffff700 1 -- 172.21.15.31:0/3618485421 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558622f1cad0 con 0x55862303f640 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd55ffff700 10 client.16659 handle_command_reply: tid=0 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 client.16659 shutdown 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 msgr2=0x558622fd5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd5740094f0 tx=0x7fd57401e600 comp rx=0 tx=0).stop 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 shutdown_connections 2023-10-06T21:26:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558622fdb3d0 0x558622fd4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55862303f640 0x558623041a60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558622fcfc00 0x558622fd5520 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 --2- 172.21.15.31:0/3618485421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558622fd0520 0x558622fd5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd55ffff700 0 client.16659 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 >> 172.21.15.31:0/3618485421 conn(0x558622f45200 msgr2=0x558622fe7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.685+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 shutdown_connections 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd59a6edb80 1 -- 172.21.15.31:0/3618485421 wait complete. 2023-10-06T21:26:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd59a6edb80 20 client.16659 trim_cache size 0 max 16384 2023-10-06T21:26:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:26:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:26:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:26:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 msgr2=0x7fd5700a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 0x7fd5700a3ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005b40 tx=0x7fd56c005a90 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 msgr2=0x7fd58019d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.686+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd578007ec0 tx=0x7fd57800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 shutdown_connections 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5700a15f0 0x7fd5700a3ab0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd580111e70 0x7fd58019d390 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd580073250 0x7fd5801a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 --2- 172.21.15.31:0/3731660578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd580073b70 0x7fd5801a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 >> 172.21.15.31:0/3731660578 conn(0x7fd5800fc950 msgr2=0x7fd5800fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 shutdown_connections 2023-10-06T21:26:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:30.687+0000 7fd586dff700 1 -- 172.21.15.31:0/3731660578 wait complete. 2023-10-06T21:26:31.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 87618 1.78 MB/sec execute 141 sec latency 30.226 ms 2023-10-06T21:26:32.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:32 smithi146 ceph-mon[102809]: pgmap v171: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 904 KiB/s wr, 82 op/s 2023-10-06T21:26:32.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 88106 1.80 MB/sec execute 142 sec latency 18.372 ms 2023-10-06T21:26:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:32 smithi142 ceph-mon[102951]: pgmap v171: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 904 KiB/s wr, 82 op/s 2023-10-06T21:26:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:32 smithi031 ceph-mon[98539]: pgmap v171: 97 pgs: 97 active+clean; 66 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 904 KiB/s wr, 82 op/s 2023-10-06T21:26:33.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 88493 1.80 MB/sec execute 143 sec latency 15.977 ms 2023-10-06T21:26:34.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 88835 1.81 MB/sec execute 144 sec latency 19.290 ms 2023-10-06T21:26:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:34 smithi142 ceph-mon[102951]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:26:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:34 smithi031 ceph-mon[98539]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:26:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:34 smithi146 ceph-mon[102809]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:26:35.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 89049 1.81 MB/sec execute 145 sec latency 22.016 ms 2023-10-06T21:26:36.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 89232 1.80 MB/sec execute 146 sec latency 18.821 ms 2023-10-06T21:26:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:36 smithi142 ceph-mon[102951]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:36 smithi031 ceph-mon[98539]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:36 smithi146 ceph-mon[102809]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:37.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 89429 1.79 MB/sec execute 147 sec latency 246.682 ms 2023-10-06T21:26:38.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 89785 1.79 MB/sec execute 148 sec latency 20.264 ms 2023-10-06T21:26:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:38 smithi142 ceph-mon[102951]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:38 smithi031 ceph-mon[98539]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:38 smithi146 ceph-mon[102809]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 862 KiB/s wr, 92 op/s 2023-10-06T21:26:39.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 90151 1.80 MB/sec execute 149 sec latency 23.594 ms 2023-10-06T21:26:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:39 smithi031 ceph-mon[98539]: pgmap v175: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-10-06T21:26:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:39 smithi146 ceph-mon[102809]: pgmap v175: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-10-06T21:26:40.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 90439 1.79 MB/sec execute 150 sec latency 29.068 ms 2023-10-06T21:26:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:39 smithi142 ceph-mon[102951]: pgmap v175: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2023-10-06T21:26:41.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 90825 1.78 MB/sec execute 151 sec latency 25.709 ms 2023-10-06T21:26:42.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 91175 1.78 MB/sec execute 152 sec latency 25.602 ms 2023-10-06T21:26:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:42 smithi142 ceph-mon[102951]: pgmap v176: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 838 KiB/s wr, 87 op/s 2023-10-06T21:26:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:42 smithi031 ceph-mon[98539]: pgmap v176: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 838 KiB/s wr, 87 op/s 2023-10-06T21:26:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:42 smithi146 ceph-mon[102809]: pgmap v176: 97 pgs: 97 active+clean; 69 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 838 KiB/s wr, 87 op/s 2023-10-06T21:26:43.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 91655 1.80 MB/sec execute 153 sec latency 19.082 ms 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 -- 172.21.15.31:0/4115490153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 msgr2=0x7f7938073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 --2- 172.21.15.31:0/4115490153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f7938073660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7928009990 tx=0x7f7928009ca0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 -- 172.21.15.31:0/4115490153 shutdown_connections 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 --2- 172.21.15.31:0/4115490153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f793810d700 0x7f793810fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 --2- 172.21.15.31:0/4115490153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f7938073660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 --2- 172.21.15.31:0/4115490153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7938073ba0 0x7f793810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.846+0000 7f793e1ff700 1 -- 172.21.15.31:0/4115490153 >> 172.21.15.31:0/4115490153 conn(0x7f79380fc930 msgr2=0x7f79380fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.847+0000 7f793e1ff700 1 -- 172.21.15.31:0/4115490153 shutdown_connections 2023-10-06T21:26:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.847+0000 7f793e1ff700 1 -- 172.21.15.31:0/4115490153 wait complete. 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 Processor -- start 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 -- start start 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f79381990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7938073ba0 0x7f7938199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793819e2d0 con 0x7f793810d700 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793819e440 con 0x7f7938073280 2023-10-06T21:26:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.848+0000 7f793e1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793819e600 con 0x7f7938073ba0 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f79377fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f79381990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46620/0 (socket says 172.21.15.31:46620) 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 -- 172.21.15.31:0/1581392043 learned_addr learned my addr 172.21.15.31:0/1581392043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7938073ba0 0x7f7938199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 -- 172.21.15.31:0/1581392043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7938073ba0 msgr2=0x7f7938199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7938073ba0 0x7f7938199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 -- 172.21.15.31:0/1581392043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 msgr2=0x7f79381990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f79381990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 -- 172.21.15.31:0/1581392043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280095f0 con 0x7f793810d700 2023-10-06T21:26:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f79377fe700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f79381990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.849+0000 7f7937fff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f793000ef90 tx=0x7f793000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.850+0000 7f7934ff9700 1 -- 172.21.15.31:0/1581392043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930018070 con 0x7f793810d700 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.850+0000 7f7934ff9700 1 -- 172.21.15.31:0/1581392043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f793000f040 con 0x7f793810d700 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.850+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793819e880 con 0x7f793810d700 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.850+0000 7f7934ff9700 1 -- 172.21.15.31:0/1581392043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000cc40 con 0x7f793810d700 2023-10-06T21:26:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.850+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79381a6120 con 0x7f793810d700 2023-10-06T21:26:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.854+0000 7f7934ff9700 1 -- 172.21.15.31:0/1581392043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7930003a00 con 0x7f793810d700 2023-10-06T21:26:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.855+0000 7f7951ab2b80 1 Processor -- start 2023-10-06T21:26:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 -- start start 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e940e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e89f400 con 0x55737e935c00 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e890bc0 con 0x55737e936520 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e88e4e0 con 0x55737e9413d0 2023-10-06T21:26:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.856+0000 7f7934ff9700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 0x7f79200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7934ff9700 1 -- 172.21.15.31:0/1581392043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f79300cccf0 con 0x7f793810d700 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46628/0 (socket says 172.21.15.31:46628) 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 -- 172.21.15.31:0/2734620480 learned_addr learned my addr 172.21.15.31:0/2734620480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7937fff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 0x7f79200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f79377fe700 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e940e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 -- 172.21.15.31:0/2734620480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 msgr2=0x55737e9437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 -- 172.21.15.31:0/2734620480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 msgr2=0x55737e940e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e940e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.857+0000 7f7936ffd700 1 -- 172.21.15.31:0/2734620480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e6b4d80 con 0x55737e935c00 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.858+0000 7f7937fff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 0x7f79200a3ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79300ca3f0 tx=0x7f7930009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.858+0000 7f7936ffd700 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f792c0138c0 tx=0x7f792c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e6c5fc780825cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.858+0000 7f791dffb700 1 -- 172.21.15.31:0/2734620480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c013170 con 0x55737e935c00 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.858+0000 7f791dffb700 1 -- 172.21.15.31:0/2734620480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f792c0132d0 con 0x55737e935c00 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.858+0000 7f791dffb700 1 -- 172.21.15.31:0/2734620480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0055c0 con 0x55737e935c00 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 -- 172.21.15.31:0/2734620480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 msgr2=0x55737e935fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f792c0138c0 tx=0x7f792c009260 comp rx=0 tx=0).stop 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 -- 172.21.15.31:0/2734620480 shutdown_connections 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e935fe0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/2734620480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e940e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.859+0000 7f7951ab2b80 1 -- 172.21.15.31:0/2734620480 >> 172.21.15.31:0/2734620480 conn(0x55737e8ab200 msgr2=0x55737e932080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.860+0000 7f7951ab2b80 1 -- 172.21.15.31:0/2734620480 shutdown_connections 2023-10-06T21:26:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.860+0000 7f7951ab2b80 1 -- 172.21.15.31:0/2734620480 wait complete. 2023-10-06T21:26:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.860+0000 7f7951ab2b80 1 -- start start 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 Processor -- start 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e940dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e93e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e889dd0 con 0x55737e9413d0 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8266b0 con 0x55737e936520 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e826350 con 0x55737e935c00 2023-10-06T21:26:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.861+0000 7f7951ab2b80 10 client.0 _ll_register_callbacks cb 0x55737e6837a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e940dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:38542/0 (socket says 172.21.15.31:38542) 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 -- 172.21.15.31:0/1202328591 learned_addr learned my addr 172.21.15.31:0/1202328591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e93e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 -- 172.21.15.31:0/1202328591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 msgr2=0x55737e940dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e940dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 -- 172.21.15.31:0/1202328591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 msgr2=0x55737e93e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e93e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 -- 172.21.15.31:0/1202328591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c01b040 con 0x55737e936520 2023-10-06T21:26:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e93e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f79377fe700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbda33d5759d10b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.862+0000 7f7916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792801d070 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f7928004910 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55737e929d50 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7928004ad0 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7951ab2b80 10 client.26396 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55737e69dc00 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.863+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55737e929d50 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7928024450 con 0x55737e936520 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 10 client.26396 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 10 client.26396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 20 client.26396 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e950dc0 0x55737e9531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 4 client.26396 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55737e882db0 con 0x55737e950dc0 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.864+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e950dc0 0x55737e9531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.865+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e950dc0 0x55737e9531e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5486216911dfc0a6 server_cookie=c547b24d36fa4c1b in_seq=0 out_seq=0 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.865+0000 7f7916ffd700 10 client.26396 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.865+0000 7f7916ffd700 1 -- 172.21.15.31:0/1202328591 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55737e882db0 con 0x55737e950dc0 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.865+0000 7f7916ffd700 10 client.26396 handle_command_reply: tid=0 2023-10-06T21:26:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.865+0000 7f7951ab2b80 1 client.26396 shutdown 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 msgr2=0x55737e93e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005270 comp rx=0 tx=0).stop 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 shutdown_connections 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e950dc0 0x55737e9531e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e93e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e940dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1202328591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e93e050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7916ffd700 0 client.26396 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 >> 172.21.15.31:0/1202328591 conn(0x55737e8ab200 msgr2=0x55737e931c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 shutdown_connections 2023-10-06T21:26:43.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1202328591 wait complete. 2023-10-06T21:26:43.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.866+0000 7f7951ab2b80 20 client.26396 trim_cache size 0 max 16384 2023-10-06T21:26:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.883+0000 7f7951ab2b80 1 Processor -- start 2023-10-06T21:26:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.883+0000 7f7951ab2b80 1 -- start start 2023-10-06T21:26:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e947eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e939430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e70a8c0 con 0x55737e9413d0 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e948420 con 0x55737e936520 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e939f40 con 0x55737e935c00 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f79377fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e939430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f79377fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:38556/0 (socket says 172.21.15.31:38556) 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f79377fe700 1 -- 172.21.15.31:0/1735984598 learned_addr learned my addr 172.21.15.31:0/1735984598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.884+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e947eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 -- 172.21.15.31:0/1735984598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 msgr2=0x55737e947eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e947eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 -- 172.21.15.31:0/1735984598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 msgr2=0x55737e939430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e939430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 -- 172.21.15.31:0/1735984598 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e6db1f0 con 0x55737e936520 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f7936ffd700 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e939430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f79377fe700 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7928005930 tx=0x7f7928005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da24991848ef3ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f791e7fc700 1 -- 172.21.15.31:0/1735984598 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792801d040 con 0x55737e936520 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f791e7fc700 1 -- 172.21.15.31:0/1735984598 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f792801e040 con 0x55737e936520 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.885+0000 7f791e7fc700 1 -- 172.21.15.31:0/1735984598 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928004eb0 con 0x55737e936520 2023-10-06T21:26:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1735984598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 msgr2=0x55737e94c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7928005930 tx=0x7f7928005670 comp rx=0 tx=0).stop 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1735984598 shutdown_connections 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e9413d0 0x55737e939430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e935c00 0x55737e947eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/1735984598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e94c610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1735984598 >> 172.21.15.31:0/1735984598 conn(0x55737e8ab200 msgr2=0x55737e9307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1735984598 shutdown_connections 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- 172.21.15.31:0/1735984598 wait complete. 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.886+0000 7f7951ab2b80 1 -- start start 2023-10-06T21:26:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 Processor -- start 2023-10-06T21:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e9445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9a40e0 con 0x55737e935c00 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9a4220 con 0x55737e936520 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9a4390 con 0x55737e9413d0 2023-10-06T21:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.887+0000 7f7951ab2b80 10 client.0 _ll_register_callbacks cb 0x7f792800bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46644/0 (socket says 172.21.15.31:46644) 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 -- 172.21.15.31:0/4120872360 learned_addr learned my addr 172.21.15.31:0/4120872360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f79377fe700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e9445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 -- 172.21.15.31:0/4120872360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 msgr2=0x55737e9a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 -- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 msgr2=0x55737e9445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e9445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 -- 172.21.15.31:0/4120872360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c02a040 con 0x55737e935c00 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7936ffd700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f792c013eb0 tx=0x7f792c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72195f8677392da9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.888+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c027070 con 0x55737e935c00 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f792c0132d0 con 0x55737e935c00 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f792c02f3a0 con 0x55737e935c00 2023-10-06T21:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f792c025040 con 0x55737e935c00 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7951ab2b80 10 client.16677 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55737e69dc00 con 0x55737e935c00 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f792c03e020 con 0x55737e935c00 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.889+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f792c0055b0 con 0x55737e935c00 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 20 client.16677 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e9a5a00 0x55737e9a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 4 client.16677 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55737e882ad0 con 0x55737e9a5a00 2023-10-06T21:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.890+0000 7f79377fe700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e9a5a00 0x55737e9a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.891+0000 7f79377fe700 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e9a5a00 0x55737e9a7df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176f6e1c4b8ec36e server_cookie=33441c272c18d028 in_seq=0 out_seq=0 2023-10-06T21:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.891+0000 7f7916ffd700 10 client.16677 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7916ffd700 1 -- 172.21.15.31:0/4120872360 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55737e882ad0 con 0x55737e9a5a00 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7916ffd700 10 client.16677 handle_command_reply: tid=0 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 client.16677 shutdown 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 msgr2=0x55737e944080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f792c013eb0 tx=0x7f792c022040 comp rx=0 tx=0).stop 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 shutdown_connections 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55737e9413d0 0x55737e9a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55737e9a5a00 0x55737e9a7df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55737e935c00 0x55737e944080 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 --2- 172.21.15.31:0/4120872360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55737e936520 0x55737e9445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7916ffd700 0 client.16677 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 >> 172.21.15.31:0/4120872360 conn(0x55737e8ab200 msgr2=0x55737e939430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 shutdown_connections 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 1 -- 172.21.15.31:0/4120872360 wait complete. 2023-10-06T21:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.913+0000 7f7951ab2b80 20 client.16677 trim_cache size 0 max 16384 2023-10-06T21:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 msgr2=0x7f79200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 0x7f79200a3ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79300ca3f0 tx=0x7f7930009040 comp rx=0 tx=0).stop 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 msgr2=0x7f793819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f793000ef90 tx=0x7f793000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 shutdown_connections 2023-10-06T21:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f79200a15f0 0x7f79200a3ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793810d700 0x7f793819dc90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7938073280 0x7f79381990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 --2- 172.21.15.31:0/1581392043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7938073ba0 0x7f7938199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 >> 172.21.15.31:0/1581392043 conn(0x7f79380fc930 msgr2=0x7f793810f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 shutdown_connections 2023-10-06T21:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:43.914+0000 7f793e1ff700 1 -- 172.21.15.31:0/1581392043 wait complete. 2023-10-06T21:26:44.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 92042 1.80 MB/sec execute 154 sec latency 16.982 ms 2023-10-06T21:26:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:44 smithi142 ceph-mon[102951]: pgmap v177: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2023-10-06T21:26:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:44 smithi031 ceph-mon[98539]: pgmap v177: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2023-10-06T21:26:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:44 smithi146 ceph-mon[102809]: pgmap v177: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2023-10-06T21:26:45.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 92385 1.81 MB/sec execute 155 sec latency 19.170 ms 2023-10-06T21:26:46.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 92596 1.80 MB/sec execute 156 sec latency 17.968 ms 2023-10-06T21:26:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:46 smithi142 ceph-mon[102951]: pgmap v178: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:46 smithi031 ceph-mon[98539]: pgmap v178: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:46 smithi146 ceph-mon[102809]: pgmap v178: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:47.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 92771 1.79 MB/sec execute 157 sec latency 21.786 ms 2023-10-06T21:26:48.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 93054 1.79 MB/sec execute 158 sec latency 23.291 ms 2023-10-06T21:26:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:48 smithi142 ceph-mon[102951]: pgmap v179: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:48 smithi031 ceph-mon[98539]: pgmap v179: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:48.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:48 smithi146 ceph-mon[102809]: pgmap v179: 97 pgs: 97 active+clean; 69 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 785 KiB/s wr, 84 op/s 2023-10-06T21:26:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 93400 1.78 MB/sec execute 159 sec latency 21.326 ms 2023-10-06T21:26:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:49 smithi031 ceph-mon[98539]: pgmap v180: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:26:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:49 smithi146 ceph-mon[102809]: pgmap v180: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:26:50.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 93762 1.79 MB/sec execute 160 sec latency 26.264 ms 2023-10-06T21:26:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:49 smithi142 ceph-mon[102951]: pgmap v180: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:26:51.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 94097 1.78 MB/sec execute 161 sec latency 27.726 ms 2023-10-06T21:26:52.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 94470 1.78 MB/sec execute 162 sec latency 25.418 ms 2023-10-06T21:26:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:52 smithi142 ceph-mon[102951]: pgmap v181: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 895 KiB/s wr, 85 op/s 2023-10-06T21:26:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:52 smithi031 ceph-mon[98539]: pgmap v181: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 895 KiB/s wr, 85 op/s 2023-10-06T21:26:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:52 smithi146 ceph-mon[102809]: pgmap v181: 97 pgs: 97 active+clean; 68 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 895 KiB/s wr, 85 op/s 2023-10-06T21:26:53.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 94776 1.78 MB/sec execute 163 sec latency 18.011 ms 2023-10-06T21:26:54.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 95262 1.79 MB/sec execute 164 sec latency 18.683 ms 2023-10-06T21:26:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:54 smithi142 ceph-mon[102951]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:26:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:54 smithi031 ceph-mon[98539]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:26:54.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:54 smithi146 ceph-mon[102809]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:26:55.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 95600 1.80 MB/sec execute 165 sec latency 19.184 ms 2023-10-06T21:26:56.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 95966 1.80 MB/sec execute 166 sec latency 21.451 ms 2023-10-06T21:26:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:56 smithi142 ceph-mon[102951]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:56 smithi031 ceph-mon[98539]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:56 smithi146 ceph-mon[102809]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.075+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2402427362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc0732e0 msgr2=0x7fd7bc0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.075+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2402427362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc0736c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098f0 tx=0x7fd7ac009c00 comp rx=0 tx=0).stop 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2402427362 shutdown_connections 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2402427362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc10d6a0 0x7fd7bc10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2402427362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc0736c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2402427362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc073c00 0x7fd7bc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2402427362 >> 172.21.15.31:0/2402427362 conn(0x7fd7bc0fc930 msgr2=0x7fd7bc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2402427362 shutdown_connections 2023-10-06T21:26:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.076+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2402427362 wait complete. 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.077+0000 7fd7c21ff700 1 Processor -- start 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.077+0000 7fd7c21ff700 1 -- start start 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.077+0000 7fd7c21ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7c21ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc073c00 0x7fd7bc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7c21ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc10d6a0 0x7fd7bc19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7c21ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19e2f0 con 0x7fd7bc073c00 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7c21ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19e460 con 0x7fd7bc0732e0 2023-10-06T21:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7c21ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19e620 con 0x7fd7bc10d6a0 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49314/0 (socket says 172.21.15.31:49314) 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2916541584 learned_addr learned my addr 172.21.15.31:0/2916541584 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc073c00 0x7fd7bc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc10d6a0 0x7fd7bc19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2916541584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc10d6a0 msgr2=0x7fd7bc19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc10d6a0 0x7fd7bc19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc073c00 msgr2=0x7fd7bc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc073c00 0x7fd7bc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.078+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2916541584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40095f0 con 0x7fd7bc0732e0 2023-10-06T21:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009230 tx=0x7fd7ac00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7b97fa700 1 -- 172.21.15.31:0/2916541584 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01c070 con 0x7fd7bc0732e0 2023-10-06T21:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b4009c40 con 0x7fd7bc0732e0 2023-10-06T21:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7b97fa700 1 -- 172.21.15.31:0/2916541584 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7ac005040 con 0x7fd7bc0732e0 2023-10-06T21:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7b97fa700 1 -- 172.21.15.31:0/2916541584 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0038e0 con 0x7fd7bc0732e0 2023-10-06T21:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.079+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac009520 con 0x7fd7bc0732e0 2023-10-06T21:26:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.083+0000 7fd7b97fa700 1 -- 172.21.15.31:0/2916541584 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd7ac020030 con 0x7fd7bc0732e0 2023-10-06T21:26:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.084+0000 7fd7b97fa700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 0x7fd7a80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 Processor -- start 2023-10-06T21:26:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7b97fa700 1 -- 172.21.15.31:0/2916541584 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd7ac0ce0c0 con 0x7fd7bc0732e0 2023-10-06T21:26:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 -- start start 2023-10-06T21:26:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 0x7fd7a80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d58fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d59520 0x555f03d63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03cc2400 con 0x555f03d59520 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03cb3bc0 con 0x555f03d643d0 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.085+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03cb14e0 con 0x555f03d58c00 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49328/0 (socket says 172.21.15.31:49328) 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 -- 172.21.15.31:0/1986809431 learned_addr learned my addr 172.21.15.31:0/1986809431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d58fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d59520 0x555f03d63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 -- 172.21.15.31:0/1986809431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 msgr2=0x555f03d58fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d58fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 -- 172.21.15.31:0/1986809431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d59520 msgr2=0x555f03d63e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d59520 0x555f03d63e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 0x7fd7a80a3ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc19a920 tx=0x7fd7a4011720 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 -- 172.21.15.31:0/1986809431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f03ad7d80 con 0x555f03d643d0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.086+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e730 tx=0x7fd7b4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3b4859732a0d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.087+0000 7fd7b1ffb700 1 -- 172.21.15.31:0/1986809431 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400e040 con 0x555f03d643d0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.087+0000 7fd7b1ffb700 1 -- 172.21.15.31:0/1986809431 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7b400f040 con 0x555f03d643d0 2023-10-06T21:26:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.087+0000 7fd7b1ffb700 1 -- 172.21.15.31:0/1986809431 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007500 con 0x555f03d643d0 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.087+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1986809431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 msgr2=0x555f03d667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.087+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e730 tx=0x7fd7b4009920 comp rx=0 tx=0).stop 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1986809431 shutdown_connections 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d667c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d58fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1986809431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d59520 0x555f03d63e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1986809431 >> 172.21.15.31:0/1986809431 conn(0x555f03cce200 msgr2=0x555f03d55080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1986809431 shutdown_connections 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1986809431 wait complete. 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 -- start start 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.088+0000 7fd7d5b50b80 1 Processor -- start 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 0x555f03d63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03cacdd0 con 0x555f03d58c00 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03c496b0 con 0x555f03d643d0 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03c49350 con 0x555f03d59520 2023-10-06T21:26:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7d5b50b80 10 client.0 _ll_register_callbacks cb 0x555f03aa67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 0x555f03d63e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 0x555f03d63e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48750/0 (socket says 172.21.15.31:48750) 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.089+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/4260757187 learned_addr learned my addr 172.21.15.31:0/4260757187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 -- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 msgr2=0x555f03d617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 -- 172.21.15.31:0/4260757187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 msgr2=0x555f03d63e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 0x555f03d63e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 -- 172.21.15.31:0/4260757187 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b400e410 con 0x555f03d59520 2023-10-06T21:26:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d610c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4005fd0 tx=0x7fd7a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e494266a627f153d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd79affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.090+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401ec70 con 0x555f03d59520 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd7a401edd0 con 0x555f03d59520 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f03d4cd50 con 0x555f03d59520 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a400fe90 con 0x555f03d59520 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd7d5b50b80 10 client.26368 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555f03ac0c00 con 0x555f03d59520 2023-10-06T21:26:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.091+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f03d4cd50 con 0x555f03d59520 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7a4026410 con 0x555f03d59520 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 20 client.26368 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03d73e60 0x555f03d76280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 4 client.26368 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f03ca5db0 con 0x555f03d73e60 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.092+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03d73e60 0x555f03d76280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.093+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03d73e60 0x555f03d76280 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31bdde7689e0e45 server_cookie=3efb639f58a62e82 in_seq=0 out_seq=0 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.093+0000 7fd79affd700 10 client.26368 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.093+0000 7fd79affd700 1 -- 172.21.15.31:0/4260757187 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f03ca5db0 con 0x555f03d73e60 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.093+0000 7fd79affd700 10 client.26368 handle_command_reply: tid=0 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.093+0000 7fd7d5b50b80 1 client.26368 shutdown 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 msgr2=0x555f03d610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d610c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4005fd0 tx=0x7fd7a40047c0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 shutdown_connections 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d643d0 0x555f03d617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03d73e60 0x555f03d76280 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d58c00 0x555f03d63e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/4260757187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d610c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 >> 172.21.15.31:0/4260757187 conn(0x555f03cce200 msgr2=0x555f03d54c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd79affd700 0 client.26368 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 shutdown_connections 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/4260757187 wait complete. 2023-10-06T21:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.094+0000 7fd7d5b50b80 20 client.26368 trim_cache size 0 max 16384 2023-10-06T21:26:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 Processor -- start 2023-10-06T21:26:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 -- start start 2023-10-06T21:26:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03b2d8c0 con 0x555f03d643d0 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03d5c0e0 con 0x555f03d58c00 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03d6b480 con 0x555f03d59520 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49346/0 (socket says 172.21.15.31:49346) 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/1740592905 learned_addr learned my addr 172.21.15.31:0/1740592905 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.111+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48762/0 (socket says 172.21.15.31:48762) 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/1740592905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 msgr2=0x555f03d6aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d6aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/1740592905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 msgr2=0x555f03d70010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/1740592905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f03afe1f0 con 0x555f03d58c00 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d6aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac09b010 tx=0x7fd7ac0a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eda552c64b42eba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7b27fc700 1 -- 172.21.15.31:0/1740592905 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac09b080 con 0x555f03d58c00 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7b27fc700 1 -- 172.21.15.31:0/1740592905 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7ac09f1d0 con 0x555f03d58c00 2023-10-06T21:26:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.112+0000 7fd7b27fc700 1 -- 172.21.15.31:0/1740592905 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac097790 con 0x555f03d58c00 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1740592905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 msgr2=0x555f03d5bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac09b010 tx=0x7fd7ac0a1040 comp rx=0 tx=0).stop 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1740592905 shutdown_connections 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d70010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d58c00 0x555f03d5bb70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/1740592905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d59520 0x555f03d6aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1740592905 >> 172.21.15.31:0/1740592905 conn(0x555f03cce200 msgr2=0x555f03d537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1740592905 shutdown_connections 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.113+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/1740592905 wait complete. 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.114+0000 7fd7d5b50b80 1 -- start start 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.114+0000 7fd7d5b50b80 1 Processor -- start 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.114+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.114+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7d5b50b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03d695b0 con 0x555f03d643d0 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03d69720 con 0x555f03d59520 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7d5b50b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f03d698e0 con 0x555f03d58c00 2023-10-06T21:26:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7d5b50b80 10 client.0 _ll_register_callbacks cb 0x7fd7a400f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55758/0 (socket says 172.21.15.31:55758) 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49360/0 (socket says 172.21.15.31:49360) 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2141751989 learned_addr learned my addr 172.21.15.31:0/2141751989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bbfff700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 msgr2=0x555f03d6abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2141751989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 msgr2=0x555f03d68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7bb7fe700 1 -- 172.21.15.31:0/2141751989 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f03ad7d80 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.115+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd7bb7fe700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0cd010 tx=0x7fd7ac09b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd206842c0dddb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0a3380 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7b400f040 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fd7ac097790 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7ac097950 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd7d5b50b80 10 client.26374 fetch_fsmap learned FSMap version 34 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fd7ac009520 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7b400f040 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.116+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7ac0a10b0 con 0x555f03d58c00 2023-10-06T21:26:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 10 client.26374 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 20 client.26374 populate_metadata read hostname 'smithi031' 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03dc89e0 0x555f03dcadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 4 client.26374 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f03ca5ad0 con 0x555f03dc89e0 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03dc89e0 0x555f03dcadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd7b3fff700 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03dc89e0 0x555f03dcadd0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2421ad25ad942fb server_cookie=6159b4370b27e900 in_seq=0 out_seq=0 2023-10-06T21:26:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.117+0000 7fd79affd700 10 client.26374 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd79affd700 1 -- 172.21.15.31:0/2141751989 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f03ca5ad0 con 0x555f03dc89e0 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd79affd700 10 client.26374 handle_command_reply: tid=0 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 client.26374 shutdown 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 msgr2=0x555f03d6a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0cd010 tx=0x7fd7ac09b860 comp rx=0 tx=0).stop 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 shutdown_connections 2023-10-06T21:26:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555f03dc89e0 0x555f03dcadd0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f03d643d0 0x555f03d68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555f03d58c00 0x555f03d6a670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 --2- 172.21.15.31:0/2141751989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f03d59520 0x555f03d6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd79affd700 0 client.26374 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.140+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 >> 172.21.15.31:0/2141751989 conn(0x555f03cce200 msgr2=0x555f03d5c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.141+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 shutdown_connections 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.141+0000 7fd7d5b50b80 1 -- 172.21.15.31:0/2141751989 wait complete. 2023-10-06T21:26:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.141+0000 7fd7d5b50b80 20 client.26374 trim_cache size 0 max 16384 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 msgr2=0x7fd7a80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 0x7fd7a80a3ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc19a920 tx=0x7fd7a4011720 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 msgr2=0x7fd7bc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009230 tx=0x7fd7ac00fa00 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 shutdown_connections 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7bc10d6a0 0x7fd7bc19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc0732e0 0x7fd7bc199100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd7a80a15f0 0x7fd7a80a3ab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 --2- 172.21.15.31:0/2916541584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7bc073c00 0x7fd7bc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 >> 172.21.15.31:0/2916541584 conn(0x7fd7bc0fc930 msgr2=0x7fd7bc10f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 shutdown_connections 2023-10-06T21:26:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:26:57.142+0000 7fd7c21ff700 1 -- 172.21.15.31:0/2916541584 wait complete. 2023-10-06T21:26:57.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 96184 1.80 MB/sec execute 167 sec latency 24.947 ms 2023-10-06T21:26:58.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 96281 1.79 MB/sec execute 168 sec latency 375.184 ms 2023-10-06T21:26:58.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:58 smithi142 ceph-mon[102951]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:58 smithi031 ceph-mon[98539]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:58 smithi146 ceph-mon[102809]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 871 KiB/s wr, 87 op/s 2023-10-06T21:26:59.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 96529 1.78 MB/sec execute 169 sec latency 23.051 ms 2023-10-06T21:27:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:26:59 smithi031 ceph-mon[98539]: pgmap v185: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-10-06T21:27:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:26:59 smithi146 ceph-mon[102809]: pgmap v185: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-10-06T21:27:00.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 96876 1.78 MB/sec execute 170 sec latency 20.159 ms 2023-10-06T21:27:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:26:59 smithi142 ceph-mon[102951]: pgmap v185: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-10-06T21:27:01.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 97252 1.79 MB/sec execute 171 sec latency 23.485 ms 2023-10-06T21:27:02.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 97560 1.78 MB/sec execute 172 sec latency 27.534 ms 2023-10-06T21:27:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:02 smithi142 ceph-mon[102951]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 913 KiB/s wr, 88 op/s 2023-10-06T21:27:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:02 smithi031 ceph-mon[98539]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 913 KiB/s wr, 88 op/s 2023-10-06T21:27:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:02 smithi146 ceph-mon[102809]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 913 KiB/s wr, 88 op/s 2023-10-06T21:27:03.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 97951 1.77 MB/sec execute 173 sec latency 27.578 ms 2023-10-06T21:27:04.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 98265 1.77 MB/sec execute 174 sec latency 25.638 ms 2023-10-06T21:27:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:04 smithi142 ceph-mon[102951]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:27:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:04 smithi031 ceph-mon[98539]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:27:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:04 smithi146 ceph-mon[102809]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-10-06T21:27:05.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 98747 1.79 MB/sec execute 175 sec latency 21.197 ms 2023-10-06T21:27:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:05 smithi142 ceph-mon[102951]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:27:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:05 smithi031 ceph-mon[98539]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:27:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:05 smithi146 ceph-mon[102809]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:27:06.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 99132 1.79 MB/sec execute 176 sec latency 17.023 ms 2023-10-06T21:27:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:06 smithi142 ceph-mon[102951]: pgmap v189: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:06 smithi031 ceph-mon[98539]: pgmap v189: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:06 smithi146 ceph-mon[102809]: pgmap v189: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:07.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 99478 1.80 MB/sec execute 177 sec latency 18.391 ms 2023-10-06T21:27:08.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 99688 1.80 MB/sec execute 178 sec latency 19.518 ms 2023-10-06T21:27:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:08 smithi142 ceph-mon[102951]: pgmap v190: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:08 smithi031 ceph-mon[98539]: pgmap v190: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:08 smithi146 ceph-mon[102809]: pgmap v190: 97 pgs: 97 active+clean; 67 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2023-10-06T21:27:09.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 99863 1.79 MB/sec execute 179 sec latency 20.089 ms 2023-10-06T21:27:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:09 smithi031 ceph-mon[98539]: pgmap v191: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:09 smithi146 ceph-mon[102809]: pgmap v191: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 -- 172.21.15.31:0/4247671114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 msgr2=0x7f14c810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 --2- 172.21.15.31:0/4247671114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c810fb10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f14c000b0a0 tx=0x7f14c000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 -- 172.21.15.31:0/4247671114 shutdown_connections 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 --2- 172.21.15.31:0/4247671114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c810fb10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 --2- 172.21.15.31:0/4247671114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c80736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 --2- 172.21.15.31:0/4247671114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 0x7f14c810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 -- 172.21.15.31:0/4247671114 >> 172.21.15.31:0/4247671114 conn(0x7f14c80fc930 msgr2=0x7f14c80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 -- 172.21.15.31:0/4247671114 shutdown_connections 2023-10-06T21:27:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.314+0000 7f14cfdff700 1 -- 172.21.15.31:0/4247671114 wait complete. 2023-10-06T21:27:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.315+0000 7f14cfdff700 1 Processor -- start 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 -- start start 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 0x7f14c81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c819e390 con 0x7f14c810d6a0 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c819e500 con 0x7f14c8073c00 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.316+0000 7f14cfdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c819e6c0 con 0x7f14c80732e0 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60882/0 (socket says 172.21.15.31:60882) 2023-10-06T21:27:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 -- 172.21.15.31:0/2874247693 learned_addr learned my addr 172.21.15.31:0/2874247693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cd39a700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 0x7f14c81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14ce39c700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 -- 172.21.15.31:0/2874247693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 msgr2=0x7f14c81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 0x7f14c81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 -- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 msgr2=0x7f14c819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 -- 172.21.15.31:0/2874247693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0009d00 con 0x7f14c80732e0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.317+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e9a0 tx=0x7f14c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.318+0000 7f14beffd700 1 -- 172.21.15.31:0/2874247693 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4009790 con 0x7f14c80732e0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.318+0000 7f14beffd700 1 -- 172.21.15.31:0/2874247693 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14c4010430 con 0x7f14c80732e0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.318+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c819e9a0 con 0x7f14c80732e0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.318+0000 7f14beffd700 1 -- 172.21.15.31:0/2874247693 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400cea0 con 0x7f14c80732e0 2023-10-06T21:27:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.318+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c81a61e0 con 0x7f14c80732e0 2023-10-06T21:27:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.322+0000 7f14beffd700 1 -- 172.21.15.31:0/2874247693 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f14c40105a0 con 0x7f14c80732e0 2023-10-06T21:27:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.323+0000 7f14e37b5b80 1 Processor -- start 2023-10-06T21:27:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.323+0000 7f14beffd700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 0x7f14b40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.323+0000 7f14e37b5b80 1 -- start start 2023-10-06T21:27:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.323+0000 7f14beffd700 1 -- 172.21.15.31:0/2874247693 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f14c4014070 con 0x7f14c80732e0 2023-10-06T21:27:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.323+0000 7f14ce39c700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 0x7f14b40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afdc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175af38400 con 0x56175afcec00 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175af29bc0 con 0x56175afda3d0 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175af274e0 con 0x56175afcf520 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39878/0 (socket says 172.21.15.31:39878) 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 -- 172.21.15.31:0/459625211 learned_addr learned my addr 172.21.15.31:0/459625211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cd39a700 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 -- 172.21.15.31:0/459625211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 msgr2=0x56175afd9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 -- 172.21.15.31:0/459625211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 msgr2=0x56175afdc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afdc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.324+0000 7f14cdb9b700 1 -- 172.21.15.31:0/459625211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56175ad4dd80 con 0x56175afcec00 2023-10-06T21:27:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.325+0000 7f14ce39c700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 0x7f14b40a3b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14c00062a0 tx=0x7f14c0009340 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.325+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14c4096cb0 tx=0x7f14c4099e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7991e9e64ee05513 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.325+0000 7f14b3fff700 1 -- 172.21.15.31:0/459625211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c409e160 con 0x56175afcec00 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.325+0000 7f14b3fff700 1 -- 172.21.15.31:0/459625211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14c409a240 con 0x56175afcec00 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.325+0000 7f14b3fff700 1 -- 172.21.15.31:0/459625211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4099730 con 0x56175afcec00 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 -- 172.21.15.31:0/459625211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 msgr2=0x56175afcefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14c4096cb0 tx=0x7f14c4099e90 comp rx=0 tx=0).stop 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 -- 172.21.15.31:0/459625211 shutdown_connections 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afdc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afcefe0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/459625211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 -- 172.21.15.31:0/459625211 >> 172.21.15.31:0/459625211 conn(0x56175af44200 msgr2=0x56175afcb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 -- 172.21.15.31:0/459625211 shutdown_connections 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.326+0000 7f14e37b5b80 1 -- 172.21.15.31:0/459625211 wait complete. 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.327+0000 7f14e37b5b80 1 -- start start 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.327+0000 7f14e37b5b80 1 Processor -- start 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.327+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afd77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175af22dd0 con 0x56175afcec00 2023-10-06T21:27:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175aebf6b0 con 0x56175afda3d0 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175aebf350 con 0x56175afcf520 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14e37b5b80 10 client.0 _ll_register_callbacks cb 0x56175ad1c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60898/0 (socket says 172.21.15.31:60898) 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 -- 172.21.15.31:0/1817130305 learned_addr learned my addr 172.21.15.31:0/1817130305 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14ce39c700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afd77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 -- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 msgr2=0x56175afd77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afd77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 -- 172.21.15.31:0/1817130305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 msgr2=0x56175afd9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afd9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cd39a700 1 -- 172.21.15.31:0/1817130305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c81a61e0 con 0x56175afcf520 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.328+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afd9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14b8012700 tx=0x7f14b8012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8579dce4f1455da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8017070 con 0x56175afcf520 2023-10-06T21:27:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56175afc2d50 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f14b8010430 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14b80105f0 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14e37b5b80 10 client.26386 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56175ad36c00 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.329+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56175afc2d50 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f14b801c6e0 con 0x56175afcf520 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 10 client.26386 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 10 client.26386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 20 client.26386 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175afe9f00 0x56175afec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 4 client.26386 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56175af1bdb0 con 0x56175afe9f00 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.330+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175afe9f00 0x56175afec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.331+0000 7f14b17fa700 10 client.26386 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.331+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175afe9f00 0x56175afec320 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f3d7de9ff4a094 server_cookie=38c9df9cb7c8b7ef in_seq=0 out_seq=0 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14b17fa700 1 -- 172.21.15.31:0/1817130305 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56175af1bdb0 con 0x56175afe9f00 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14b17fa700 10 client.26386 handle_command_reply: tid=0 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 client.26386 shutdown 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 msgr2=0x56175afd70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14b8012700 tx=0x7f14b8012a10 comp rx=0 tx=0).stop 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 shutdown_connections 2023-10-06T21:27:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175afd77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175afe9f00 0x56175afec320 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afd9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1817130305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afd70d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14b17fa700 0 client.26386 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 >> 172.21.15.31:0/1817130305 conn(0x56175af44200 msgr2=0x56175afcac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 shutdown_connections 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1817130305 wait complete. 2023-10-06T21:27:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.332+0000 7f14e37b5b80 20 client.26386 trim_cache size 0 max 16384 2023-10-06T21:27:10.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 100142 1.78 MB/sec execute 180 sec latency 22.444 ms 2023-10-06T21:27:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.350+0000 7f14e37b5b80 1 Processor -- start 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.350+0000 7f14e37b5b80 1 -- start start 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcec00 0x56175afd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 0x56175afd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175ada38c0 con 0x56175afda3d0 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175afcbe40 con 0x56175afcf520 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175afc9eb0 con 0x56175afcec00 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14cd39a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14ce39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 0x56175afd99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14cd39a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34928/0 (socket says 172.21.15.31:34928) 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14cd39a700 1 -- 172.21.15.31:0/2675344081 learned_addr learned my addr 172.21.15.31:0/2675344081 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.351+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcec00 0x56175afd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 -- 172.21.15.31:0/2675344081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcec00 msgr2=0x56175afd7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcec00 0x56175afd7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 -- 172.21.15.31:0/2675344081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 msgr2=0x56175afd99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 0x56175afd99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 -- 172.21.15.31:0/2675344081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56175ad741f0 con 0x56175afcf520 2023-10-06T21:27:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14ce39c700 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 0x56175afd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14cd39a700 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f14b8001eb0 tx=0x7f14b8009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944920c448ea6a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14bcbff700 1 -- 172.21.15.31:0/2675344081 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8017040 con 0x56175afcf520 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14bcbff700 1 -- 172.21.15.31:0/2675344081 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14b801c6e0 con 0x56175afcf520 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.352+0000 7f14bcbff700 1 -- 172.21.15.31:0/2675344081 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8003b00 con 0x56175afcf520 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- 172.21.15.31:0/2675344081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 msgr2=0x56175afd6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f14b8001eb0 tx=0x7f14b8009750 comp rx=0 tx=0).stop 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- 172.21.15.31:0/2675344081 shutdown_connections 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afda3d0 0x56175afd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcec00 0x56175afd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/2675344081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afcf520 0x56175afd6890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- 172.21.15.31:0/2675344081 >> 172.21.15.31:0/2675344081 conn(0x56175af44200 msgr2=0x56175afd23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- 172.21.15.31:0/2675344081 shutdown_connections 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- 172.21.15.31:0/2675344081 wait complete. 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.353+0000 7f14e37b5b80 1 -- start start 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 Processor -- start 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afe0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175b03cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175afe0f90 con 0x56175afcec00 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b03d270 con 0x56175afda3d0 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b03d3b0 con 0x56175afcf520 2023-10-06T21:27:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14e37b5b80 10 client.0 _ll_register_callbacks cb 0x7f14b8009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14cdb9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39902/0 (socket says 172.21.15.31:39902) 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.354+0000 7f14cdb9b700 1 -- 172.21.15.31:0/1611744151 learned_addr learned my addr 172.21.15.31:0/1611744151 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afe0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14ce39c700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175b03cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 -- 172.21.15.31:0/1611744151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 msgr2=0x56175afe0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afe0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 -- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 msgr2=0x56175b03cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175b03cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 -- 172.21.15.31:0/1611744151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56175ad36c00 con 0x56175afcec00 2023-10-06T21:27:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14cdb9b700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c2d0 tx=0x7f14c4099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f058f076ff2949 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:27:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40cd220 con 0x56175afcec00 2023-10-06T21:27:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f14c40a25d0 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.355+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14c409a240 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c409a010 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 10 client.16713 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56175ad4c250 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14c409a240 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f14c4096030 con 0x56175afcec00 2023-10-06T21:27:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 20 client.16713 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175b03e6e0 0x56175b040b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 4 client.16713 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.356+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56175af1bad0 con 0x56175b03e6e0 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.357+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175b03e6e0 0x56175b040b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.357+0000 7f14cd39a700 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175b03e6e0 0x56175b040b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e433dc3480b97c4 server_cookie=a3799f167d96071 in_seq=0 out_seq=0 2023-10-06T21:27:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.357+0000 7f14b17fa700 10 client.16713 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14b17fa700 1 -- 172.21.15.31:0/1611744151 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56175af1bad0 con 0x56175b03e6e0 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14b17fa700 10 client.16713 handle_command_reply: tid=0 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 client.16713 shutdown 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 msgr2=0x56175afe0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c2d0 tx=0x7f14c4099a40 comp rx=0 tx=0).stop 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 shutdown_connections 2023-10-06T21:27:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56175afda3d0 0x56175b03cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56175b03e6e0 0x56175b040b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56175afcec00 0x56175afe0140 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 --2- 172.21.15.31:0/1611744151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56175afcf520 0x56175afe0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14b17fa700 0 client.16713 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.378+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 >> 172.21.15.31:0/1611744151 conn(0x56175af44200 msgr2=0x56175afca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 shutdown_connections 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14e37b5b80 1 -- 172.21.15.31:0/1611744151 wait complete. 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14e37b5b80 20 client.16713 trim_cache size 0 max 16384 2023-10-06T21:27:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:27:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:27:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:27:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:27:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 msgr2=0x7f14b40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 0x7f14b40a3b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14c00062a0 tx=0x7f14c0009340 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 msgr2=0x7f14c81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.379+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e9a0 tx=0x7f14c400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 shutdown_connections 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14b40a1670 0x7f14b40a3b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c810d6a0 0x7f14c819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c80732e0 0x7f14c81991a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 --2- 172.21.15.31:0/2874247693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14c8073c00 0x7f14c81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 >> 172.21.15.31:0/2874247693 conn(0x7f14c80fc930 msgr2=0x7f14c810f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 shutdown_connections 2023-10-06T21:27:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:10.380+0000 7f14cfdff700 1 -- 172.21.15.31:0/2874247693 wait complete. 2023-10-06T21:27:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:09 smithi142 ceph-mon[102951]: pgmap v191: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:11.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 100487 1.78 MB/sec execute 181 sec latency 17.646 ms 2023-10-06T21:27:12.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 100858 1.79 MB/sec execute 182 sec latency 26.345 ms 2023-10-06T21:27:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:12 smithi142 ceph-mon[102951]: pgmap v192: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:12 smithi031 ceph-mon[98539]: pgmap v192: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:12 smithi146 ceph-mon[102809]: pgmap v192: 97 pgs: 97 active+clean; 71 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-10-06T21:27:13.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 101189 1.78 MB/sec execute 183 sec latency 28.652 ms 2023-10-06T21:27:13.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:14.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 101561 1.77 MB/sec execute 184 sec latency 26.924 ms 2023-10-06T21:27:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:14 smithi142 ceph-mon[102951]: pgmap v193: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:14 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:14 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:14 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:27:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:14 smithi142 ceph-mon[102951]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:27:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:14 smithi031 ceph-mon[98539]: pgmap v193: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:14 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:14 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:14 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:27:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:14 smithi031 ceph-mon[98539]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:27:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:14 smithi146 ceph-mon[102809]: pgmap v193: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:14 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:14 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:14 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:27:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:14 smithi146 ceph-mon[102809]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:27:15.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 101866 1.77 MB/sec execute 185 sec latency 18.265 ms 2023-10-06T21:27:16.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 102354 1.79 MB/sec execute 186 sec latency 18.971 ms 2023-10-06T21:27:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:16 smithi142 ceph-mon[102951]: pgmap v195: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:16 smithi031 ceph-mon[98539]: pgmap v195: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:16 smithi146 ceph-mon[102809]: pgmap v195: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:17.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 102693 1.79 MB/sec execute 187 sec latency 17.904 ms 2023-10-06T21:27:18.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 103066 1.80 MB/sec execute 188 sec latency 19.122 ms 2023-10-06T21:27:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:18 smithi142 ceph-mon[102951]: pgmap v196: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:18 smithi031 ceph-mon[98539]: pgmap v196: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:18 smithi146 ceph-mon[102809]: pgmap v196: 97 pgs: 97 active+clean; 71 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-10-06T21:27:19.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 103275 1.79 MB/sec execute 189 sec latency 19.119 ms 2023-10-06T21:27:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:19 smithi031 ceph-mon[98539]: pgmap v197: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:19 smithi146 ceph-mon[102809]: pgmap v197: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:20.342 INFO:tasks.workunit.client.0.smithi031.stdout: 1 103452 1.79 MB/sec execute 190 sec latency 26.025 ms 2023-10-06T21:27:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:19 smithi142 ceph-mon[102951]: pgmap v197: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:21.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 103773 1.78 MB/sec execute 191 sec latency 23.548 ms 2023-10-06T21:27:22.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 104091 1.78 MB/sec execute 192 sec latency 20.005 ms 2023-10-06T21:27:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:22 smithi142 ceph-mon[102951]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:22 smithi031 ceph-mon[98539]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:22 smithi146 ceph-mon[102809]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:27:23.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 104452 1.78 MB/sec execute 193 sec latency 24.849 ms 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 -- 172.21.15.31:0/3992462718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc068c90 msgr2=0x7f25bc10dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3992462718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc068c90 0x7f25bc10dd00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f25b400b0a0 tx=0x7f25b400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 -- 172.21.15.31:0/3992462718 shutdown_connections 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3992462718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc068c90 0x7f25bc10dd00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3992462718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc106f60 0x7f25bc107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3992462718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc0682d0 0x7f25bc068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 -- 172.21.15.31:0/3992462718 >> 172.21.15.31:0/3992462718 conn(0x7f25bc076b70 msgr2=0x7f25bc076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 -- 172.21.15.31:0/3992462718 shutdown_connections 2023-10-06T21:27:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.549+0000 7f25c3bff700 1 -- 172.21.15.31:0/3992462718 wait complete. 2023-10-06T21:27:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 Processor -- start 2023-10-06T21:27:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 -- start start 2023-10-06T21:27:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc0682d0 0x7f25bc196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc106f60 0x7f25bc19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c190 con 0x7f25bc106f60 2023-10-06T21:27:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c300 con 0x7f25bc068c90 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.550+0000 7f25c3bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c4c0 con 0x7f25bc0682d0 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc106f60 0x7f25bc19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41560/0 (socket says 172.21.15.31:41560) 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 -- 172.21.15.31:0/3978033400 learned_addr learned my addr 172.21.15.31:0/3978033400 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c199b700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc0682d0 0x7f25bc196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 -- 172.21.15.31:0/3978033400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc0682d0 msgr2=0x7f25bc196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc0682d0 0x7f25bc196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 -- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc106f60 msgr2=0x7f25bc19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc106f60 0x7f25bc19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 -- 172.21.15.31:0/3978033400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4009d00 con 0x7f25bc068c90 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.551+0000 7f25c119a700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e9a0 tx=0x7f25ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.552+0000 7f25b2ffd700 1 -- 172.21.15.31:0/3978033400 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009790 con 0x7f25bc068c90 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.552+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc19c7a0 con 0x7f25bc068c90 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.552+0000 7f25b2ffd700 1 -- 172.21.15.31:0/3978033400 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25ac010430 con 0x7f25bc068c90 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.552+0000 7f25b2ffd700 1 -- 172.21.15.31:0/3978033400 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00cea0 con 0x7f25bc068c90 2023-10-06T21:27:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.552+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1a3fe0 con 0x7f25bc068c90 2023-10-06T21:27:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.556+0000 7f25b2ffd700 1 -- 172.21.15.31:0/3978033400 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f25ac0105a0 con 0x7f25bc068c90 2023-10-06T21:27:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 Processor -- start 2023-10-06T21:27:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 -- start start 2023-10-06T21:27:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da80cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da81a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da776400 con 0x5634da80cc00 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da767bc0 con 0x5634da8183d0 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25d7619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da7654e0 con 0x5634da80d520 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.557+0000 7f25b2ffd700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 0x7f25a80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:33288/0 (socket says 172.21.15.31:33288) 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 -- 172.21.15.31:0/98994183 learned_addr learned my addr 172.21.15.31:0/98994183 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c219c700 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da80cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25b2ffd700 1 -- 172.21.15.31:0/3978033400 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f25ac014070 con 0x7f25bc068c90 2023-10-06T21:27:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c219c700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 0x7f25a80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c119a700 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da81a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 -- 172.21.15.31:0/98994183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 msgr2=0x5634da81a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da81a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 -- 172.21.15.31:0/98994183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 msgr2=0x5634da80cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da80cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.558+0000 7f25c199b700 1 -- 172.21.15.31:0/98994183 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634da58bd80 con 0x5634da80d520 2023-10-06T21:27:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.559+0000 7f25c199b700 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25b800bcf0 tx=0x7f25b80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12486dfa596c21f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.559+0000 7f25c219c700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 0x7f25a80a3ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006130 tx=0x7f25b400b720 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.559+0000 7f25a7fff700 1 -- 172.21.15.31:0/98994183 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8010c50 con 0x5634da80d520 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.559+0000 7f25a7fff700 1 -- 172.21.15.31:0/98994183 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25b8015070 con 0x5634da80d520 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.559+0000 7f25a7fff700 1 -- 172.21.15.31:0/98994183 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004e80 con 0x5634da80d520 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 -- 172.21.15.31:0/98994183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 msgr2=0x5634da817e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25b800bcf0 tx=0x7f25b80103d0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 -- 172.21.15.31:0/98994183 shutdown_connections 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da81a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da80cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 --2- 172.21.15.31:0/98994183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da817e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 -- 172.21.15.31:0/98994183 >> 172.21.15.31:0/98994183 conn(0x5634da782200 msgr2=0x5634da809080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 -- 172.21.15.31:0/98994183 shutdown_connections 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.560+0000 7f25d7619b80 1 -- 172.21.15.31:0/98994183 wait complete. 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.561+0000 7f25d7619b80 1 -- start start 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.561+0000 7f25d7619b80 1 Processor -- start 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.561+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.561+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80d520 0x5634da815020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.561+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 0x5634da8156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25d7619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da760dd0 con 0x5634da80cc00 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25d7619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da6fd6b0 con 0x5634da80d520 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25d7619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da6fd350 con 0x5634da8183d0 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25d7619b80 10 client.0 _ll_register_callbacks cb 0x5634da55a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56902/0 (socket says 172.21.15.31:56902) 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 -- 172.21.15.31:0/2067327908 learned_addr learned my addr 172.21.15.31:0/2067327908 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c219c700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 0x5634da8156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c119a700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80d520 0x5634da815020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 -- 172.21.15.31:0/2067327908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 msgr2=0x5634da8156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 0x5634da8156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 -- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80d520 msgr2=0x5634da815020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80d520 0x5634da815020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c199b700 1 -- 172.21.15.31:0/2067327908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b800b920 con 0x5634da80cc00 2023-10-06T21:27:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.562+0000 7f25c219c700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 0x5634da8156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25c199b700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004a80 tx=0x7f25b8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148e2834afab150f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8020070 con 0x5634da80cc00 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634da800d50 con 0x5634da80cc00 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f25b8017070 con 0x5634da80cc00 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8005ce0 con 0x5634da80cc00 2023-10-06T21:27:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25d7619b80 10 client.16725 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5634da574c00 con 0x5634da80cc00 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.563+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5634da800d50 con 0x5634da80cc00 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25b8011440 con 0x5634da80cc00 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 20 client.16725 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da827cf0 0x5634da82a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 4 client.16725 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634da759db0 con 0x5634da827cf0 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.564+0000 7f25c119a700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da827cf0 0x5634da82a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.565+0000 7f25a57fa700 10 client.16725 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.565+0000 7f25c119a700 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da827cf0 0x5634da82a110 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9410569b2cb2f3 server_cookie=184672e11126b297 in_seq=0 out_seq=0 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.565+0000 7f25a57fa700 1 -- 172.21.15.31:0/2067327908 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5634da759db0 con 0x5634da827cf0 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.565+0000 7f25a57fa700 10 client.16725 handle_command_reply: tid=0 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.565+0000 7f25d7619b80 1 client.16725 shutdown 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 msgr2=0x5634da817da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f25b8004a80 tx=0x7f25b8010c80 comp rx=0 tx=0).stop 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 shutdown_connections 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da8183d0 0x5634da8156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da827cf0 0x5634da82a110 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80cc00 0x5634da817da0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 --2- 172.21.15.31:0/2067327908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80d520 0x5634da815020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25a57fa700 0 client.16725 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 >> 172.21.15.31:0/2067327908 conn(0x5634da782200 msgr2=0x5634da808c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 shutdown_connections 2023-10-06T21:27:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 1 -- 172.21.15.31:0/2067327908 wait complete. 2023-10-06T21:27:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.566+0000 7f25d7619b80 20 client.16725 trim_cache size 0 max 16384 2023-10-06T21:27:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 Processor -- start 2023-10-06T21:27:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 -- start start 2023-10-06T21:27:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80cc00 0x5634da8077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da810390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da5e18c0 con 0x5634da80d520 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da805860 con 0x5634da8183d0 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25d7619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da807cf0 con 0x5634da80cc00 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25c119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25c119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56906/0 (socket says 172.21.15.31:56906) 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25c119a700 1 -- 172.21.15.31:0/3640984373 learned_addr learned my addr 172.21.15.31:0/3640984373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.585+0000 7f25c219c700 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da810390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c199b700 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80cc00 0x5634da8077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 -- 172.21.15.31:0/3640984373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80cc00 msgr2=0x5634da8077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80cc00 0x5634da8077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 -- 172.21.15.31:0/3640984373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 msgr2=0x5634da810390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da810390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 -- 172.21.15.31:0/3640984373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634da5b21f0 con 0x5634da80d520 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25c119a700 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25ac099fb0 tx=0x7f25ac0cd410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7f6ece728d25db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25b0bff700 1 -- 172.21.15.31:0/3640984373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac09a9b0 con 0x5634da80d520 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25b0bff700 1 -- 172.21.15.31:0/3640984373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25ac09a2b0 con 0x5634da80d520 2023-10-06T21:27:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.586+0000 7f25b0bff700 1 -- 172.21.15.31:0/3640984373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac09a770 con 0x5634da80d520 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 -- 172.21.15.31:0/3640984373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 msgr2=0x5634da824110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25ac099fb0 tx=0x7f25ac0cd410 comp rx=0 tx=0).stop 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 -- 172.21.15.31:0/3640984373 shutdown_connections 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da8183d0 0x5634da810390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80cc00 0x5634da8077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 --2- 172.21.15.31:0/3640984373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da80d520 0x5634da824110 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 -- 172.21.15.31:0/3640984373 >> 172.21.15.31:0/3640984373 conn(0x5634da782200 msgr2=0x5634da82ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 -- 172.21.15.31:0/3640984373 shutdown_connections 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.587+0000 7f25d7619b80 1 -- 172.21.15.31:0/3640984373 wait complete. 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 -- start start 2023-10-06T21:27:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 Processor -- start 2023-10-06T21:27:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80cc00 0x5634da812360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da8128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da812eb0 con 0x5634da8183d0 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da813020 con 0x5634da80cc00 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.588+0000 7f25d7619b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634da811820 con 0x5634da80d520 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25d7619b80 10 client.0 _ll_register_callbacks cb 0x7f25b4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56914/0 (socket says 172.21.15.31:56914) 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 -- 172.21.15.31:0/430532188 learned_addr learned my addr 172.21.15.31:0/430532188 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c119a700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da8128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c199b700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80cc00 0x5634da812360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 -- 172.21.15.31:0/430532188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 msgr2=0x5634da8128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da8128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 -- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80cc00 msgr2=0x5634da812360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80cc00 0x5634da812360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.589+0000 7f25c219c700 1 -- 172.21.15.31:0/430532188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1a3fe0 con 0x5634da8183d0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25c219c700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f25b400b0a0 tx=0x7f25b40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c7a4bdd70496e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400e040 con 0x5634da8183d0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b8015070 con 0x5634da8183d0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f25b40078a0 con 0x5634da8183d0 2023-10-06T21:27:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b4005040 con 0x5634da8183d0 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25d7619b80 10 client.16731 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5634da58a250 con 0x5634da8183d0 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.590+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25b8015070 con 0x5634da8183d0 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25b402c040 con 0x5634da8183d0 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 10 client.16731 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 20 client.16731 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da880b70 0x5634da882f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 4 client.16731 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634da759ad0 con 0x5634da880b70 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.591+0000 7f25c199b700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da880b70 0x5634da882f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.592+0000 7f25c199b700 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da880b70 0x5634da882f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5568f6856938b7 server_cookie=9ffde3cb2af76afb in_seq=0 out_seq=0 2023-10-06T21:27:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.592+0000 7f25a57fa700 10 client.16731 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.615+0000 7f25a57fa700 1 -- 172.21.15.31:0/430532188 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634da759ad0 con 0x5634da880b70 2023-10-06T21:27:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.615+0000 7f25a57fa700 10 client.16731 handle_command_reply: tid=0 2023-10-06T21:27:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 client.16731 shutdown 2023-10-06T21:27:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 msgr2=0x5634da8112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f25b400b0a0 tx=0x7f25b40099e0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 shutdown_connections 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5634da880b70 0x5634da882f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634da8183d0 0x5634da8112e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634da80cc00 0x5634da812360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 --2- 172.21.15.31:0/430532188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634da80d520 0x5634da8128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 >> 172.21.15.31:0/430532188 conn(0x5634da782200 msgr2=0x5634da824110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25a57fa700 0 client.16731 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 shutdown_connections 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 1 -- 172.21.15.31:0/430532188 wait complete. 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.616+0000 7f25d7619b80 20 client.16731 trim_cache size 0 max 16384 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:27:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 msgr2=0x7f25a80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 0x7f25a80a3ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006130 tx=0x7f25b400b720 comp rx=0 tx=0).stop 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 msgr2=0x7f25bc1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e9a0 tx=0x7f25ac00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 shutdown_connections 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f25a80a15f0 0x7f25a80a3ab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc106f60 0x7f25bc19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc0682d0 0x7f25bc196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 --2- 172.21.15.31:0/3978033400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25bc068c90 0x7f25bc1974e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 >> 172.21.15.31:0/3978033400 conn(0x7f25bc076b70 msgr2=0x7f25bc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 shutdown_connections 2023-10-06T21:27:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:23.617+0000 7f25c3bff700 1 -- 172.21.15.31:0/3978033400 wait complete. 2023-10-06T21:27:24.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 104784 1.78 MB/sec execute 194 sec latency 26.902 ms 2023-10-06T21:27:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:24 smithi142 ceph-mon[102951]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:27:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:24 smithi031 ceph-mon[98539]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:27:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:24 smithi146 ceph-mon[102809]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:27:25.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 105211 1.77 MB/sec execute 195 sec latency 25.963 ms 2023-10-06T21:27:26.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 105492 1.77 MB/sec execute 196 sec latency 16.345 ms 2023-10-06T21:27:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:26 smithi142 ceph-mon[102951]: pgmap v200: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1020 KiB/s wr, 93 op/s 2023-10-06T21:27:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:26 smithi031 ceph-mon[98539]: pgmap v200: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1020 KiB/s wr, 93 op/s 2023-10-06T21:27:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:26 smithi146 ceph-mon[102809]: pgmap v200: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1020 KiB/s wr, 93 op/s 2023-10-06T21:27:27.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 106083 1.79 MB/sec execute 197 sec latency 19.522 ms 2023-10-06T21:27:28.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 106280 1.79 MB/sec execute 198 sec latency 19.391 ms 2023-10-06T21:27:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:28 smithi142 ceph-mon[102951]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 941 KiB/s wr, 86 op/s 2023-10-06T21:27:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:28 smithi031 ceph-mon[98539]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 941 KiB/s wr, 86 op/s 2023-10-06T21:27:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:28 smithi146 ceph-mon[102809]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 941 KiB/s wr, 86 op/s 2023-10-06T21:27:29.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 106678 1.80 MB/sec execute 199 sec latency 19.790 ms 2023-10-06T21:27:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:29 smithi031 ceph-mon[98539]: pgmap v202: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:27:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:29 smithi146 ceph-mon[102809]: pgmap v202: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:27:30.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 106792 1.79 MB/sec execute 200 sec latency 295.997 ms 2023-10-06T21:27:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:29 smithi142 ceph-mon[102951]: pgmap v202: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-10-06T21:27:31.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 106958 1.78 MB/sec execute 201 sec latency 19.326 ms 2023-10-06T21:27:32.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 107236 1.78 MB/sec execute 202 sec latency 25.741 ms 2023-10-06T21:27:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:32 smithi142 ceph-mon[102951]: pgmap v203: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:27:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:32 smithi031 ceph-mon[98539]: pgmap v203: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:27:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:32 smithi146 ceph-mon[102809]: pgmap v203: 97 pgs: 97 active+clean; 74 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:27:33.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 107576 1.77 MB/sec execute 203 sec latency 20.101 ms 2023-10-06T21:27:34.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 107942 1.78 MB/sec execute 204 sec latency 25.920 ms 2023-10-06T21:27:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:34 smithi142 ceph-mon[102951]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:27:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:34 smithi031 ceph-mon[98539]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:27:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:34 smithi146 ceph-mon[102809]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:27:35.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 108260 1.77 MB/sec execute 205 sec latency 31.026 ms 2023-10-06T21:27:36.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 108631 1.77 MB/sec execute 206 sec latency 27.692 ms 2023-10-06T21:27:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:36 smithi142 ceph-mon[102951]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:36 smithi031 ceph-mon[98539]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:36 smithi146 ceph-mon[102809]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 -- 172.21.15.31:0/3284560470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e4101a80 msgr2=0x7f02e4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 --2- 172.21.15.31:0/3284560470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e4101a80 0x7f02e4105ac0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009990 tx=0x7f02d8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 -- 172.21.15.31:0/3284560470 shutdown_connections 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 --2- 172.21.15.31:0/3284560470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 0x7f02e410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 --2- 172.21.15.31:0/3284560470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e41010d0 0x7f02e41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 --2- 172.21.15.31:0/3284560470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e4101a80 0x7f02e4105ac0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 -- 172.21.15.31:0/3284560470 >> 172.21.15.31:0/3284560470 conn(0x7f02e40fc930 msgr2=0x7f02e40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 -- 172.21.15.31:0/3284560470 shutdown_connections 2023-10-06T21:27:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.785+0000 7f02e95ff700 1 -- 172.21.15.31:0/3284560470 wait complete. 2023-10-06T21:27:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.786+0000 7f02e95ff700 1 Processor -- start 2023-10-06T21:27:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 -- start start 2023-10-06T21:27:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 0x7f02e4197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e419bac0 0x7f02e419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e419c540 con 0x7f02e41010d0 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e419c6b0 con 0x7f02e41061f0 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e419c870 con 0x7f02e419bac0 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 0x7f02e4197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46756/0 (socket says 172.21.15.31:46756) 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e2ffd700 1 -- 172.21.15.31:0/2116872252 learned_addr learned my addr 172.21.15.31:0/2116872252 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.787+0000 7f02e37fe700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e419bac0 0x7f02e419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 -- 172.21.15.31:0/2116872252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e419bac0 msgr2=0x7f02e419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e419bac0 0x7f02e419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 -- 172.21.15.31:0/2116872252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 msgr2=0x7f02e4197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 0x7f02e4197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 -- 172.21.15.31:0/2116872252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80095f0 con 0x7f02e41010d0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f02d400e9a0 tx=0x7f02d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02cbfff700 1 -- 172.21.15.31:0/2116872252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4009790 con 0x7f02e41010d0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02cbfff700 1 -- 172.21.15.31:0/2116872252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d4010430 con 0x7f02e41010d0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e4073140 con 0x7f02e41010d0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02cbfff700 1 -- 172.21.15.31:0/2116872252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400cea0 con 0x7f02e41010d0 2023-10-06T21:27:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.788+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4073620 con 0x7f02e41010d0 2023-10-06T21:27:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.793+0000 7f02cbfff700 1 -- 172.21.15.31:0/2116872252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f02d40105a0 con 0x7f02e41010d0 2023-10-06T21:27:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 Processor -- start 2023-10-06T21:27:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 -- start start 2023-10-06T21:27:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02cbfff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 0x7f02cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870aae5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02cbfff700 1 -- 172.21.15.31:0/2116872252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f02d4014070 con 0x7f02e41010d0 2023-10-06T21:27:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aa44400 con 0x55870aadb520 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aa35bc0 con 0x55870aadac00 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aa334e0 con 0x55870aae63d0 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.794+0000 7f02e37fe700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 0x7f02cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870aae5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:46034/0 (socket says 172.21.15.31:46034) 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1294053260 learned_addr learned my addr 172.21.15.31:0/1294053260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e37fe700 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1294053260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 msgr2=0x55870aae87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1294053260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 msgr2=0x55870aae5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870aae5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1294053260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870a859d80 con 0x55870aadac00 2023-10-06T21:27:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e37fe700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 0x7f02cc0a3ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02dc003fd0 tx=0x7f02dc004040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.795+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02d4096d20 tx=0x7f02d4099f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc02dff3d1716693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.796+0000 7f02c97fa700 1 -- 172.21.15.31:0/1294053260 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40b0020 con 0x55870aadac00 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.796+0000 7f02c97fa700 1 -- 172.21.15.31:0/1294053260 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d409a2b0 con 0x55870aadac00 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.796+0000 7f02c97fa700 1 -- 172.21.15.31:0/1294053260 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d409a770 con 0x55870aadac00 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.796+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1294053260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 msgr2=0x55870aadafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.796+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02d4096d20 tx=0x7f02d4099f00 comp rx=0 tx=0).stop 2023-10-06T21:27:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1294053260 shutdown_connections 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadac00 0x55870aadafe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1294053260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870aae5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1294053260 >> 172.21.15.31:0/1294053260 conn(0x55870aa50200 msgr2=0x55870aad7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1294053260 shutdown_connections 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1294053260 wait complete. 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.797+0000 7f02fcfa8b80 1 -- start start 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 Processor -- start 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aae36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aae63d0 0x55870aae3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aa2edd0 con 0x55870aae63d0 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a9cb6b0 con 0x55870aadb520 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.798+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a9cb350 con 0x55870aadac00 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02fcfa8b80 10 client.0 _ll_register_callbacks cb 0x55870a8287a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:45020/0 (socket says 172.21.15.31:45020) 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1796736824 learned_addr learned my addr 172.21.15.31:0/1796736824 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e37fe700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aae63d0 0x55870aae3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e27fc700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aae36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 msgr2=0x55870aae36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aae36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1796736824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aae63d0 msgr2=0x55870aae3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aae63d0 0x55870aae3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.799+0000 7f02e2ffd700 1 -- 172.21.15.31:0/1796736824 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4073620 con 0x55870aadac00 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02d409a240 tx=0x7f02d4010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af8d3cd8051bbff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40b0070 con 0x55870aadac00 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f02d40a2640 con 0x55870aadac00 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55870aaced50 con 0x55870aadac00 2023-10-06T21:27:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d4098650 con 0x55870aadac00 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02fcfa8b80 10 client.26422 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55870a842c00 con 0x55870aadac00 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.800+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55870aaced50 con 0x55870aadac00 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02d40b82c0 con 0x55870aadac00 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 10 client.26422 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 10 client.26422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 20 client.26422 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870aaf5a80 0x55870aaf7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 4 client.26422 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55870aa27db0 con 0x55870aaf5a80 2023-10-06T21:27:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.801+0000 7f02e27fc700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870aaf5a80 0x55870aaf7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.802+0000 7f02c6ff5700 10 client.26422 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.802+0000 7f02e27fc700 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870aaf5a80 0x55870aaf7ea0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f03dce1b1264344 server_cookie=65222e58399a1d6b in_seq=0 out_seq=0 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.802+0000 7f02c6ff5700 1 -- 172.21.15.31:0/1796736824 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55870aa27db0 con 0x55870aaf5a80 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.802+0000 7f02c6ff5700 10 client.26422 handle_command_reply: tid=0 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 client.26422 shutdown 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 msgr2=0x55870aae31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02d409a240 tx=0x7f02d4010b80 comp rx=0 tx=0).stop 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 shutdown_connections 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870aaf5a80 0x55870aaf7ea0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aae63d0 0x55870aae3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870aae31b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/1796736824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aae36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 >> 172.21.15.31:0/1796736824 conn(0x55870aa50200 msgr2=0x55870aad6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02c6ff5700 0 client.26422 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 shutdown_connections 2023-10-06T21:27:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/1796736824 wait complete. 2023-10-06T21:27:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.803+0000 7f02fcfa8b80 20 client.26422 trim_cache size 0 max 16384 2023-10-06T21:27:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 Processor -- start 2023-10-06T21:27:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 -- start start 2023-10-06T21:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 0x55870aaecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a8af8c0 con 0x55870aadac00 2023-10-06T21:27:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aaed490 con 0x55870aadb520 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.822+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870aaddfe0 con 0x55870aae63d0 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 0x55870aaecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:46064/0 (socket says 172.21.15.31:46064) 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e27fc700 1 -- 172.21.15.31:0/616306481 learned_addr learned my addr 172.21.15.31:0/616306481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e37fe700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 -- 172.21.15.31:0/616306481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 msgr2=0x55870aae0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 -- 172.21.15.31:0/616306481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 msgr2=0x55870aaddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 -- 172.21.15.31:0/616306481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870a8801f0 con 0x55870aadac00 2023-10-06T21:27:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e27fc700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e37fe700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:27:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.823+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 0x55870aaecf20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f02d4096d20 tx=0x7f02d4010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281954cf8db95ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02c9ffb700 1 -- 172.21.15.31:0/616306481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40b0070 con 0x55870aadac00 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02c9ffb700 1 -- 172.21.15.31:0/616306481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d4096420 con 0x55870aadac00 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02c9ffb700 1 -- 172.21.15.31:0/616306481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d409a720 con 0x55870aadac00 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/616306481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 msgr2=0x55870aaecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 0x55870aaecf20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f02d4096d20 tx=0x7f02d4010bf0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/616306481 shutdown_connections 2023-10-06T21:27:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aae63d0 0x55870aae0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadac00 0x55870aaecf20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/616306481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aadb520 0x55870aaddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.824+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/616306481 >> 172.21.15.31:0/616306481 conn(0x55870aa50200 msgr2=0x55870aade4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.825+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/616306481 shutdown_connections 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.825+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/616306481 wait complete. 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.825+0000 7f02fcfa8b80 1 -- start start 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.825+0000 7f02fcfa8b80 1 Processor -- start 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870ab3da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aae63d0 0x55870ab42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870ab42dd0 con 0x55870aadb520 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870ab42f40 con 0x55870aae63d0 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870ab43100 con 0x55870aadac00 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02fcfa8b80 10 client.0 _ll_register_callbacks cb 0x7f02d4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46796/0 (socket says 172.21.15.31:46796) 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02e27fc700 1 -- 172.21.15.31:0/2048512967 learned_addr learned my addr 172.21.15.31:0/2048512967 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870ab3da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.826+0000 7f02e37fe700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aae63d0 0x55870ab42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 -- 172.21.15.31:0/2048512967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 msgr2=0x55870ab3da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870ab3da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 -- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aae63d0 msgr2=0x55870ab42890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aae63d0 0x55870ab42890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 -- 172.21.15.31:0/2048512967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e410e5e0 con 0x55870aadb520 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02e27fc700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02e406a460 tx=0x7f02d80054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa2de408ad2b0e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d802c070 con 0x55870aadb520 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d409a2b0 con 0x55870aadb520 2023-10-06T21:27:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.827+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f02d800b750 con 0x55870aadb520 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d801d040 con 0x55870aadb520 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 10 client.16749 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55870a842c00 con 0x55870aadb520 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d803c020 con 0x55870aadb520 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02d8004ba0 con 0x55870aadb520 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 20 client.16749 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870ab4a920 0x55870ab4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 4 client.16749 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:27:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.828+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55870aa27ad0 con 0x55870ab4a920 2023-10-06T21:27:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.829+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870ab4a920 0x55870ab4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.829+0000 7f02e2ffd700 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870ab4a920 0x55870ab4cd40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ae8023f80748a7 server_cookie=d03a5ae40d6457fc in_seq=0 out_seq=0 2023-10-06T21:27:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.829+0000 7f02c6ff5700 10 client.16749 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.854+0000 7f02c6ff5700 1 -- 172.21.15.31:0/2048512967 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55870aa27ad0 con 0x55870ab4a920 2023-10-06T21:27:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.854+0000 7f02c6ff5700 10 client.16749 handle_command_reply: tid=0 2023-10-06T21:27:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.854+0000 7f02fcfa8b80 1 client.16749 shutdown 2023-10-06T21:27:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 msgr2=0x55870ab3df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02e406a460 tx=0x7f02d80054f0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 shutdown_connections 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55870aae63d0 0x55870ab42890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55870aadac00 0x55870ab3da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55870ab4a920 0x55870ab4cd40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 --2- 172.21.15.31:0/2048512967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55870aadb520 0x55870ab3df70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02c6ff5700 0 client.16749 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 >> 172.21.15.31:0/2048512967 conn(0x55870aa50200 msgr2=0x55870aae4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 shutdown_connections 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 1 -- 172.21.15.31:0/2048512967 wait complete. 2023-10-06T21:27:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.855+0000 7f02fcfa8b80 20 client.16749 trim_cache size 0 max 16384 2023-10-06T21:27:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:27:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:27:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:27:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:27:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 msgr2=0x7f02cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 0x7f02cc0a3ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02dc003fd0 tx=0x7f02dc004040 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 msgr2=0x7f02e4196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f02d400e9a0 tx=0x7f02d400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 shutdown_connections 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e419bac0 0x7f02e419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f02cc0a15f0 0x7f02cc0a3ab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02e41010d0 0x7f02e4196f10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 --2- 172.21.15.31:0/2116872252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e41061f0 0x7f02e4197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 >> 172.21.15.31:0/2116872252 conn(0x7f02e40fc930 msgr2=0x7f02e40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 shutdown_connections 2023-10-06T21:27:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:36.856+0000 7f02e95ff700 1 -- 172.21.15.31:0/2116872252 wait complete. 2023-10-06T21:27:37.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 108925 1.77 MB/sec execute 207 sec latency 26.500 ms 2023-10-06T21:27:38.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 109417 1.78 MB/sec execute 208 sec latency 19.259 ms 2023-10-06T21:27:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:38 smithi142 ceph-mon[102951]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:38 smithi031 ceph-mon[98539]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:38 smithi146 ceph-mon[102809]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 864 KiB/s wr, 85 op/s 2023-10-06T21:27:39.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 109773 1.78 MB/sec execute 209 sec latency 21.056 ms 2023-10-06T21:27:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:39 smithi031 ceph-mon[98539]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:27:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:39 smithi146 ceph-mon[102809]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:27:40.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 110116 1.79 MB/sec execute 210 sec latency 25.184 ms 2023-10-06T21:27:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:39 smithi142 ceph-mon[102951]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:27:41.343 INFO:tasks.workunit.client.0.smithi031.stdout: 1 110337 1.79 MB/sec execute 211 sec latency 16.621 ms 2023-10-06T21:27:42.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 110506 1.78 MB/sec execute 212 sec latency 17.607 ms 2023-10-06T21:27:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:42 smithi142 ceph-mon[102951]: pgmap v208: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 826 KiB/s wr, 88 op/s 2023-10-06T21:27:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:42 smithi031 ceph-mon[98539]: pgmap v208: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 826 KiB/s wr, 88 op/s 2023-10-06T21:27:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:42 smithi146 ceph-mon[102809]: pgmap v208: 97 pgs: 97 active+clean; 72 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 826 KiB/s wr, 88 op/s 2023-10-06T21:27:43.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 110783 1.77 MB/sec execute 213 sec latency 20.849 ms 2023-10-06T21:27:44.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 111122 1.77 MB/sec execute 214 sec latency 20.971 ms 2023-10-06T21:27:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:44 smithi142 ceph-mon[102951]: pgmap v209: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:27:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:44 smithi031 ceph-mon[98539]: pgmap v209: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:27:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:44 smithi146 ceph-mon[102809]: pgmap v209: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-10-06T21:27:45.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 111492 1.78 MB/sec execute 215 sec latency 27.190 ms 2023-10-06T21:27:46.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 111826 1.77 MB/sec execute 216 sec latency 27.185 ms 2023-10-06T21:27:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:46 smithi142 ceph-mon[102951]: pgmap v210: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:46 smithi031 ceph-mon[98539]: pgmap v210: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:46 smithi146 ceph-mon[102809]: pgmap v210: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:47.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 112185 1.77 MB/sec execute 217 sec latency 28.964 ms 2023-10-06T21:27:48.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 112495 1.77 MB/sec execute 218 sec latency 17.934 ms 2023-10-06T21:27:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:48 smithi142 ceph-mon[102951]: pgmap v211: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:48 smithi031 ceph-mon[98539]: pgmap v211: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:48 smithi146 ceph-mon[102809]: pgmap v211: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 903 KiB/s wr, 85 op/s 2023-10-06T21:27:49.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 112983 1.78 MB/sec execute 219 sec latency 18.588 ms 2023-10-06T21:27:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 -- 172.21.15.31:0/2093982406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 msgr2=0x7fad7c10fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 --2- 172.21.15.31:0/2093982406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c10fdc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fad7400b0a0 tx=0x7fad7400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 -- 172.21.15.31:0/2093982406 shutdown_connections 2023-10-06T21:27:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 --2- 172.21.15.31:0/2093982406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c10fdc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 --2- 172.21.15.31:0/2093982406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 --2- 172.21.15.31:0/2093982406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c103ad0 0x7fad7c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 -- 172.21.15.31:0/2093982406 >> 172.21.15.31:0/2093982406 conn(0x7fad7c0fe9c0 msgr2=0x7fad7c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 -- 172.21.15.31:0/2093982406 shutdown_connections 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.027+0000 7fad809fe700 1 -- 172.21.15.31:0/2093982406 wait complete. 2023-10-06T21:27:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.028+0000 7fad809fe700 1 Processor -- start 2023-10-06T21:27:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 -- start start 2023-10-06T21:27:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19e470 con 0x7fad7c108240 2023-10-06T21:27:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19e5e0 con 0x7fad7c19d9f0 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.029+0000 7fad809fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19e7a0 con 0x7fad7c103120 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56222/0 (socket says 172.21.15.31:56222) 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 -- 172.21.15.31:0/573643614 learned_addr learned my addr 172.21.15.31:0/573643614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 -- 172.21.15.31:0/573643614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 msgr2=0x7fad7c199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 -- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 msgr2=0x7fad7c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 -- 172.21.15.31:0/573643614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74009d00 con 0x7fad7c19d9f0 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.030+0000 7fad7b59e700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad7400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.031+0000 7fad63fff700 1 -- 172.21.15.31:0/573643614 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400e040 con 0x7fad7c19d9f0 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.031+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c1a5ca0 con 0x7fad7c19d9f0 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.031+0000 7fad63fff700 1 -- 172.21.15.31:0/573643614 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad740094c0 con 0x7fad7c19d9f0 2023-10-06T21:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.031+0000 7fad63fff700 1 -- 172.21.15.31:0/573643614 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74005040 con 0x7fad7c19d9f0 2023-10-06T21:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.031+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1a6170 con 0x7fad7c19d9f0 2023-10-06T21:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.034+0000 7fad63fff700 1 -- 172.21.15.31:0/573643614 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fad7401d470 con 0x7fad7c19d9f0 2023-10-06T21:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad63fff700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 0x7fad640a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad94a82b80 1 Processor -- start 2023-10-06T21:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad63fff700 1 -- 172.21.15.31:0/573643614 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fad740ce1c0 con 0x7fad7c19d9f0 2023-10-06T21:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 0x7fad640a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad94a82b80 1 -- start start 2023-10-06T21:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.036+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fcf680 0x560f37fd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fdc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad94a82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37f38400 con 0x560f37fced60 2023-10-06T21:27:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad94a82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37f29bc0 con 0x560f37fcf680 2023-10-06T21:27:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad94a82b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37f274e0 con 0x560f37fda530 2023-10-06T21:27:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 0x7fad640a3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad6c011490 tx=0x7fad6c009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42066/0 (socket says 172.21.15.31:42066) 2023-10-06T21:27:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fdc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7ad9d700 1 -- 172.21.15.31:0/2092617285 learned_addr learned my addr 172.21.15.31:0/2092617285 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7a59c700 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fcf680 0x560f37fd9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 -- 172.21.15.31:0/2092617285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fcf680 msgr2=0x560f37fd9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fcf680 0x560f37fd9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 -- 172.21.15.31:0/2092617285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 msgr2=0x560f37fcf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7b59e700 1 -- 172.21.15.31:0/2092617285 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f37d4dd80 con 0x560f37fda530 2023-10-06T21:27:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.037+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.038+0000 7fad7b59e700 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fdc920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad740cb8a0 tx=0x7fad74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6643c84fbf1a469f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.038+0000 7fad617fa700 1 -- 172.21.15.31:0/2092617285 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7409f120 con 0x560f37fda530 2023-10-06T21:27:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.038+0000 7fad617fa700 1 -- 172.21.15.31:0/2092617285 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad7409ab20 con 0x560f37fda530 2023-10-06T21:27:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.038+0000 7fad617fa700 1 -- 172.21.15.31:0/2092617285 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740a34c0 con 0x560f37fda530 2023-10-06T21:27:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 -- 172.21.15.31:0/2092617285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 msgr2=0x560f37fdc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fdc920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad740cb8a0 tx=0x7fad74009c00 comp rx=0 tx=0).stop 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 -- 172.21.15.31:0/2092617285 shutdown_connections 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fdc920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fced60 0x560f37fcf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 --2- 172.21.15.31:0/2092617285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fcf680 0x560f37fd9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 -- 172.21.15.31:0/2092617285 >> 172.21.15.31:0/2092617285 conn(0x560f37f44200 msgr2=0x560f37fcb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 -- 172.21.15.31:0/2092617285 shutdown_connections 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.039+0000 7fad94a82b80 1 -- 172.21.15.31:0/2092617285 wait complete. 2023-10-06T21:27:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.040+0000 7fad94a82b80 1 -- start start 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.040+0000 7fad94a82b80 1 Processor -- start 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fcf680 0x560f37fd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 0x560f37fd7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37f22dd0 con 0x560f37fda530 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37ebf6b0 con 0x560f37fced60 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37ebf350 con 0x560f37fcf680 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad94a82b80 10 client.0 _ll_register_callbacks cb 0x560f37d1c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56236/0 (socket says 172.21.15.31:56236) 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 -- 172.21.15.31:0/3074472146 learned_addr learned my addr 172.21.15.31:0/3074472146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7b59e700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 0x560f37fd7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fcf680 0x560f37fd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 -- 172.21.15.31:0/3074472146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fcf680 msgr2=0x560f37fd7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fcf680 0x560f37fd7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 -- 172.21.15.31:0/3074472146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 msgr2=0x560f37fd7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 0x560f37fd7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.041+0000 7fad7ad9d700 1 -- 172.21.15.31:0/3074472146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c017070 con 0x560f37fced60 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad7b59e700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 0x560f37fd7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0046e0 tx=0x7fad6c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e27879391cf645f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c01fca0 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fad6c01fe00 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f37fc2eb0 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6c028440 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad94a82b80 10 client.26480 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.042+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fad7c1a6170 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad6c033020 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad6c02f4b0 con 0x560f37fced60 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 20 client.26480 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f37ff2700 0x560f37ff4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 4 client.26480 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f37f1bdb0 con 0x560f37ff2700 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.043+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f37ff2700 0x560f37ff4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.044+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f37ff2700 0x560f37ff4b20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf087bd12fd32244 server_cookie=f0712c46ea62ea4b in_seq=0 out_seq=0 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.044+0000 7fad5eff5700 10 client.26480 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad5eff5700 1 -- 172.21.15.31:0/3074472146 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f37f1bdb0 con 0x560f37ff2700 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad5eff5700 10 client.26480 handle_command_reply: tid=0 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 client.26480 shutdown 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 msgr2=0x560f37fd9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0046e0 tx=0x7fad6c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 shutdown_connections 2023-10-06T21:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f37ff2700 0x560f37ff4b20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fda530 0x560f37fd7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fd9f10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad5eff5700 0 client.26480 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3074472146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fcf680 0x560f37fd7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.045+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 >> 172.21.15.31:0/3074472146 conn(0x560f37f44200 msgr2=0x560f37fcada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.046+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 shutdown_connections 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.046+0000 7fad94a82b80 1 -- 172.21.15.31:0/3074472146 wait complete. 2023-10-06T21:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.046+0000 7fad94a82b80 20 client.26480 trim_cache size 0 max 16384 2023-10-06T21:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 Processor -- start 2023-10-06T21:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 -- start start 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fd6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fda530 0x560f37fd6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37da38c0 con 0x560f37fcf680 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37fcc6b0 con 0x560f37fda530 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.064+0000 7fad94a82b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37fd5830 con 0x560f37fced60 2023-10-06T21:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fd6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47750/0 (socket says 172.21.15.31:47750) 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fd6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42080/0 (socket says 172.21.15.31:42080) 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 -- 172.21.15.31:0/410532883 learned_addr learned my addr 172.21.15.31:0/410532883 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7b59e700 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fda530 0x560f37fd6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 -- 172.21.15.31:0/410532883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fda530 msgr2=0x560f37fd6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fda530 0x560f37fd6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 -- 172.21.15.31:0/410532883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 msgr2=0x560f37fd6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fd6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 -- 172.21.15.31:0/410532883 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f37d36c00 con 0x560f37fced60 2023-10-06T21:27:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.065+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004790 tx=0x7fad6c003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f2773c551bdd2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad61ffb700 1 -- 172.21.15.31:0/410532883 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02f620 con 0x560f37fced60 2023-10-06T21:27:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad61ffb700 1 -- 172.21.15.31:0/410532883 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad6c033040 con 0x560f37fced60 2023-10-06T21:27:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad61ffb700 1 -- 172.21.15.31:0/410532883 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0286a0 con 0x560f37fced60 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 -- 172.21.15.31:0/410532883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 msgr2=0x560f37fd9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004790 tx=0x7fad6c003f90 comp rx=0 tx=0).stop 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 -- 172.21.15.31:0/410532883 shutdown_connections 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fda530 0x560f37fd6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fced60 0x560f37fd9a40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 --2- 172.21.15.31:0/410532883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fd6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.066+0000 7fad94a82b80 1 -- 172.21.15.31:0/410532883 >> 172.21.15.31:0/410532883 conn(0x560f37f44200 msgr2=0x560f37fd2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.067+0000 7fad94a82b80 1 -- 172.21.15.31:0/410532883 shutdown_connections 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.067+0000 7fad94a82b80 1 -- 172.21.15.31:0/410532883 wait complete. 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.067+0000 7fad94a82b80 1 -- start start 2023-10-06T21:27:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.067+0000 7fad94a82b80 1 Processor -- start 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fe9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fe7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37fe81f0 con 0x560f37fcf680 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37fe8360 con 0x560f37fced60 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f37fe8520 con 0x560f37fda530 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad94a82b80 10 client.0 _ll_register_callbacks cb 0x7fad6c003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42092/0 (socket says 172.21.15.31:42092) 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 -- 172.21.15.31:0/3907690612 learned_addr learned my addr 172.21.15.31:0/3907690612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:27:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7b59e700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fe7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fe9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 -- 172.21.15.31:0/3907690612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 msgr2=0x560f37fe7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fe7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 -- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 msgr2=0x560f37fe9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.068+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fe9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad7a59c700 1 -- 172.21.15.31:0/3907690612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1a6170 con 0x560f37fcf680 2023-10-06T21:27:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad7a59c700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fad700097c0 tx=0x7fad70010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab9bcbc054895a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70017070 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad7409ab20 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fad7001f9f0 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.069+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad7001f580 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 10 client.16767 fetch_fsmap learned FSMap version 34 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560f37d741f0 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad70033080 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad7002fc10 con 0x560f37fcf680 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:27:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 20 client.16767 populate_metadata read hostname 'smithi031' 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f38047980 0x560f38049da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 4 client.16767 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f37f1bad0 con 0x560f38047980 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.070+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f38047980 0x560f38049da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.071+0000 7fad7ad9d700 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f38047980 0x560f38049da0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8f206d6afad8d4 server_cookie=ea0f5a8903c808d3 in_seq=0 out_seq=0 2023-10-06T21:27:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.071+0000 7fad5eff5700 10 client.16767 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.096+0000 7fad5eff5700 1 -- 172.21.15.31:0/3907690612 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f37f1bad0 con 0x560f38047980 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.096+0000 7fad5eff5700 10 client.16767 handle_command_reply: tid=0 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.096+0000 7fad94a82b80 1 client.16767 shutdown 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.096+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 msgr2=0x560f37fe7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.096+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fad700097c0 tx=0x7fad70010710 comp rx=0 tx=0).stop 2023-10-06T21:27:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 shutdown_connections 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560f37fda530 0x560f37fe7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f37fced60 0x560f37fe9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560f38047980 0x560f38049da0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 --2- 172.21.15.31:0/3907690612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f37fcf680 0x560f37fe7770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 >> 172.21.15.31:0/3907690612 conn(0x560f37f44200 msgr2=0x560f37fd0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad5eff5700 0 client.16767 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 shutdown_connections 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 1 -- 172.21.15.31:0/3907690612 wait complete. 2023-10-06T21:27:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.097+0000 7fad94a82b80 20 client.16767 trim_cache size 0 max 16384 2023-10-06T21:27:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:27:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:27:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:27:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 msgr2=0x7fad640a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 0x7fad640a3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad6c011490 tx=0x7fad6c009260 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 msgr2=0x7fad7c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad7400bc80 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 shutdown_connections 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7c19d9f0 0x7fad7c19de70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad7c103120 0x7fad7c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fad640a1670 0x7fad640a3b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 --2- 172.21.15.31:0/573643614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad7c108240 0x7fad7c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 >> 172.21.15.31:0/573643614 conn(0x7fad7c0fe9c0 msgr2=0x7fad7c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 shutdown_connections 2023-10-06T21:27:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:27:50.098+0000 7fad809fe700 1 -- 172.21.15.31:0/573643614 wait complete. 2023-10-06T21:27:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:49 smithi031 ceph-mon[98539]: pgmap v212: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2023-10-06T21:27:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:49 smithi146 ceph-mon[102809]: pgmap v212: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2023-10-06T21:27:50.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 113325 1.78 MB/sec execute 220 sec latency 24.868 ms 2023-10-06T21:27:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:49 smithi142 ceph-mon[102951]: pgmap v212: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2023-10-06T21:27:51.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 113681 1.79 MB/sec execute 221 sec latency 17.206 ms 2023-10-06T21:27:52.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 113887 1.79 MB/sec execute 222 sec latency 18.414 ms 2023-10-06T21:27:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:52 smithi142 ceph-mon[102951]: pgmap v213: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 870 KiB/s wr, 83 op/s 2023-10-06T21:27:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:52 smithi031 ceph-mon[98539]: pgmap v213: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 870 KiB/s wr, 83 op/s 2023-10-06T21:27:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:52 smithi146 ceph-mon[102809]: pgmap v213: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 870 KiB/s wr, 83 op/s 2023-10-06T21:27:53.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 114067 1.78 MB/sec execute 223 sec latency 17.539 ms 2023-10-06T21:27:54.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 114363 1.77 MB/sec execute 224 sec latency 24.820 ms 2023-10-06T21:27:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:54 smithi142 ceph-mon[102951]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2023-10-06T21:27:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:54 smithi031 ceph-mon[98539]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2023-10-06T21:27:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:54 smithi146 ceph-mon[102809]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2023-10-06T21:27:55.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 114699 1.77 MB/sec execute 225 sec latency 18.678 ms 2023-10-06T21:27:56.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 115050 1.78 MB/sec execute 226 sec latency 25.448 ms 2023-10-06T21:27:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:56 smithi142 ceph-mon[102951]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:56 smithi031 ceph-mon[98539]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:56 smithi146 ceph-mon[102809]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:57.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 115387 1.77 MB/sec execute 227 sec latency 28.005 ms 2023-10-06T21:27:58.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 115748 1.76 MB/sec execute 228 sec latency 27.827 ms 2023-10-06T21:27:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:58 smithi142 ceph-mon[102951]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:58 smithi031 ceph-mon[98539]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:58 smithi146 ceph-mon[102809]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 941 KiB/s wr, 88 op/s 2023-10-06T21:27:59.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 116049 1.77 MB/sec execute 229 sec latency 19.712 ms 2023-10-06T21:28:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:27:59 smithi031 ceph-mon[98539]: pgmap v217: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:28:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:27:59 smithi146 ceph-mon[102809]: pgmap v217: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:28:00.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 116535 1.78 MB/sec execute 230 sec latency 19.091 ms 2023-10-06T21:28:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:27:59 smithi142 ceph-mon[102951]: pgmap v217: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:28:01.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 116874 1.78 MB/sec execute 231 sec latency 19.100 ms 2023-10-06T21:28:02.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 117238 1.79 MB/sec execute 232 sec latency 19.708 ms 2023-10-06T21:28:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:02 smithi142 ceph-mon[102951]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:28:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:02 smithi031 ceph-mon[98539]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:28:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:02 smithi146 ceph-mon[102809]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 873 KiB/s wr, 89 op/s 2023-10-06T21:28:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 -- 172.21.15.31:0/1174998609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e4101a80 msgr2=0x7f56e4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/1174998609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e4101a80 0x7f56e4105ac0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009990 tx=0x7f56d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 -- 172.21.15.31:0/1174998609 shutdown_connections 2023-10-06T21:28:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/1174998609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41061f0 0x7f56e410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/1174998609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 0x7f56e41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/1174998609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e4101a80 0x7f56e4105ac0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 -- 172.21.15.31:0/1174998609 >> 172.21.15.31:0/1174998609 conn(0x7f56e40fc930 msgr2=0x7f56e40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 -- 172.21.15.31:0/1174998609 shutdown_connections 2023-10-06T21:28:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.262+0000 7f56eb7ff700 1 -- 172.21.15.31:0/1174998609 wait complete. 2023-10-06T21:28:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.263+0000 7f56eb7ff700 1 Processor -- start 2023-10-06T21:28:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.263+0000 7f56eb7ff700 1 -- start start 2023-10-06T21:28:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.263+0000 7f56eb7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 0x7f56e4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.263+0000 7f56eb7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56eb7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 0x7f56e41a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56eb7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5a20 con 0x7f56e41061f0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56eb7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5b60 con 0x7f56e41a30f0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56eb7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5cd0 con 0x7f56e41010d0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37804/0 (socket says 172.21.15.31:37804) 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2499144252 learned_addr learned my addr 172.21.15.31:0/2499144252 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e959b700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 0x7f56e4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 0x7f56e41a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2499144252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 msgr2=0x7f56e4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 0x7f56e4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2499144252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 msgr2=0x7f56e41a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 0x7f56e41a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2499144252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40095f0 con 0x7f56e41061f0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.264+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 0x7f56e41a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005770 tx=0x7f56d40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56da7fc700 1 -- 172.21.15.31:0/2499144252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d401d070 con 0x7f56e41061f0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56da7fc700 1 -- 172.21.15.31:0/2499144252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56d4022470 con 0x7f56e41061f0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41a5f50 con 0x7f56e41061f0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56da7fc700 1 -- 172.21.15.31:0/2499144252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4017650 con 0x7f56e41061f0 2023-10-06T21:28:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.265+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41a6310 con 0x7f56e41061f0 2023-10-06T21:28:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.269+0000 7f56da7fc700 1 -- 172.21.15.31:0/2499144252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f56d4022850 con 0x7f56e41061f0 2023-10-06T21:28:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56da7fc700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 0x7f56d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56ff121b80 1 Processor -- start 2023-10-06T21:28:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56da7fc700 1 -- 172.21.15.31:0/2499144252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f56d40ce870 con 0x7f56e41061f0 2023-10-06T21:28:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56e959b700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 0x7f56d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56ff121b80 1 -- start start 2023-10-06T21:28:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.271+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccaffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56ff121b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cc19400 con 0x56342ccbb3d0 2023-10-06T21:28:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56ff121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cc0abc0 con 0x56342ccafc00 2023-10-06T21:28:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56ff121b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cc084e0 con 0x56342ccb0520 2023-10-06T21:28:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37818/0 (socket says 172.21.15.31:37818) 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e9d9c700 1 -- 172.21.15.31:0/2341979191 learned_addr learned my addr 172.21.15.31:0/2341979191 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e959b700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 0x7f56d00a3ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007be0 tx=0x7f56e00073d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e959b700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccaffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2341979191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 msgr2=0x56342ccaffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccaffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2341979191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 msgr2=0x56342ccbd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e8d9a700 1 -- 172.21.15.31:0/2341979191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56342ca2ed80 con 0x56342ccb0520 2023-10-06T21:28:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.272+0000 7f56e959b700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccaffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.273+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbae90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56d40cbef0 tx=0x7f56d409b170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab459ad2206221c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.273+0000 7f56cf7fe700 1 -- 172.21.15.31:0/2341979191 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d409f960 con 0x56342ccb0520 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.273+0000 7f56cf7fe700 1 -- 172.21.15.31:0/2341979191 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56d40a3b10 con 0x56342ccb0520 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.273+0000 7f56cf7fe700 1 -- 172.21.15.31:0/2341979191 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40860b0 con 0x56342ccb0520 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- 172.21.15.31:0/2341979191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 msgr2=0x56342ccbae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbae90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56d40cbef0 tx=0x7f56d409b170 comp rx=0 tx=0).stop 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- 172.21.15.31:0/2341979191 shutdown_connections 2023-10-06T21:28:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccaffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 --2- 172.21.15.31:0/2341979191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbae90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- 172.21.15.31:0/2341979191 >> 172.21.15.31:0/2341979191 conn(0x56342cc25200 msgr2=0x56342ccac080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- 172.21.15.31:0/2341979191 shutdown_connections 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- 172.21.15.31:0/2341979191 wait complete. 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.274+0000 7f56ff121b80 1 -- start start 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 Processor -- start 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccbbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccbb3d0 0x56342ccc6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cc03dd0 con 0x56342ccb0520 2023-10-06T21:28:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cba06b0 con 0x56342ccbb3d0 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342cba0350 con 0x56342ccafc00 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.275+0000 7f56ff121b80 10 client.0 _ll_register_callbacks cb 0x56342c9fd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44606/0 (socket says 172.21.15.31:44606) 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e9d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccbb3d0 0x56342ccc6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccbbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 -- 172.21.15.31:0/1837254786 learned_addr learned my addr 172.21.15.31:0/1837254786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 -- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccbb3d0 msgr2=0x56342ccc6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccbb3d0 0x56342ccc6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 -- 172.21.15.31:0/1837254786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 msgr2=0x56342ccbbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccbbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 -- 172.21.15.31:0/1837254786 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc0090f0 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56e959b700 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56e0004360 tx=0x7f56e00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ed01d4887603ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.276+0000 7f56ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0003a90 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f56e0017070 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56342cca3d50 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e0003bf0 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ff121b80 10 client.26455 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f56e41a6310 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56342cca3d50 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.277+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56e0020410 con 0x56342ccafc00 2023-10-06T21:28:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 10 client.26455 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 20 client.26455 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cccaba0 0x56342ccccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 4 client.26455 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56342cbfcdb0 con 0x56342cccaba0 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.278+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cccaba0 0x56342ccccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.279+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cccaba0 0x56342ccccfc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb1f8fd5ddd1775 server_cookie=549462b8e16a91d5 in_seq=0 out_seq=0 2023-10-06T21:28:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.279+0000 7f56ccff9700 10 client.26455 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.279+0000 7f56ccff9700 1 -- 172.21.15.31:0/1837254786 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56342cbfcdb0 con 0x56342cccaba0 2023-10-06T21:28:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.279+0000 7f56ccff9700 10 client.26455 handle_command_reply: tid=0 2023-10-06T21:28:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.279+0000 7f56ff121b80 1 client.26455 shutdown 2023-10-06T21:28:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 msgr2=0x56342ccbb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56e0004360 tx=0x7f56e00047c0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 shutdown_connections 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccbb3d0 0x56342ccc6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccafc00 0x56342ccbb7b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cccaba0 0x56342ccccfc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 --2- 172.21.15.31:0/1837254786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccbbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ccff9700 0 client.26455 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 >> 172.21.15.31:0/1837254786 conn(0x56342cc25200 msgr2=0x56342ccabc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 shutdown_connections 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 1 -- 172.21.15.31:0/1837254786 wait complete. 2023-10-06T21:28:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.280+0000 7f56ff121b80 20 client.26455 trim_cache size 0 max 16384 2023-10-06T21:28:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 Processor -- start 2023-10-06T21:28:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 -- start start 2023-10-06T21:28:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccc07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ca848c0 con 0x56342ccbb3d0 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ccb5320 con 0x56342ccafc00 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.302+0000 7f56ff121b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ccc0d20 con 0x56342ccb0520 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccc07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37840/0 (socket says 172.21.15.31:37840) 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccc07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39822/0 (socket says 172.21.15.31:39822) 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 -- 172.21.15.31:0/486918128 learned_addr learned my addr 172.21.15.31:0/486918128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 -- 172.21.15.31:0/486918128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 msgr2=0x56342ccbb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 -- 172.21.15.31:0/486918128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 msgr2=0x56342ccc07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccc07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 -- 172.21.15.31:0/486918128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56342ca17c00 con 0x56342ccbb3d0 2023-10-06T21:28:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.303+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0077a0 tx=0x7f56dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17dd854d4171ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.304+0000 7f56cffff700 1 -- 172.21.15.31:0/486918128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00e040 con 0x56342ccbb3d0 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.304+0000 7f56cffff700 1 -- 172.21.15.31:0/486918128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56dc00f040 con 0x56342ccbb3d0 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.304+0000 7f56cffff700 1 -- 172.21.15.31:0/486918128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc005370 con 0x56342ccbb3d0 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- 172.21.15.31:0/486918128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 msgr2=0x56342ccbbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0077a0 tx=0x7f56dc00b040 comp rx=0 tx=0).stop 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- 172.21.15.31:0/486918128 shutdown_connections 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccbb3d0 0x56342ccbbcf0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccc07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 --2- 172.21.15.31:0/486918128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccb0520 0x56342ccbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- 172.21.15.31:0/486918128 >> 172.21.15.31:0/486918128 conn(0x56342cc25200 msgr2=0x56342ccb5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- 172.21.15.31:0/486918128 shutdown_connections 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- 172.21.15.31:0/486918128 wait complete. 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 -- start start 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.305+0000 7f56ff121b80 1 Processor -- start 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccbb3d0 0x56342ccb6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ccb8550 con 0x56342ccb0520 2023-10-06T21:28:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ccb71e0 con 0x56342ccafc00 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342ccb73a0 con 0x56342ccbb3d0 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56ff121b80 10 client.0 _ll_register_callbacks cb 0x7f56e0010d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56e959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56e8d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56e959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39834/0 (socket says 172.21.15.31:39834) 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56e959b700 1 -- 172.21.15.31:0/3079468385 learned_addr learned my addr 172.21.15.31:0/3079468385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.306+0000 7f56e9d9c700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccbb3d0 0x56342ccb6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 -- 172.21.15.31:0/3079468385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccbb3d0 msgr2=0x56342ccb6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccbb3d0 0x56342ccb6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 -- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 msgr2=0x56342ccb7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 -- 172.21.15.31:0/3079468385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc007500 con 0x56342ccb0520 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e959b700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56e8d9a700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccb6760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x56342ccb0a70 tx=0x7f56d409c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0f9f0904e9dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40b99a0 con 0x56342ccb0520 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56d40a3b10 con 0x56342ccb0520 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.307+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f56d4099bb0 con 0x56342ccb0520 2023-10-06T21:28:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d400fb40 con 0x56342ccb0520 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 10 client.16785 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56342ca551f0 con 0x56342ccb0520 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56d40a3b10 con 0x56342ccb0520 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56d409ba70 con 0x56342ccb0520 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 20 client.16785 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cd1f940 0x56342cd21d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 4 client.16785 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.308+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56342cbfcad0 con 0x56342cd1f940 2023-10-06T21:28:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.309+0000 7f56e959b700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cd1f940 0x56342cd21d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.309+0000 7f56ccff9700 10 client.16785 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.309+0000 7f56e959b700 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cd1f940 0x56342cd21d60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111e082d50ddf8dd server_cookie=17b2fceaf129b6e4 in_seq=0 out_seq=0 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ccff9700 1 -- 172.21.15.31:0/3079468385 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56342cbfcad0 con 0x56342cd1f940 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ccff9700 10 client.16785 handle_command_reply: tid=0 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 client.16785 shutdown 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 msgr2=0x56342ccb6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccb6760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x56342ccb0a70 tx=0x7f56d409c050 comp rx=0 tx=0).stop 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 shutdown_connections 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56342ccbb3d0 0x56342ccb6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56342ccafc00 0x56342ccb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56342cd1f940 0x56342cd21d60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 --2- 172.21.15.31:0/3079468385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56342ccb0520 0x56342ccb6760 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ccff9700 0 client.16785 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.332+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 >> 172.21.15.31:0/3079468385 conn(0x56342cc25200 msgr2=0x56342ccbb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 shutdown_connections 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56ff121b80 1 -- 172.21.15.31:0/3079468385 wait complete. 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56ff121b80 20 client.16785 trim_cache size 0 max 16384 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:28:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:28:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 msgr2=0x7f56d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 0x7f56d00a3ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007be0 tx=0x7f56e00073d0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 msgr2=0x7f56e40738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.333+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005770 tx=0x7f56d40057a0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 shutdown_connections 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e41a30f0 0x7f56e41a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56e41010d0 0x7f56e4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f56d00a15f0 0x7f56d00a3ab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 --2- 172.21.15.31:0/2499144252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56e41061f0 0x7f56e40738d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 >> 172.21.15.31:0/2499144252 conn(0x7f56e40fc930 msgr2=0x7f56e40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 shutdown_connections 2023-10-06T21:28:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:03.334+0000 7f56eb7ff700 1 -- 172.21.15.31:0/2499144252 wait complete. 2023-10-06T21:28:03.344 INFO:tasks.workunit.client.0.smithi031.stdout: 1 117450 1.78 MB/sec execute 233 sec latency 18.547 ms 2023-10-06T21:28:04.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 117592 1.78 MB/sec execute 234 sec latency 130.031 ms 2023-10-06T21:28:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:04 smithi142 ceph-mon[102951]: pgmap v219: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-10-06T21:28:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:04 smithi031 ceph-mon[98539]: pgmap v219: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-10-06T21:28:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:04 smithi146 ceph-mon[102809]: pgmap v219: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-10-06T21:28:05.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 117755 1.77 MB/sec execute 235 sec latency 328.118 ms 2023-10-06T21:28:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:05 smithi142 ceph-mon[102951]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:28:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:05 smithi031 ceph-mon[98539]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:28:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:05 smithi146 ceph-mon[102809]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:28:06.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 118068 1.76 MB/sec execute 236 sec latency 405.834 ms 2023-10-06T21:28:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:06 smithi142 ceph-mon[102951]: pgmap v221: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:06 smithi031 ceph-mon[98539]: pgmap v221: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:06 smithi146 ceph-mon[102809]: pgmap v221: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:07.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 118474 1.78 MB/sec execute 237 sec latency 17.984 ms 2023-10-06T21:28:08.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 118747 1.77 MB/sec execute 238 sec latency 27.298 ms 2023-10-06T21:28:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:08 smithi142 ceph-mon[102951]: pgmap v222: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:08 smithi031 ceph-mon[98539]: pgmap v222: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:08 smithi146 ceph-mon[102809]: pgmap v222: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:28:09.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 119128 1.76 MB/sec execute 239 sec latency 30.376 ms 2023-10-06T21:28:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:09 smithi031 ceph-mon[98539]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:09 smithi146 ceph-mon[102809]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:10.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 119486 1.76 MB/sec execute 240 sec latency 25.106 ms 2023-10-06T21:28:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:09 smithi142 ceph-mon[102951]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:11.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 119869 1.77 MB/sec execute 241 sec latency 18.263 ms 2023-10-06T21:28:12.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 120382 1.78 MB/sec execute 242 sec latency 20.552 ms 2023-10-06T21:28:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:12 smithi142 ceph-mon[102951]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:12 smithi031 ceph-mon[98539]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:12 smithi146 ceph-mon[102809]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-10-06T21:28:13.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 120643 1.78 MB/sec execute 243 sec latency 20.804 ms 2023-10-06T21:28:14.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 120905 1.78 MB/sec execute 244 sec latency 19.951 ms 2023-10-06T21:28:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:14 smithi142 ceph-mon[102951]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:14 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:14 smithi142 ceph-mon[102951]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:28:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:14 smithi031 ceph-mon[98539]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:14 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:14 smithi031 ceph-mon[98539]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:28:14.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:14 smithi146 ceph-mon[102809]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:14.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:14 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:14.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:14 smithi146 ceph-mon[102809]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:28:15.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 121031 1.77 MB/sec execute 245 sec latency 240.337 ms 2023-10-06T21:28:15.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:15 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:15.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:15 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:15.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:15 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:28:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:15 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:15 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:15 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:28:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:15 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:15 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:15 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:28:16.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 121200 1.77 MB/sec execute 246 sec latency 304.714 ms 2023-10-06T21:28:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:16 smithi142 ceph-mon[102951]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.502+0000 7fd53b3ff700 1 -- 172.21.15.31:0/3382502577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5340682d0 msgr2=0x7fd534068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.502+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/3382502577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5340682d0 0x7fd534068750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd524009990 tx=0x7fd524009ca0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 -- 172.21.15.31:0/3382502577 shutdown_connections 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/3382502577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd534068c90 0x7fd53410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/3382502577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 0x7fd534107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/3382502577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5340682d0 0x7fd534068750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 -- 172.21.15.31:0/3382502577 >> 172.21.15.31:0/3382502577 conn(0x7fd534076b70 msgr2=0x7fd534076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 -- 172.21.15.31:0/3382502577 shutdown_connections 2023-10-06T21:28:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.503+0000 7fd53b3ff700 1 -- 172.21.15.31:0/3382502577 wait complete. 2023-10-06T21:28:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.504+0000 7fd53b3ff700 1 Processor -- start 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.504+0000 7fd53b3ff700 1 -- start start 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340682d0 0x7fd534100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 0x7fd5341a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341a55b0 con 0x7fd534068c90 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341a56f0 con 0x7fd534106f60 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341a5830 con 0x7fd5340682d0 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41610/0 (socket says 172.21.15.31:41610) 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 -- 172.21.15.31:0/2642216500 learned_addr learned my addr 172.21.15.31:0/2642216500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53919b700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340682d0 0x7fd534100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53999c700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 0x7fd5341a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 -- 172.21.15.31:0/2642216500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340682d0 msgr2=0x7fd534100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340682d0 0x7fd534100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 -- 172.21.15.31:0/2642216500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 msgr2=0x7fd5341a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 0x7fd5341a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.505+0000 7fd53899a700 1 -- 172.21.15.31:0/2642216500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240095f0 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd53899a700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd524005770 tx=0x7fd5240057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd52a7fc700 1 -- 172.21.15.31:0/2642216500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52401d070 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5341a5a60 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd52a7fc700 1 -- 172.21.15.31:0/2642216500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd524021410 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd52a7fc700 1 -- 172.21.15.31:0/2642216500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524017650 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.506+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5341a5f10 con 0x7fd534068c90 2023-10-06T21:28:16.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:16 smithi031 ceph-mon[98539]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.510+0000 7fd52a7fc700 1 -- 172.21.15.31:0/2642216500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd5240217f0 con 0x7fd534068c90 2023-10-06T21:28:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.511+0000 7fd54ece9b80 1 Processor -- start 2023-10-06T21:28:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.511+0000 7fd54ece9b80 1 -- start start 2023-10-06T21:28:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd52a7fc700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 0x7fd5200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd52a7fc700 1 -- 172.21.15.31:0/2642216500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd5240ce4b0 con 0x7fd534068c90 2023-10-06T21:28:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375dc3d0 0x5629375de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293753a400 con 0x5629375d1520 2023-10-06T21:28:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293752bbc0 con 0x5629375dc3d0 2023-10-06T21:28:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375294e0 con 0x5629375d0c00 2023-10-06T21:28:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.512+0000 7fd53999c700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 0x7fd5200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41490/0 (socket says 172.21.15.31:41490) 2023-10-06T21:28:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 -- 172.21.15.31:0/2895399187 learned_addr learned my addr 172.21.15.31:0/2895399187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53999c700 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375dc3d0 0x5629375de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 -- 172.21.15.31:0/2895399187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375dc3d0 msgr2=0x5629375de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375dc3d0 0x5629375de7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 -- 172.21.15.31:0/2895399187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 msgr2=0x5629375dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375dbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.513+0000 7fd53919b700 1 -- 172.21.15.31:0/2895399187 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293734fd80 con 0x5629375d0c00 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.514+0000 7fd53999c700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 0x7fd5200a3ab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009fd0 tx=0x7fd52c00f850 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.514+0000 7fd53919b700 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd530009d00 tx=0x7fd530011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc456987911439b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.514+0000 7fd51f7fe700 1 -- 172.21.15.31:0/2895399187 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530004230 con 0x5629375d0c00 2023-10-06T21:28:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.514+0000 7fd51f7fe700 1 -- 172.21.15.31:0/2895399187 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd530004390 con 0x5629375d0c00 2023-10-06T21:28:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.514+0000 7fd51f7fe700 1 -- 172.21.15.31:0/2895399187 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530019660 con 0x5629375d0c00 2023-10-06T21:28:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2895399187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 msgr2=0x5629375d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd530009d00 tx=0x7fd530011720 comp rx=0 tx=0).stop 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2895399187 shutdown_connections 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375dc3d0 0x5629375de7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375d0fe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2895399187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2895399187 >> 172.21.15.31:0/2895399187 conn(0x562937546200 msgr2=0x5629375cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2895399187 shutdown_connections 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.515+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2895399187 wait complete. 2023-10-06T21:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 -- start start 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 Processor -- start 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375dc3d0 0x5629375d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562937524dd0 con 0x5629375dc3d0 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629374c16b0 con 0x5629375d1520 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629374c1350 con 0x5629375d0c00 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.516+0000 7fd54ece9b80 10 client.0 _ll_register_callbacks cb 0x56293731e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375dc3d0 0x5629375d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:41496/0 (socket says 172.21.15.31:41496) 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53919b700 1 -- 172.21.15.31:0/154451723 learned_addr learned my addr 172.21.15.31:0/154451723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 -- 172.21.15.31:0/154451723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 msgr2=0x5629375dbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 -- 172.21.15.31:0/154451723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375dc3d0 msgr2=0x5629375d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375dc3d0 0x5629375d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 -- 172.21.15.31:0/154451723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd530009960 con 0x5629375d1520 2023-10-06T21:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53919b700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd53899a700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d8f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5240cbb90 tx=0x7fd52400b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65329a8225dba5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.517+0000 7fd51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52409aba0 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd52409bbb0 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629375c4d50 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd52409ae10 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 10 client.26516 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562937338c00 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5629375c4d50 con 0x5629375d1520 2023-10-06T21:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5240a33b0 con 0x5629375d1520 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 20 client.26516 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.518+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629375ebb40 0x5629375edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.519+0000 7fd54ece9b80 4 client.26516 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.519+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56293751ddb0 con 0x5629375ebb40 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.519+0000 7fd53919b700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629375ebb40 0x5629375edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.519+0000 7fd53919b700 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629375ebb40 0x5629375edf60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36213a40266a5152 server_cookie=6e75d366393dc57 in_seq=0 out_seq=0 2023-10-06T21:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.519+0000 7fd51cff9700 10 client.26516 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd51cff9700 1 -- 172.21.15.31:0/154451723 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56293751ddb0 con 0x5629375ebb40 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd51cff9700 10 client.26516 handle_command_reply: tid=0 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 client.26516 shutdown 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 msgr2=0x5629375d8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d8f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5240cbb90 tx=0x7fd52400b310 comp rx=0 tx=0).stop 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 shutdown_connections 2023-10-06T21:28:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629375ebb40 0x5629375edf60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375dc3d0 0x5629375d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375d0c00 0x5629375dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd51cff9700 0 client.26516 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/154451723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d8f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.520+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 >> 172.21.15.31:0/154451723 conn(0x562937546200 msgr2=0x5629375ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.521+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 shutdown_connections 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.521+0000 7fd54ece9b80 1 -- 172.21.15.31:0/154451723 wait complete. 2023-10-06T21:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.521+0000 7fd54ece9b80 20 client.26516 trim_cache size 0 max 16384 2023-10-06T21:28:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 Processor -- start 2023-10-06T21:28:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 -- start start 2023-10-06T21:28:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629373a58c0 con 0x5629375d1520 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375cde20 con 0x5629375d0c00 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.538+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375e3500 con 0x5629375dc3d0 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:47908/0 (socket says 172.21.15.31:47908) 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 -- 172.21.15.31:0/2406352337 learned_addr learned my addr 172.21.15.31:0/2406352337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53999c700 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 -- 172.21.15.31:0/2406352337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 msgr2=0x5629375d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 -- 172.21.15.31:0/2406352337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 msgr2=0x5629375db880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375db880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 -- 172.21.15.31:0/2406352337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629373761f0 con 0x5629375d0c00 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53899a700 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.539+0000 7fd53919b700 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd530003e80 tx=0x7fd530009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af1079920901c29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd51ffff700 1 -- 172.21.15.31:0/2406352337 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530005ce0 con 0x5629375d0c00 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd51ffff700 1 -- 172.21.15.31:0/2406352337 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd530004500 con 0x5629375d0c00 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd51ffff700 1 -- 172.21.15.31:0/2406352337 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300046f0 con 0x5629375d0c00 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2406352337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 msgr2=0x5629375e2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd530003e80 tx=0x7fd530009260 comp rx=0 tx=0).stop 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2406352337 shutdown_connections 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d0c00 0x5629375e2f40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.540+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/2406352337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d1520 0x5629375db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.541+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2406352337 >> 172.21.15.31:0/2406352337 conn(0x562937546200 msgr2=0x5629375cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.541+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2406352337 shutdown_connections 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.541+0000 7fd54ece9b80 1 -- 172.21.15.31:0/2406352337 wait complete. 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.541+0000 7fd54ece9b80 1 -- start start 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.541+0000 7fd54ece9b80 1 Processor -- start 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375d5720 con 0x5629375d0c00 2023-10-06T21:28:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375d5890 con 0x5629375d1520 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629375d5a50 con 0x5629375dc3d0 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd54ece9b80 10 client.0 _ll_register_callbacks cb 0x7fd5240977f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53899a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41648/0 (socket says 172.21.15.31:41648) 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 -- 172.21.15.31:0/3488413078 learned_addr learned my addr 172.21.15.31:0/3488413078 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53999c700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 -- 172.21.15.31:0/3488413078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 msgr2=0x5629375d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 -- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 msgr2=0x5629375d6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.542+0000 7fd53919b700 1 -- 172.21.15.31:0/3488413078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562937338c00 con 0x5629375d0c00 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd53919b700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd530003e50 tx=0x7fd530005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf761b32e8125b09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530005240 con 0x5629375d0c00 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd5300053a0 con 0x5629375d0c00 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd530004390 con 0x5629375d0c00 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd530003bd0 con 0x5629375d0c00 2023-10-06T21:28:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd54ece9b80 10 client.16803 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56293734fd80 con 0x5629375d0c00 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.543+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd530004390 con 0x5629375d0c00 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd530031050 con 0x5629375d0c00 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 20 client.16803 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629376404e0 0x5629376428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 4 client.16803 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56293751dad0 con 0x5629376404e0 2023-10-06T21:28:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.544+0000 7fd53899a700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629376404e0 0x5629376428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.545+0000 7fd53899a700 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629376404e0 0x5629376428d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3aa5b485bf7e610 server_cookie=424e861659514ef8 in_seq=0 out_seq=0 2023-10-06T21:28:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.545+0000 7fd51cff9700 10 client.16803 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:16.553 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:16 smithi146 ceph-mon[102809]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd51cff9700 1 -- 172.21.15.31:0/3488413078 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56293751dad0 con 0x5629376404e0 2023-10-06T21:28:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd51cff9700 10 client.16803 handle_command_reply: tid=0 2023-10-06T21:28:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 client.16803 shutdown 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 msgr2=0x5629375d6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd530003e50 tx=0x7fd530005c20 comp rx=0 tx=0).stop 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 shutdown_connections 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629375dc3d0 0x5629375d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5629376404e0 0x5629376428d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629375d0c00 0x5629375d6340 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 --2- 172.21.15.31:0/3488413078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5629375d1520 0x5629375d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd51cff9700 0 client.16803 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.568+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 >> 172.21.15.31:0/3488413078 conn(0x562937546200 msgr2=0x5629375d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.569+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 shutdown_connections 2023-10-06T21:28:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.569+0000 7fd54ece9b80 1 -- 172.21.15.31:0/3488413078 wait complete. 2023-10-06T21:28:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.569+0000 7fd54ece9b80 20 client.16803 trim_cache size 0 max 16384 2023-10-06T21:28:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:28:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:28:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:28:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 msgr2=0x7fd5200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 0x7fd5200a3ab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009fd0 tx=0x7fd52c00f850 comp rx=0 tx=0).stop 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 msgr2=0x7fd534100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd524005770 tx=0x7fd5240057a0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 shutdown_connections 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd534106f60 0x7fd5341a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340682d0 0x7fd534100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd5200a15f0 0x7fd5200a3ab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 --2- 172.21.15.31:0/2642216500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd534068c90 0x7fd534100580 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 >> 172.21.15.31:0/2642216500 conn(0x7fd534076b70 msgr2=0x7fd5340feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 shutdown_connections 2023-10-06T21:28:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:16.570+0000 7fd53b3ff700 1 -- 172.21.15.31:0/2642216500 wait complete. 2023-10-06T21:28:17.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 121514 1.76 MB/sec execute 247 sec latency 19.212 ms 2023-10-06T21:28:18.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 121844 1.76 MB/sec execute 248 sec latency 16.217 ms 2023-10-06T21:28:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:18 smithi142 ceph-mon[102951]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:18 smithi031 ceph-mon[98539]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:18 smithi146 ceph-mon[102809]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-06T21:28:19.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 122203 1.77 MB/sec execute 249 sec latency 26.453 ms 2023-10-06T21:28:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:19 smithi031 ceph-mon[98539]: pgmap v229: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:20.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:19 smithi146 ceph-mon[102809]: pgmap v229: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:20.345 INFO:tasks.workunit.client.0.smithi031.stdout: 1 122542 1.76 MB/sec execute 250 sec latency 28.835 ms 2023-10-06T21:28:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:19 smithi142 ceph-mon[102951]: pgmap v229: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:21.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 122957 1.76 MB/sec execute 251 sec latency 25.990 ms 2023-10-06T21:28:22.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 123229 1.76 MB/sec execute 252 sec latency 17.794 ms 2023-10-06T21:28:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:22 smithi142 ceph-mon[102951]: pgmap v230: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:22 smithi031 ceph-mon[98539]: pgmap v230: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:22 smithi146 ceph-mon[102809]: pgmap v230: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2023-10-06T21:28:23.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 123818 1.77 MB/sec execute 253 sec latency 18.601 ms 2023-10-06T21:28:24.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 124008 1.77 MB/sec execute 254 sec latency 19.090 ms 2023-10-06T21:28:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:24 smithi142 ceph-mon[102951]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2023-10-06T21:28:24.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:24 smithi031 ceph-mon[98539]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2023-10-06T21:28:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:24 smithi146 ceph-mon[102809]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2023-10-06T21:28:25.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 124406 1.78 MB/sec execute 255 sec latency 16.678 ms 2023-10-06T21:28:26.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 124567 1.77 MB/sec execute 256 sec latency 20.268 ms 2023-10-06T21:28:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:26 smithi142 ceph-mon[102951]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 952 KiB/s wr, 96 op/s 2023-10-06T21:28:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:26 smithi031 ceph-mon[98539]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 952 KiB/s wr, 96 op/s 2023-10-06T21:28:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:26 smithi146 ceph-mon[102809]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 952 KiB/s wr, 96 op/s 2023-10-06T21:28:27.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 124757 1.77 MB/sec execute 257 sec latency 17.006 ms 2023-10-06T21:28:28.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 125085 1.76 MB/sec execute 258 sec latency 24.974 ms 2023-10-06T21:28:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:28 smithi142 ceph-mon[102951]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 878 KiB/s wr, 89 op/s 2023-10-06T21:28:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:28 smithi031 ceph-mon[98539]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 878 KiB/s wr, 89 op/s 2023-10-06T21:28:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:28 smithi146 ceph-mon[102809]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 878 KiB/s wr, 89 op/s 2023-10-06T21:28:29.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 125412 1.76 MB/sec execute 259 sec latency 17.597 ms 2023-10-06T21:28:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1642127555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 msgr2=0x7fc0b0068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1642127555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0068750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1642127555 shutdown_connections 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1642127555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 0x7fc0b010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1642127555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 0x7fc0b0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1642127555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0068750 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1642127555 >> 172.21.15.31:0/1642127555 conn(0x7fc0b0076b70 msgr2=0x7fc0b0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1642127555 shutdown_connections 2023-10-06T21:28:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.738+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1642127555 wait complete. 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.739+0000 7fc0b51ff700 1 Processor -- start 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.739+0000 7fc0b51ff700 1 -- start start 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 0x7fc0b0100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 0x7fc0b01a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b01a5560 con 0x7fc0b00682d0 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b01a56a0 con 0x7fc0b0106f60 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0b51ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b01a5810 con 0x7fc0b0068c90 2023-10-06T21:28:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46230/0 (socket says 172.21.15.31:46230) 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0aed9d700 1 -- 172.21.15.31:0/1557062927 learned_addr learned my addr 172.21.15.31:0/1557062927 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 0x7fc0b0100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.740+0000 7fc0af59e700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 0x7fc0b01a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 -- 172.21.15.31:0/1557062927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 msgr2=0x7fc0b0100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 0x7fc0b0100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 -- 172.21.15.31:0/1557062927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 msgr2=0x7fc0b01a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 0x7fc0b01a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 -- 172.21.15.31:0/1557062927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40095f0 con 0x7fc0b00682d0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d950 tx=0x7fc0a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc097fff700 1 -- 172.21.15.31:0/1557062927 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000f890 con 0x7fc0b00682d0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc097fff700 1 -- 172.21.15.31:0/1557062927 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0a000fe80 con 0x7fc0b00682d0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc097fff700 1 -- 172.21.15.31:0/1557062927 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000e3b0 con 0x7fc0b00682d0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b01a5af0 con 0x7fc0b00682d0 2023-10-06T21:28:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.741+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b01a6000 con 0x7fc0b00682d0 2023-10-06T21:28:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.745+0000 7fc097fff700 1 -- 172.21.15.31:0/1557062927 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc0a0010460 con 0x7fc0b00682d0 2023-10-06T21:28:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.746+0000 7fc0c8b98b80 1 Processor -- start 2023-10-06T21:28:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.746+0000 7fc0c8b98b80 1 -- start start 2023-10-06T21:28:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856d73400 con 0x55a856e09c00 2023-10-06T21:28:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856d64bc0 con 0x55a856e0a520 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856d624e0 con 0x55a856e153d0 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc097fff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 0x7fc0980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0ae59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0ae59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:50840/0 (socket says 172.21.15.31:50840) 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0ae59c700 1 -- 172.21.15.31:0/1695760491 learned_addr learned my addr 172.21.15.31:0/1695760491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc0af59e700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 0x7fc0980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.747+0000 7fc097fff700 1 -- 172.21.15.31:0/1557062927 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc0a00ccf20 con 0x7fc0b00682d0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 -- 172.21.15.31:0/1695760491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 msgr2=0x55a856e14e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e14e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 -- 172.21.15.31:0/1695760491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 msgr2=0x55a856e09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 -- 172.21.15.31:0/1695760491 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a856b88d80 con 0x55a856e153d0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f675ff0fa155da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0af59e700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 0x7fc0980a3b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009550 tx=0x7fc0a800b040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0957fa700 1 -- 172.21.15.31:0/1695760491 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a401d070 con 0x55a856e153d0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0957fa700 1 -- 172.21.15.31:0/1695760491 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0a401e040 con 0x55a856e153d0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.748+0000 7fc0957fa700 1 -- 172.21.15.31:0/1695760491 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40047d0 con 0x55a856e153d0 2023-10-06T21:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/1695760491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 msgr2=0x55a856e177c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4005f00 comp rx=0 tx=0).stop 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/1695760491 shutdown_connections 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e153d0 0x55a856e177c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/1695760491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e14e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/1695760491 >> 172.21.15.31:0/1695760491 conn(0x55a856d7f200 msgr2=0x55a856e06080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/1695760491 shutdown_connections 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.749+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/1695760491 wait complete. 2023-10-06T21:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.750+0000 7fc0c8b98b80 1 -- start start 2023-10-06T21:28:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.750+0000 7fc0c8b98b80 1 Processor -- start 2023-10-06T21:28:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e14d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856d5ddd0 con 0x55a856e09c00 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856cfa6b0 con 0x55a856e153d0 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856cfa350 con 0x55a856e0a520 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0c8b98b80 10 client.0 _ll_register_callbacks cb 0x55a856b577a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0af59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e14d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0ae59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0af59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50574/0 (socket says 172.21.15.31:50574) 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.751+0000 7fc0af59e700 1 -- 172.21.15.31:0/3412755744 learned_addr learned my addr 172.21.15.31:0/3412755744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 -- 172.21.15.31:0/3412755744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 msgr2=0x55a856e11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 -- 172.21.15.31:0/3412755744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 msgr2=0x55a856e14d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e14d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 -- 172.21.15.31:0/3412755744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b010e5c0 con 0x55a856e153d0 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e14d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0af59e700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8003f60 tx=0x7fc0a8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc1ba0e503595eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc092ff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:28:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8022070 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a856dfdd50 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.752+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc0a801ae60 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.753+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0a800e070 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.753+0000 7fc0c8b98b80 10 client.26534 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.753+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a856b71c00 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.753+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a856dfdd50 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.753+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0a801b870 con 0x55a856e153d0 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 10 client.26534 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 10 client.26534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 20 client.26534 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e24d50 0x55a856e27170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 4 client.26534 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a856d56db0 con 0x55a856e24d50 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.754+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e24d50 0x55a856e27170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.755+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e24d50 0x55a856e27170 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162377e18c99640b server_cookie=43e9d4b5c971a939 in_seq=0 out_seq=0 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.755+0000 7fc092ff5700 10 client.26534 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.755+0000 7fc092ff5700 1 -- 172.21.15.31:0/3412755744 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a856d56db0 con 0x55a856e24d50 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.755+0000 7fc092ff5700 10 client.26534 handle_command_reply: tid=0 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.755+0000 7fc0c8b98b80 1 client.26534 shutdown 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 msgr2=0x55a856e12580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8003f60 tx=0x7fc0a8004ad0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 shutdown_connections 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e12580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e24d50 0x55a856e27170 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e14d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/3412755744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc092ff5700 0 client.26534 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 >> 172.21.15.31:0/3412755744 conn(0x55a856d7f200 msgr2=0x55a856e05c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 shutdown_connections 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/3412755744 wait complete. 2023-10-06T21:28:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.756+0000 7fc0c8b98b80 20 client.26534 trim_cache size 0 max 16384 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.772+0000 7fc0c8b98b80 1 Processor -- start 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 -- start start 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 0x55a856e1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856bde8c0 con 0x55a856e153d0 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856e02150 con 0x55a856e0a520 2023-10-06T21:28:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856e0b3f0 con 0x55a856e09c00 2023-10-06T21:28:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50580/0 (socket says 172.21.15.31:50580) 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0af59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46262/0 (socket says 172.21.15.31:46262) 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0ae59c700 1 -- 172.21.15.31:0/2687048300 learned_addr learned my addr 172.21.15.31:0/2687048300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.773+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 0x55a856e1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 -- 172.21.15.31:0/2687048300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 msgr2=0x55a856e1bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 0x55a856e1bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 -- 172.21.15.31:0/2687048300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 msgr2=0x55a856e10ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 -- 172.21.15.31:0/2687048300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a856baf1f0 con 0x55a856e153d0 2023-10-06T21:28:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 0x55a856e1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc0af59e700 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009e10 tx=0x7fc0a80087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1835c74084b1a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc095ffb700 1 -- 172.21.15.31:0/2687048300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8022070 con 0x55a856e153d0 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc095ffb700 1 -- 172.21.15.31:0/2687048300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0a800f040 con 0x55a856e153d0 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.774+0000 7fc095ffb700 1 -- 172.21.15.31:0/2687048300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a800e040 con 0x55a856e153d0 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2687048300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 msgr2=0x55a856e14890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009e10 tx=0x7fc0a80087c0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2687048300 shutdown_connections 2023-10-06T21:28:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e153d0 0x55a856e14890 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e09c00 0x55a856e1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2687048300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e0a520 0x55a856e10ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2687048300 >> 172.21.15.31:0/2687048300 conn(0x55a856d7f200 msgr2=0x55a856e10790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2687048300 shutdown_connections 2023-10-06T21:28:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.775+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2687048300 wait complete. 2023-10-06T21:28:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.776+0000 7fc0c8b98b80 1 -- start start 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.776+0000 7fc0c8b98b80 1 Processor -- start 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e17ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856e18490 con 0x55a856e09c00 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856e18600 con 0x55a856e153d0 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a856e187c0 con 0x55a856e0a520 2023-10-06T21:28:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0c8b98b80 10 client.0 _ll_register_callbacks cb 0x7fc0a80083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46278/0 (socket says 172.21.15.31:46278) 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 -- 172.21.15.31:0/2546655861 learned_addr learned my addr 172.21.15.31:0/2546655861 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0af59e700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e17ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 -- 172.21.15.31:0/2546655861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 msgr2=0x55a856e723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 -- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 msgr2=0x55a856e17ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e17ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.777+0000 7fc0aed9d700 1 -- 172.21.15.31:0/2546655861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a009c040 con 0x55a856e09c00 2023-10-06T21:28:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc0aed9d700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0099dc0 tx=0x7fc0a0065630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52ebd948a8870b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc092ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0099e30 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc0a009df80 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0a401e040 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0a00a21c0 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc0c8b98b80 10 client.16821 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a856b88d80 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.778+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0a401e040 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0a009a090 con 0x55a856e09c00 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 10 client.16821 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 20 client.16821 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e79cf0 0x55a856e7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 4 client.16821 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a856d56ad0 con 0x55a856e79cf0 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.779+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e79cf0 0x55a856e7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.780+0000 7fc0ae59c700 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e79cf0 0x55a856e7c110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16939cd01014f04 server_cookie=fd1b4b89a18f49a7 in_seq=0 out_seq=0 2023-10-06T21:28:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.780+0000 7fc092ff5700 10 client.16821 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.804+0000 7fc092ff5700 1 -- 172.21.15.31:0/2546655861 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a856d56ad0 con 0x55a856e79cf0 2023-10-06T21:28:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.804+0000 7fc092ff5700 10 client.16821 handle_command_reply: tid=0 2023-10-06T21:28:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.804+0000 7fc0c8b98b80 1 client.16821 shutdown 2023-10-06T21:28:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 msgr2=0x55a856e71ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0099dc0 tx=0x7fc0a0065630 comp rx=0 tx=0).stop 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 shutdown_connections 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a856e153d0 0x55a856e17ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a856e79cf0 0x55a856e7c110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a856e09c00 0x55a856e71ea0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 --2- 172.21.15.31:0/2546655861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a856e0a520 0x55a856e723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc092ff5700 0 client.16821 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 >> 172.21.15.31:0/2546655861 conn(0x55a856d7f200 msgr2=0x55a856e11020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 shutdown_connections 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 1 -- 172.21.15.31:0/2546655861 wait complete. 2023-10-06T21:28:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.805+0000 7fc0c8b98b80 20 client.16821 trim_cache size 0 max 16384 2023-10-06T21:28:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:28:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:28:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:28:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 msgr2=0x7fc0980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 0x7fc0980a3b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009550 tx=0x7fc0a800b040 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 msgr2=0x7fc0b0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d950 tx=0x7fc0a000dc60 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 shutdown_connections 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b0106f60 0x7fc0b01a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc0980a1670 0x7fc0980a3b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b00682d0 0x7fc0b0100220 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 --2- 172.21.15.31:0/1557062927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0b0068c90 0x7fc0b0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 >> 172.21.15.31:0/1557062927 conn(0x7fc0b0076b70 msgr2=0x7fc0b00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 shutdown_connections 2023-10-06T21:28:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:29.806+0000 7fc0b51ff700 1 -- 172.21.15.31:0/1557062927 wait complete. 2023-10-06T21:28:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:29 smithi031 ceph-mon[98539]: pgmap v234: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:28:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:29 smithi146 ceph-mon[102809]: pgmap v234: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:28:30.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 125772 1.77 MB/sec execute 260 sec latency 30.348 ms 2023-10-06T21:28:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:29 smithi142 ceph-mon[102951]: pgmap v234: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:28:31.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 126104 1.76 MB/sec execute 261 sec latency 26.233 ms 2023-10-06T21:28:32.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 126522 1.76 MB/sec execute 262 sec latency 27.812 ms 2023-10-06T21:28:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:32 smithi142 ceph-mon[102951]: pgmap v235: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 856 KiB/s wr, 83 op/s 2023-10-06T21:28:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:32 smithi031 ceph-mon[98539]: pgmap v235: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 856 KiB/s wr, 83 op/s 2023-10-06T21:28:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:32 smithi146 ceph-mon[102809]: pgmap v235: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 856 KiB/s wr, 83 op/s 2023-10-06T21:28:33.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 126794 1.76 MB/sec execute 263 sec latency 17.402 ms 2023-10-06T21:28:34.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 127425 1.77 MB/sec execute 264 sec latency 17.046 ms 2023-10-06T21:28:34.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:34 smithi142 ceph-mon[102951]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:28:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:34 smithi031 ceph-mon[98539]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:28:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:34 smithi146 ceph-mon[102809]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-10-06T21:28:35.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 127579 1.77 MB/sec execute 265 sec latency 21.185 ms 2023-10-06T21:28:36.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 127963 1.78 MB/sec execute 266 sec latency 17.991 ms 2023-10-06T21:28:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:36 smithi142 ceph-mon[102951]: pgmap v237: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:36 smithi031 ceph-mon[98539]: pgmap v237: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:36 smithi146 ceph-mon[102809]: pgmap v237: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:37.346 INFO:tasks.workunit.client.0.smithi031.stdout: 1 128123 1.77 MB/sec execute 267 sec latency 20.059 ms 2023-10-06T21:28:38.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 128306 1.76 MB/sec execute 268 sec latency 15.921 ms 2023-10-06T21:28:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:38 smithi142 ceph-mon[102951]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:38 smithi031 ceph-mon[98539]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:38 smithi146 ceph-mon[102809]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 865 KiB/s wr, 88 op/s 2023-10-06T21:28:39.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 128601 1.76 MB/sec execute 269 sec latency 100.718 ms 2023-10-06T21:28:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:39 smithi031 ceph-mon[98539]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:28:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:39 smithi146 ceph-mon[102809]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:28:40.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 128883 1.76 MB/sec execute 270 sec latency 194.373 ms 2023-10-06T21:28:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:39 smithi142 ceph-mon[102951]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2023-10-06T21:28:41.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 129238 1.76 MB/sec execute 271 sec latency 24.580 ms 2023-10-06T21:28:42.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 129574 1.76 MB/sec execute 272 sec latency 28.265 ms 2023-10-06T21:28:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:42 smithi142 ceph-mon[102951]: pgmap v240: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 942 KiB/s wr, 85 op/s 2023-10-06T21:28:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:42 smithi031 ceph-mon[98539]: pgmap v240: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 942 KiB/s wr, 85 op/s 2023-10-06T21:28:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:42 smithi146 ceph-mon[102809]: pgmap v240: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 942 KiB/s wr, 85 op/s 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.971+0000 7f438b5ff700 1 -- 172.21.15.31:0/1046089435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384105e20 msgr2=0x7f43840682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.971+0000 7f438b5ff700 1 --2- 172.21.15.31:0/1046089435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384105e20 0x7f43840682d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4380009990 tx=0x7f4380009ca0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 -- 172.21.15.31:0/1046089435 shutdown_connections 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 --2- 172.21.15.31:0/1046089435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43841038b0 0x7f4384103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 --2- 172.21.15.31:0/1046089435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384105e20 0x7f43840682d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 --2- 172.21.15.31:0/1046089435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4384068810 0x7f4384068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 -- 172.21.15.31:0/1046089435 >> 172.21.15.31:0/1046089435 conn(0x7f4384076b70 msgr2=0x7f4384076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 -- 172.21.15.31:0/1046089435 shutdown_connections 2023-10-06T21:28:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.972+0000 7f438b5ff700 1 -- 172.21.15.31:0/1046089435 wait complete. 2023-10-06T21:28:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.973+0000 7f438b5ff700 1 Processor -- start 2023-10-06T21:28:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.974+0000 7f438b5ff700 1 -- start start 2023-10-06T21:28:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384068810 0x7f438419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4384105e20 0x7f43841a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a25a0 con 0x7f43841038b0 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a2710 con 0x7f4384105e20 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438b5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a28d0 con 0x7f4384068810 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57930/0 (socket says 172.21.15.31:57930) 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 -- 172.21.15.31:0/2871140561 learned_addr learned my addr 172.21.15.31:0/2871140561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f438939b700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384068810 0x7f438419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4389b9c700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4384105e20 0x7f43841a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 -- 172.21.15.31:0/2871140561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384068810 msgr2=0x7f438419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384068810 0x7f438419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 -- 172.21.15.31:0/2871140561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4384105e20 msgr2=0x7f43841a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.975+0000 7f4388b9a700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4384105e20 0x7f43841a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f4388b9a700 1 -- 172.21.15.31:0/2871140561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800095f0 con 0x7f43841038b0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f4388b9a700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9a0 tx=0x7f437400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f437a7fc700 1 -- 172.21.15.31:0/2871140561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374009790 con 0x7f43841038b0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43841a2bb0 con 0x7f43841038b0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f437a7fc700 1 -- 172.21.15.31:0/2871140561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4374010430 con 0x7f43841038b0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f437a7fc700 1 -- 172.21.15.31:0/2871140561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437400cea0 con 0x7f43841038b0 2023-10-06T21:28:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.976+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43841aa3f0 con 0x7f43841038b0 2023-10-06T21:28:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.980+0000 7f437a7fc700 1 -- 172.21.15.31:0/2871140561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f43740105a0 con 0x7f43841038b0 2023-10-06T21:28:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.981+0000 7f439eebab80 1 Processor -- start 2023-10-06T21:28:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.981+0000 7f439eebab80 1 -- start start 2023-10-06T21:28:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.981+0000 7f439eebab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x564606184fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.981+0000 7f439eebab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f439eebab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646061903d0 0x5646061927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f439eebab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646060ee400 con 0x564606184c00 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f439eebab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646060dfbc0 con 0x564606185520 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f439eebab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646060dd4e0 con 0x5646061903d0 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f437a7fc700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 0x7f43700a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f4389b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x564606184fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40136/0 (socket says 172.21.15.31:40136) 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 -- 172.21.15.31:0/2828546896 learned_addr learned my addr 172.21.15.31:0/2828546896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f437a7fc700 1 -- 172.21.15.31:0/2871140561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f4374014070 con 0x7f43841038b0 2023-10-06T21:28:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f4388b9a700 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646061903d0 0x5646061927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f4389b9c700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 0x7f43700a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 -- 172.21.15.31:0/2828546896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646061903d0 msgr2=0x5646061927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646061903d0 0x5646061927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 -- 172.21.15.31:0/2828546896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 msgr2=0x564606184fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x564606184fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.982+0000 7f438939b700 1 -- 172.21.15.31:0/2828546896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564605f03d80 con 0x564606185520 2023-10-06T21:28:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.983+0000 7f438939b700 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4380015000 tx=0x7f4380005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5350e63c7b6c018 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.983+0000 7f4389b9c700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 0x7f43700a3ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f437c009fd0 tx=0x7f437c004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.983+0000 7f436f7fe700 1 -- 172.21.15.31:0/2828546896 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d070 con 0x564606185520 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.983+0000 7f436f7fe700 1 -- 172.21.15.31:0/2828546896 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f438001e040 con 0x564606185520 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.983+0000 7f436f7fe700 1 -- 172.21.15.31:0/2828546896 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380004880 con 0x564606185520 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 -- 172.21.15.31:0/2828546896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 msgr2=0x56460618fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4380015000 tx=0x7f4380005f00 comp rx=0 tx=0).stop 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 -- 172.21.15.31:0/2828546896 shutdown_connections 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646061903d0 0x5646061927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x564606184fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 --2- 172.21.15.31:0/2828546896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606185520 0x56460618fe90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 -- 172.21.15.31:0/2828546896 >> 172.21.15.31:0/2828546896 conn(0x5646060fa200 msgr2=0x564606181080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 -- 172.21.15.31:0/2828546896 shutdown_connections 2023-10-06T21:28:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.984+0000 7f439eebab80 1 -- 172.21.15.31:0/2828546896 wait complete. 2023-10-06T21:28:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.985+0000 7f439eebab80 1 -- start start 2023-10-06T21:28:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.985+0000 7f439eebab80 1 Processor -- start 2023-10-06T21:28:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x56460618fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x56460618d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646060d8dd0 con 0x5646061903d0 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646060756b0 con 0x564606184c00 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564606075350 con 0x564606185520 2023-10-06T21:28:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f439eebab80 10 client.0 _ll_register_callbacks cb 0x564605ed27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f4388b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f4388b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39110/0 (socket says 172.21.15.31:39110) 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.986+0000 7f4388b9a700 1 -- 172.21.15.31:0/3020658854 learned_addr learned my addr 172.21.15.31:0/3020658854 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f438939b700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x56460618fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4389b9c700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x56460618d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 -- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 msgr2=0x56460618fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x56460618fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 -- 172.21.15.31:0/3020658854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 msgr2=0x56460618d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x56460618d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 -- 172.21.15.31:0/3020658854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4384114dd0 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f438939b700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x56460618fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43740cb010 tx=0x7f4374099ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2359be65a45fa166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f436cff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740a2630 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564606178d50 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.987+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f437409a2a0 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43740999a0 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 10 client.26509 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564605eecc00 con 0x564606185520 2023-10-06T21:28:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43740b8020 con 0x564606185520 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4374096090 con 0x564606185520 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 10 client.26509 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 20 client.26509 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56460619fe20 0x5646061a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 4 client.26509 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.988+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646060d1db0 con 0x56460619fe20 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.989+0000 7f438939b700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56460619fe20 0x5646061a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.989+0000 7f438939b700 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56460619fe20 0x5646061a2240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ee5ff1da01e463 server_cookie=e63042ce3c2677cc in_seq=0 out_seq=0 2023-10-06T21:28:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.989+0000 7f436cff9700 10 client.26509 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f436cff9700 1 -- 172.21.15.31:0/3020658854 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5646060d1db0 con 0x56460619fe20 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f436cff9700 10 client.26509 handle_command_reply: tid=0 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 client.26509 shutdown 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 msgr2=0x56460618d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43740cb010 tx=0x7f4374099ef0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 shutdown_connections 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56460619fe20 0x5646061a2240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x56460618d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x56460618fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 --2- 172.21.15.31:0/3020658854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618d0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 >> 172.21.15.31:0/3020658854 conn(0x5646060fa200 msgr2=0x564606180c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.990+0000 7f436cff9700 0 client.26509 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.991+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 shutdown_connections 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.991+0000 7f439eebab80 1 -- 172.21.15.31:0/3020658854 wait complete. 2023-10-06T21:28:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:42.991+0000 7f439eebab80 20 client.26509 trim_cache size 0 max 16384 2023-10-06T21:28:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.009+0000 7f439eebab80 1 Processor -- start 2023-10-06T21:28:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.009+0000 7f439eebab80 1 -- start start 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x564606188470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564605f598c0 con 0x5646061903d0 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460618e560 con 0x564606184c00 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f439eebab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460617d860 con 0x564606185520 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f438939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f438939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40156/0 (socket says 172.21.15.31:40156) 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f438939b700 1 -- 172.21.15.31:0/2805239182 learned_addr learned my addr 172.21.15.31:0/2805239182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.010+0000 7f4389b9c700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x564606188470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f4388b9a700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 -- 172.21.15.31:0/2805239182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 msgr2=0x56460618f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 -- 172.21.15.31:0/2805239182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 msgr2=0x564606188470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x564606188470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 -- 172.21.15.31:0/2805239182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564605f2a1f0 con 0x564606184c00 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f4389b9c700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x564606188470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f438939b700 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43800098e0 tx=0x7f438001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd8d925ebcdfbc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f436ffff700 1 -- 172.21.15.31:0/2805239182 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d040 con 0x564606184c00 2023-10-06T21:28:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f436ffff700 1 -- 172.21.15.31:0/2805239182 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43800056f0 con 0x564606184c00 2023-10-06T21:28:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.011+0000 7f436ffff700 1 -- 172.21.15.31:0/2805239182 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380034e50 con 0x564606184c00 2023-10-06T21:28:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 -- 172.21.15.31:0/2805239182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 msgr2=0x564606196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43800098e0 tx=0x7f438001a040 comp rx=0 tx=0).stop 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 -- 172.21.15.31:0/2805239182 shutdown_connections 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5646061903d0 0x564606188470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564606184c00 0x564606196eb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 --2- 172.21.15.31:0/2805239182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 -- 172.21.15.31:0/2805239182 >> 172.21.15.31:0/2805239182 conn(0x5646060fa200 msgr2=0x56460617f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 -- 172.21.15.31:0/2805239182 shutdown_connections 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.012+0000 7f439eebab80 1 -- 172.21.15.31:0/2805239182 wait complete. 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.013+0000 7f439eebab80 1 -- start start 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.013+0000 7f439eebab80 1 Processor -- start 2023-10-06T21:28:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 0x564606189390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460618af60 con 0x564606184c00 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460618b0d0 con 0x5646061903d0 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646061898d0 con 0x564606185520 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f439eebab80 10 client.0 _ll_register_callbacks cb 0x7f4374099c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57966/0 (socket says 172.21.15.31:57966) 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 -- 172.21.15.31:0/3088147659 learned_addr learned my addr 172.21.15.31:0/3088147659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f4389b9c700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 0x564606189390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 -- 172.21.15.31:0/3088147659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 msgr2=0x56460618a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 -- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 msgr2=0x564606189390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 0x564606189390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f438939b700 1 -- 172.21.15.31:0/3088147659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564605f03d80 con 0x564606184c00 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.014+0000 7f4389b9c700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 0x564606189390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f438939b700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4380006010 tx=0x7f4380009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e04c4855df2e899 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f436cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d040 con 0x564606184c00 2023-10-06T21:28:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f4380031040 con 0x564606184c00 2023-10-06T21:28:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f438001e040 con 0x564606184c00 2023-10-06T21:28:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4380034560 con 0x564606184c00 2023-10-06T21:28:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f439eebab80 10 client.16839 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.015+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564605f02250 con 0x564606184c00 2023-10-06T21:28:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4380049080 con 0x564606184c00 2023-10-06T21:28:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4380004950 con 0x564606184c00 2023-10-06T21:28:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 10 client.16839 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 20 client.16839 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5646061f42d0 0x5646061f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 4 client.16839 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646060d1ad0 con 0x5646061f42d0 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.016+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5646061f42d0 0x5646061f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.017+0000 7f4388b9a700 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5646061f42d0 0x5646061f66c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e648f57fe6d4154 server_cookie=4eedc13b7a9ca542 in_seq=0 out_seq=0 2023-10-06T21:28:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.017+0000 7f436cff9700 10 client.16839 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.040+0000 7f436cff9700 1 -- 172.21.15.31:0/3088147659 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646060d1ad0 con 0x5646061f42d0 2023-10-06T21:28:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.040+0000 7f436cff9700 10 client.16839 handle_command_reply: tid=0 2023-10-06T21:28:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.040+0000 7f439eebab80 1 client.16839 shutdown 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 msgr2=0x56460618a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4380006010 tx=0x7f4380009290 comp rx=0 tx=0).stop 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 shutdown_connections 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646061903d0 0x564606189390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5646061f42d0 0x5646061f66c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564606184c00 0x56460618a410 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 --2- 172.21.15.31:0/3088147659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564606185520 0x56460618a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 >> 172.21.15.31:0/3088147659 conn(0x5646060fa200 msgr2=0x564606188470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f436cff9700 0 client.16839 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 shutdown_connections 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 1 -- 172.21.15.31:0/3088147659 wait complete. 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.041+0000 7f439eebab80 20 client.16839 trim_cache size 0 max 16384 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:28:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:28:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 msgr2=0x7f43700a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 0x7f43700a3ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f437c009fd0 tx=0x7f437c004b40 comp rx=0 tx=0).stop 2023-10-06T21:28:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 msgr2=0x7f438419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9a0 tx=0x7f437400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 shutdown_connections 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4384105e20 0x7f43841a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384068810 0x7f438419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f43700a15f0 0x7f43700a3ab0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 --2- 172.21.15.31:0/2871140561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43841038b0 0x7f438419d8f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 >> 172.21.15.31:0/2871140561 conn(0x7f4384076b70 msgr2=0x7f43840feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 shutdown_connections 2023-10-06T21:28:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:43.042+0000 7f438b5ff700 1 -- 172.21.15.31:0/2871140561 wait complete. 2023-10-06T21:28:43.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 129941 1.75 MB/sec execute 273 sec latency 27.515 ms 2023-10-06T21:28:44.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 130287 1.76 MB/sec execute 274 sec latency 16.580 ms 2023-10-06T21:28:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:44 smithi142 ceph-mon[102951]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:28:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:44 smithi031 ceph-mon[98539]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:28:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:44 smithi146 ceph-mon[102809]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2023-10-06T21:28:45.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 130855 1.77 MB/sec execute 275 sec latency 18.949 ms 2023-10-06T21:28:46.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 131091 1.77 MB/sec execute 276 sec latency 18.126 ms 2023-10-06T21:28:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:46 smithi142 ceph-mon[102951]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:46 smithi031 ceph-mon[98539]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:46 smithi146 ceph-mon[102809]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:47.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 131502 1.78 MB/sec execute 277 sec latency 17.677 ms 2023-10-06T21:28:48.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 131674 1.77 MB/sec execute 278 sec latency 19.926 ms 2023-10-06T21:28:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:48 smithi142 ceph-mon[102951]: pgmap v243: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:48 smithi031 ceph-mon[98539]: pgmap v243: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:48 smithi146 ceph-mon[102809]: pgmap v243: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 813 KiB/s wr, 82 op/s 2023-10-06T21:28:49.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 131870 1.76 MB/sec execute 279 sec latency 19.589 ms 2023-10-06T21:28:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:49 smithi031 ceph-mon[98539]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:28:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:49 smithi146 ceph-mon[102809]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:28:50.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 132214 1.76 MB/sec execute 280 sec latency 23.170 ms 2023-10-06T21:28:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:49 smithi142 ceph-mon[102951]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2023-10-06T21:28:51.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 132546 1.76 MB/sec execute 281 sec latency 19.152 ms 2023-10-06T21:28:52.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 132901 1.76 MB/sec execute 282 sec latency 27.146 ms 2023-10-06T21:28:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:52 smithi142 ceph-mon[102951]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 951 KiB/s wr, 88 op/s 2023-10-06T21:28:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:52 smithi031 ceph-mon[98539]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 951 KiB/s wr, 88 op/s 2023-10-06T21:28:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:52 smithi146 ceph-mon[102809]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 951 KiB/s wr, 88 op/s 2023-10-06T21:28:53.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 133280 1.76 MB/sec execute 283 sec latency 25.935 ms 2023-10-06T21:28:54.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 133656 1.76 MB/sec execute 284 sec latency 26.917 ms 2023-10-06T21:28:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:54 smithi142 ceph-mon[102951]: pgmap v246: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:28:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:54 smithi031 ceph-mon[98539]: pgmap v246: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:28:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:54 smithi146 ceph-mon[102809]: pgmap v246: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:28:55.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 134048 1.77 MB/sec execute 285 sec latency 18.601 ms 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 -- 172.21.15.31:0/3942210819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80103320 msgr2=0x7fef80103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 --2- 172.21.15.31:0/3942210819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80103320 0x7fef80103700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009990 tx=0x7fef7c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 -- 172.21.15.31:0/3942210819 shutdown_connections 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 --2- 172.21.15.31:0/3942210819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80108440 0x7fef8010ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 --2- 172.21.15.31:0/3942210819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80103320 0x7fef80103700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 --2- 172.21.15.31:0/3942210819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef80103cd0 0x7fef80107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 -- 172.21.15.31:0/3942210819 >> 172.21.15.31:0/3942210819 conn(0x7fef800feb60 msgr2=0x7fef80100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.211+0000 7fef87bff700 1 -- 172.21.15.31:0/3942210819 shutdown_connections 2023-10-06T21:28:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.212+0000 7fef87bff700 1 -- 172.21.15.31:0/3942210819 wait complete. 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.212+0000 7fef87bff700 1 Processor -- start 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 -- start start 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 0x7fef801996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef8019dd30 0x7fef8019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8019e7b0 con 0x7fef80103320 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8019e920 con 0x7fef80108440 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef87bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8019eae0 con 0x7fef8019dd30 2023-10-06T21:28:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef8599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef8599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60972/0 (socket says 172.21.15.31:60972) 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef8519a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 0x7fef801996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.213+0000 7fef8599b700 1 -- 172.21.15.31:0/1300568510 learned_addr learned my addr 172.21.15.31:0/1300568510 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8619c700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef8019dd30 0x7fef8019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 -- 172.21.15.31:0/1300568510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef8019dd30 msgr2=0x7fef8019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef8019dd30 0x7fef8019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 -- 172.21.15.31:0/1300568510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 msgr2=0x7fef801996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 0x7fef801996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 -- 172.21.15.31:0/1300568510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0095f0 con 0x7fef80103320 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8519a700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 0x7fef801996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef8599b700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000c00 tx=0x7fef7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef76ffd700 1 -- 172.21.15.31:0/1300568510 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01d070 con 0x7fef80103320 2023-10-06T21:28:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef76ffd700 1 -- 172.21.15.31:0/1300568510 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef7c022470 con 0x7fef80103320 2023-10-06T21:28:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef801a5fe0 con 0x7fef80103320 2023-10-06T21:28:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.214+0000 7fef76ffd700 1 -- 172.21.15.31:0/1300568510 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c017650 con 0x7fef80103320 2023-10-06T21:28:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.215+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef801a6410 con 0x7fef80103320 2023-10-06T21:28:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.218+0000 7fef76ffd700 1 -- 172.21.15.31:0/1300568510 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fef7c0225e0 con 0x7fef80103320 2023-10-06T21:28:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.219+0000 7fef9b5f0b80 1 Processor -- start 2023-10-06T21:28:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.219+0000 7fef9b5f0b80 1 -- start start 2023-10-06T21:28:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28ab97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28a15400 con 0x55da28ab73d0 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28a06bc0 con 0x55da28aac520 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28a044e0 con 0x55da28aabc00 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef8599b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef8599b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45548/0 (socket says 172.21.15.31:45548) 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef8599b700 1 -- 172.21.15.31:0/2955892758 learned_addr learned my addr 172.21.15.31:0/2955892758 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef8519a700 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28ab97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef8619c700 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.220+0000 7fef76ffd700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 0x7fef6c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 -- 172.21.15.31:0/2955892758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 msgr2=0x55da28aabfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef76ffd700 1 -- 172.21.15.31:0/1300568510 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fef7c0ce480 con 0x7fef80103320 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aabfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 -- 172.21.15.31:0/2955892758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 msgr2=0x55da28ab6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 -- 172.21.15.31:0/2955892758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da2882ad80 con 0x55da28ab73d0 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8619c700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 0x7fef6c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef8519a700 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28ab97c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fef80068f90 tx=0x7fef700117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf75f7d1b5ecea49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.221+0000 7fef6bfff700 1 -- 172.21.15.31:0/2955892758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70004230 con 0x55da28ab73d0 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef6bfff700 1 -- 172.21.15.31:0/2955892758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef70004cc0 con 0x55da28ab73d0 2023-10-06T21:28:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef6bfff700 1 -- 172.21.15.31:0/2955892758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70005520 con 0x55da28ab73d0 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2955892758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 msgr2=0x55da28ab97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef8619c700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 0x7fef6c0a3b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef780075c0 tx=0x7fef78009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28ab97c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fef80068f90 tx=0x7fef700117a0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2955892758 shutdown_connections 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28ab97c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2955892758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.222+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2955892758 >> 172.21.15.31:0/2955892758 conn(0x55da28a21200 msgr2=0x55da28aa8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.223+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2955892758 shutdown_connections 2023-10-06T21:28:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.223+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2955892758 wait complete. 2023-10-06T21:28:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.223+0000 7fef9b5f0b80 1 -- start start 2023-10-06T21:28:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.223+0000 7fef9b5f0b80 1 Processor -- start 2023-10-06T21:28:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aac520 0x55da28ab4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28ab73d0 0x55da28ab4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da289ffdd0 con 0x55da28aabc00 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2899c6b0 con 0x55da28ab73d0 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2899c350 con 0x55da28aac520 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef9b5f0b80 10 client.0 _ll_register_callbacks cb 0x55da287f97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef8599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef8599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60996/0 (socket says 172.21.15.31:60996) 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef8599b700 1 -- 172.21.15.31:0/2682042689 learned_addr learned my addr 172.21.15.31:0/2682042689 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.224+0000 7fef8619c700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28ab73d0 0x55da28ab4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8519a700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aac520 0x55da28ab4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 -- 172.21.15.31:0/2682042689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aac520 msgr2=0x55da28ab4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aac520 0x55da28ab4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 -- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28ab73d0 msgr2=0x55da28ab4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28ab73d0 0x55da28ab4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 -- 172.21.15.31:0/2682042689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef70009c90 con 0x55da28aabc00 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef8599b700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0ce6e0 tx=0x7fef7c0cbaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8df20983e48ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef697fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c09b880 con 0x55da28aabc00 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da28a9fd50 con 0x55da28aabc00 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.225+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fef7c0228c0 con 0x55da28aabc00 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7c09af60 con 0x55da28aabc00 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 10 client.16851 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55da28813c00 con 0x55da28aabc00 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55da28a9fd50 con 0x55da28aabc00 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef7c09f100 con 0x55da28aabc00 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 10 client.16851 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 20 client.16851 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28ac6b10 0x55da28ac8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 4 client.16851 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.226+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da289f8db0 con 0x55da28ac6b10 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.227+0000 7fef8519a700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28ac6b10 0x55da28ac8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.227+0000 7fef8519a700 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28ac6b10 0x55da28ac8f30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88281376dd8d9af5 server_cookie=53f8328cc79e207d in_seq=0 out_seq=0 2023-10-06T21:28:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.227+0000 7fef697fa700 10 client.16851 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef697fa700 1 -- 172.21.15.31:0/2682042689 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da289f8db0 con 0x55da28ac6b10 2023-10-06T21:28:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef697fa700 10 client.16851 handle_command_reply: tid=0 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 client.16851 shutdown 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 msgr2=0x55da28ab6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0ce6e0 tx=0x7fef7c0cbaa0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 shutdown_connections 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28ab73d0 0x55da28ab4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28ac6b10 0x55da28ac8f30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aabc00 0x55da28ab6e50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/2682042689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aac520 0x55da28ab4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef697fa700 0 client.16851 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 >> 172.21.15.31:0/2682042689 conn(0x55da28a21200 msgr2=0x55da28aa7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 shutdown_connections 2023-10-06T21:28:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.228+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/2682042689 wait complete. 2023-10-06T21:28:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.229+0000 7fef9b5f0b80 20 client.16851 trim_cache size 0 max 16384 2023-10-06T21:28:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.252+0000 7fef9b5f0b80 1 Processor -- start 2023-10-06T21:28:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.252+0000 7fef9b5f0b80 1 -- start start 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aaeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28aaf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da288808c0 con 0x55da28ab73d0 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28aad6f0 con 0x55da28aac520 2023-10-06T21:28:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28aaff00 con 0x55da28aabc00 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef8519a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef8519a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45564/0 (socket says 172.21.15.31:45564) 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef8519a700 1 -- 172.21.15.31:0/3975034670 learned_addr learned my addr 172.21.15.31:0/3975034670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef8619c700 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28aaf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.253+0000 7fef8599b700 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aaeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 -- 172.21.15.31:0/3975034670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 msgr2=0x55da28aaeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aaeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 -- 172.21.15.31:0/3975034670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 msgr2=0x55da28aaf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28aaf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 -- 172.21.15.31:0/3975034670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da288511f0 con 0x55da28aac520 2023-10-06T21:28:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef8519a700 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef70011af0 tx=0x7fef70009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85032c7212ab82ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef74bff700 1 -- 172.21.15.31:0/3975034670 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7002d030 con 0x55da28aac520 2023-10-06T21:28:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef74bff700 1 -- 172.21.15.31:0/3975034670 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef70004e30 con 0x55da28aac520 2023-10-06T21:28:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.254+0000 7fef74bff700 1 -- 172.21.15.31:0/3975034670 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70029070 con 0x55da28aac520 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/3975034670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 msgr2=0x55da28ab6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef70011af0 tx=0x7fef70009fb0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/3975034670 shutdown_connections 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28ab73d0 0x55da28aaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28aabc00 0x55da28aaeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/3975034670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aac520 0x55da28ab6980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/3975034670 >> 172.21.15.31:0/3975034670 conn(0x55da28a21200 msgr2=0x55da28aa67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/3975034670 shutdown_connections 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.255+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/3975034670 wait complete. 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 -- start start 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 Processor -- start 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aabc00 0x55da28abdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 0x55da28aba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28abaa30 con 0x55da28aac520 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28ababa0 con 0x55da28aabc00 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da28abad60 con 0x55da28ab73d0 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.256+0000 7fef9b5f0b80 10 client.0 _ll_register_callbacks cb 0x55da287f97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 0x55da28aba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32768/0 (socket says 172.21.15.31:32768) 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8519a700 1 -- 172.21.15.31:0/1013810771 learned_addr learned my addr 172.21.15.31:0/1013810771 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8599b700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aabc00 0x55da28abdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 -- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aabc00 msgr2=0x55da28abdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aabc00 0x55da28abdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 -- 172.21.15.31:0/1013810771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 msgr2=0x55da28ab9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8619c700 1 -- 172.21.15.31:0/1013810771 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7001d040 con 0x55da28ab73d0 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.257+0000 7fef8519a700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef8619c700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 0x55da28aba4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef78009c10 tx=0x7fef78004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f607bbc0935395a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef697fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800e040 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef70004cc0 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fef7801c070 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef78017b70 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef9b5f0b80 10 client.26533 fetch_fsmap learned FSMap version 34 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55da28829250 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.258+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef70004cc0 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef78026cb0 con 0x55da28ab73d0 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 10 client.26533 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 20 client.26533 populate_metadata read hostname 'smithi031' 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28b1be70 0x55da28b1e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 4 client.26533 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da289f8ad0 con 0x55da28b1be70 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.259+0000 7fef8599b700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28b1be70 0x55da28b1e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.260+0000 7fef697fa700 10 client.26533 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.260+0000 7fef8599b700 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28b1be70 0x55da28b1e290 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c630f074578edc server_cookie=3306935d9d50292a in_seq=0 out_seq=0 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.281+0000 7fef697fa700 1 -- 172.21.15.31:0/1013810771 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da289f8ad0 con 0x55da28b1be70 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.281+0000 7fef697fa700 10 client.26533 handle_command_reply: tid=0 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.281+0000 7fef9b5f0b80 1 client.26533 shutdown 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 msgr2=0x55da28aba4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 0x55da28aba4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef78009c10 tx=0x7fef78004ad0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 shutdown_connections 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55da28ab73d0 0x55da28aba4f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da28aabc00 0x55da28abdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55da28b1be70 0x55da28b1e290 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 --2- 172.21.15.31:0/1013810771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55da28aac520 0x55da28ab9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 >> 172.21.15.31:0/1013810771 conn(0x55da28a21200 msgr2=0x55da28aaf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef697fa700 0 client.26533 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 shutdown_connections 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 1 -- 172.21.15.31:0/1013810771 wait complete. 2023-10-06T21:28:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.282+0000 7fef9b5f0b80 20 client.26533 trim_cache size 0 max 16384 2023-10-06T21:28:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:28:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:28:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:28:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:28:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 msgr2=0x7fef6c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 0x7fef6c0a3b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef780075c0 tx=0x7fef78009040 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 msgr2=0x7fef80199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000c00 tx=0x7fef7c004730 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 shutdown_connections 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef8019dd30 0x7fef8019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fef6c0a1670 0x7fef6c0a3b30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef80103320 0x7fef80199180 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 --2- 172.21.15.31:0/1300568510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef80108440 0x7fef801996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 >> 172.21.15.31:0/1300568510 conn(0x7fef800feb60 msgr2=0x7fef800ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 shutdown_connections 2023-10-06T21:28:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:28:56.283+0000 7fef87bff700 1 -- 172.21.15.31:0/1300568510 wait complete. 2023-10-06T21:28:56.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 134567 1.77 MB/sec execute 286 sec latency 24.826 ms 2023-10-06T21:28:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:56 smithi142 ceph-mon[102951]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:56 smithi031 ceph-mon[98539]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:56 smithi146 ceph-mon[102809]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:57.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 134841 1.78 MB/sec execute 287 sec latency 16.262 ms 2023-10-06T21:28:58.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 135114 1.77 MB/sec execute 288 sec latency 17.689 ms 2023-10-06T21:28:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:58 smithi142 ceph-mon[102951]: pgmap v248: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:58 smithi031 ceph-mon[98539]: pgmap v248: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:58 smithi146 ceph-mon[102809]: pgmap v248: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1002 KiB/s wr, 92 op/s 2023-10-06T21:28:59.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 135289 1.77 MB/sec execute 289 sec latency 16.374 ms 2023-10-06T21:29:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:28:59 smithi031 ceph-mon[98539]: pgmap v249: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2023-10-06T21:29:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:28:59 smithi146 ceph-mon[102809]: pgmap v249: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2023-10-06T21:29:00.347 INFO:tasks.workunit.client.0.smithi031.stdout: 1 135542 1.76 MB/sec execute 290 sec latency 24.916 ms 2023-10-06T21:29:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:28:59 smithi142 ceph-mon[102951]: pgmap v249: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2023-10-06T21:29:01.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 135914 1.76 MB/sec execute 291 sec latency 16.668 ms 2023-10-06T21:29:02.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 136284 1.77 MB/sec execute 292 sec latency 25.617 ms 2023-10-06T21:29:02.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:02 smithi142 ceph-mon[102951]: pgmap v250: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 980 KiB/s wr, 90 op/s 2023-10-06T21:29:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:02 smithi031 ceph-mon[98539]: pgmap v250: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 980 KiB/s wr, 90 op/s 2023-10-06T21:29:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:02 smithi146 ceph-mon[102809]: pgmap v250: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 980 KiB/s wr, 90 op/s 2023-10-06T21:29:03.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 136599 1.76 MB/sec execute 293 sec latency 27.102 ms 2023-10-06T21:29:04.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 136970 1.76 MB/sec execute 294 sec latency 26.188 ms 2023-10-06T21:29:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:04 smithi142 ceph-mon[102951]: pgmap v251: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2023-10-06T21:29:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:04 smithi031 ceph-mon[98539]: pgmap v251: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2023-10-06T21:29:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:04 smithi146 ceph-mon[102809]: pgmap v251: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2023-10-06T21:29:05.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 137292 1.76 MB/sec execute 295 sec latency 18.782 ms 2023-10-06T21:29:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:05 smithi142 ceph-mon[102951]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:29:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:05 smithi031 ceph-mon[98539]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:29:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:05 smithi146 ceph-mon[102809]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:29:06.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 137791 1.77 MB/sec execute 296 sec latency 20.313 ms 2023-10-06T21:29:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:06 smithi142 ceph-mon[102951]: pgmap v253: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:06 smithi031 ceph-mon[98539]: pgmap v253: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:06 smithi146 ceph-mon[102809]: pgmap v253: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:07.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 138142 1.77 MB/sec execute 297 sec latency 19.579 ms 2023-10-06T21:29:08.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 138495 1.78 MB/sec execute 298 sec latency 19.234 ms 2023-10-06T21:29:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:08 smithi142 ceph-mon[102951]: pgmap v254: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:08 smithi031 ceph-mon[98539]: pgmap v254: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:08 smithi146 ceph-mon[102809]: pgmap v254: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-10-06T21:29:09.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 138709 1.77 MB/sec execute 299 sec latency 17.610 ms 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/241993380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f8101a80 msgr2=0x7fb0f8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/241993380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f8101a80 0x7fb0f8105ac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0098f0 tx=0x7fb0ec009c00 comp rx=0 tx=0).stop 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/241993380 shutdown_connections 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/241993380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/241993380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/241993380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f8101a80 0x7fb0f8105ac0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.452+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/241993380 >> 172.21.15.31:0/241993380 conn(0x7fb0f80fc930 msgr2=0x7fb0f80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.453+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/241993380 shutdown_connections 2023-10-06T21:29:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.453+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/241993380 wait complete. 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 Processor -- start 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 -- start start 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 0x7fb0f81a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81a5af0 con 0x7fb0f81061f0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81a5c30 con 0x7fb0f81010d0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.454+0000 7fb0fd5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81a5da0 con 0x7fb0f81a31c0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 0x7fb0f81a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42942/0 (socket says 172.21.15.31:42942) 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/136402180 learned_addr learned my addr 172.21.15.31:0/136402180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/136402180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 msgr2=0x7fb0f81a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 0x7fb0f81a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 msgr2=0x7fb0f80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/136402180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0009c40 con 0x7fb0f81010d0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 0x7fb0f81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.455+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9c0 tx=0x7fb0e800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.456+0000 7fb0dffff700 1 -- 172.21.15.31:0/136402180 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004a30 con 0x7fb0f81010d0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.456+0000 7fb0dffff700 1 -- 172.21.15.31:0/136402180 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0e800cc50 con 0x7fb0f81010d0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.456+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec009520 con 0x7fb0f81010d0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.456+0000 7fb0dffff700 1 -- 172.21.15.31:0/136402180 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8005440 con 0x7fb0f81010d0 2023-10-06T21:29:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.456+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81a6420 con 0x7fb0f81010d0 2023-10-06T21:29:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.459+0000 7fb0dffff700 1 -- 172.21.15.31:0/136402180 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb0e80055a0 con 0x7fb0f81010d0 2023-10-06T21:29:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.461+0000 7fb110fa4b80 1 Processor -- start 2023-10-06T21:29:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 -- start start 2023-10-06T21:29:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb0dffff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 0x7fb0e00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 0x55de92d18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c82400 con 0x55de92d18c00 2023-10-06T21:29:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c73bc0 con 0x55de92d19520 2023-10-06T21:29:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb0dffff700 1 -- 172.21.15.31:0/136402180 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fb0e8014070 con 0x7fb0f81010d0 2023-10-06T21:29:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.462+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c714e0 con 0x55de92d243d0 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 0x7fb0e00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 0x55de92d18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42944/0 (socket says 172.21.15.31:42944) 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f67fc700 1 -- 172.21.15.31:0/4029226247 learned_addr learned my addr 172.21.15.31:0/4029226247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/4029226247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 msgr2=0x55de92d267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/4029226247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 msgr2=0x55de92d23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/4029226247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de92a97d80 con 0x55de92d18c00 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 0x55de92d18fe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ca8b0 tx=0x7fb0e8099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ee268f2c93ff08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 0x7fb0e00a3b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009fd0 tx=0x7fb0f0009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.463+0000 7fb0dd7fa700 1 -- 172.21.15.31:0/4029226247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e809a140 con 0x55de92d18c00 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.464+0000 7fb0dd7fa700 1 -- 172.21.15.31:0/4029226247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0e809e290 con 0x55de92d18c00 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.464+0000 7fb0dd7fa700 1 -- 172.21.15.31:0/4029226247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8096850 con 0x55de92d18c00 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.464+0000 7fb110fa4b80 1 -- 172.21.15.31:0/4029226247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 msgr2=0x55de92d18fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.464+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 0x55de92d18fe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ca8b0 tx=0x7fb0e8099d50 comp rx=0 tx=0).stop 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 -- 172.21.15.31:0/4029226247 shutdown_connections 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d18c00 0x55de92d18fe0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/4029226247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d19520 0x55de92d23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 -- 172.21.15.31:0/4029226247 >> 172.21.15.31:0/4029226247 conn(0x55de92c8e200 msgr2=0x55de92d15080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 -- 172.21.15.31:0/4029226247 shutdown_connections 2023-10-06T21:29:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 -- 172.21.15.31:0/4029226247 wait complete. 2023-10-06T21:29:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.465+0000 7fb110fa4b80 1 -- start start 2023-10-06T21:29:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 Processor -- start 2023-10-06T21:29:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c6cdd0 con 0x55de92d19520 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c096b0 con 0x55de92d18c00 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92c09350 con 0x55de92d243d0 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb110fa4b80 10 client.0 _ll_register_callbacks cb 0x55de92a667a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42948/0 (socket says 172.21.15.31:42948) 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.466+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/1012397625 learned_addr learned my addr 172.21.15.31:0/1012397625 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/1012397625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 msgr2=0x55de92d21ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d21ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/1012397625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 msgr2=0x55de92d21660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d21660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 -- 172.21.15.31:0/1012397625 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f810e5e0 con 0x55de92d18c00 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80caeb0 tx=0x7fb0e8004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed481f75fef63fa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0daff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e809a140 con 0x55de92d18c00 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de92d0cd50 con 0x55de92d18c00 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.467+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fb0e80a24d0 con 0x55de92d18c00 2023-10-06T21:29:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e809a5b0 con 0x55de92d18c00 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 10 client.26588 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fb0f81a6420 con 0x55de92d18c00 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de92d0cd50 con 0x55de92d18c00 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0e80586f0 con 0x55de92d18c00 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 10 client.26588 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 20 client.26588 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d33a90 0x55de92d35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 4 client.26588 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de92c65db0 con 0x55de92d33a90 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.468+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d33a90 0x55de92d35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.469+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d33a90 0x55de92d35e80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f684f4dd5250b server_cookie=9ab20fead5a04527 in_seq=0 out_seq=0 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.469+0000 7fb0daff5700 10 client.26588 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb0daff5700 1 -- 172.21.15.31:0/1012397625 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de92c65db0 con 0x55de92d33a90 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb0daff5700 10 client.26588 handle_command_reply: tid=0 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 client.26588 shutdown 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 msgr2=0x55de92d21120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80caeb0 tx=0x7fb0e8004dc0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 shutdown_connections 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d21ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d21120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d33a90 0x55de92d35e80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/1012397625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d21660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb0daff5700 0 client.26588 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 >> 172.21.15.31:0/1012397625 conn(0x55de92c8e200 msgr2=0x55de92d14c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 shutdown_connections 2023-10-06T21:29:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 1 -- 172.21.15.31:0/1012397625 wait complete. 2023-10-06T21:29:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.470+0000 7fb110fa4b80 20 client.26588 trim_cache size 0 max 16384 2023-10-06T21:29:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 Processor -- start 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 -- start start 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d1bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92aed8c0 con 0x55de92d19520 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92d1f680 con 0x55de92d18c00 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.489+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92d1cf70 con 0x55de92d243d0 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d1bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54748/0 (socket says 172.21.15.31:54748) 2023-10-06T21:29:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 -- 172.21.15.31:0/153670079 learned_addr learned my addr 172.21.15.31:0/153670079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 -- 172.21.15.31:0/153670079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 msgr2=0x55de92d14bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d14bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 -- 172.21.15.31:0/153670079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 msgr2=0x55de92d1bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d1bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 -- 172.21.15.31:0/153670079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de92a80c00 con 0x55de92d19520 2023-10-06T21:29:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d14bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009810 tx=0x7fb0ec005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b8ed0dbcc68049 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.490+0000 7fb0ddffb700 1 -- 172.21.15.31:0/153670079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec027a80 con 0x55de92d19520 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb0ddffb700 1 -- 172.21.15.31:0/153670079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0ec01d040 con 0x55de92d19520 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb0ddffb700 1 -- 172.21.15.31:0/153670079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec027580 con 0x55de92d19520 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 -- 172.21.15.31:0/153670079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 msgr2=0x55de92d208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009810 tx=0x7fb0ec005dc0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 -- 172.21.15.31:0/153670079 shutdown_connections 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d14bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/153670079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d208e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 -- 172.21.15.31:0/153670079 >> 172.21.15.31:0/153670079 conn(0x55de92c8e200 msgr2=0x55de92d1c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 -- 172.21.15.31:0/153670079 shutdown_connections 2023-10-06T21:29:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.491+0000 7fb110fa4b80 1 -- 172.21.15.31:0/153670079 wait complete. 2023-10-06T21:29:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 -- start start 2023-10-06T21:29:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 Processor -- start 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d29410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92d2b4b0 con 0x55de92d19520 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92d29950 con 0x55de92d18c00 2023-10-06T21:29:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.492+0000 7fb110fa4b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de92d29ae0 con 0x55de92d243d0 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb110fa4b80 10 client.0 _ll_register_callbacks cb 0x7fb0f00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d29410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54756/0 (socket says 172.21.15.31:54756) 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42978/0 (socket says 172.21.15.31:42978) 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 -- 172.21.15.31:0/44880863 learned_addr learned my addr 172.21.15.31:0/44880863 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f77fe700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d29410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46032/0 (socket says 172.21.15.31:46032) 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 -- 172.21.15.31:0/44880863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 msgr2=0x55de92d29410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d29410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 -- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 msgr2=0x55de92d2abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f67fc700 1 -- 172.21.15.31:0/44880863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81a6420 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.493+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0f67fc700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009c40 tx=0x7fb0ec027b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690e8a2acee963ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0daff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00b740 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0e809e290 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fb0ec00b8a0 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0ec00ba80 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb110fa4b80 10 client.16875 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fb0ec00b500 con 0x55de92d19520 2023-10-06T21:29:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.494+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0ec044080 con 0x55de92d19520 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0ec03e030 con 0x55de92d19520 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 20 client.16875 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d88610 0x55de92d8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 4 client.16875 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de92c65ad0 con 0x55de92d88610 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.495+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d88610 0x55de92d8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.496+0000 7fb0f6ffd700 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d88610 0x55de92d8aa30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a0849f6247b55c server_cookie=c04c695eb007757 in_seq=0 out_seq=0 2023-10-06T21:29:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.496+0000 7fb0daff5700 10 client.16875 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb0daff5700 1 -- 172.21.15.31:0/44880863 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de92c65ad0 con 0x55de92d88610 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb0daff5700 10 client.16875 handle_command_reply: tid=0 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 client.16875 shutdown 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 msgr2=0x55de92d28ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009c40 tx=0x7fb0ec027b10 comp rx=0 tx=0).stop 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 shutdown_connections 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de92d243d0 0x55de92d29410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de92d18c00 0x55de92d2abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55de92d88610 0x55de92d8aa30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 --2- 172.21.15.31:0/44880863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de92d19520 0x55de92d28ed0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 >> 172.21.15.31:0/44880863 conn(0x55de92c8e200 msgr2=0x55de92d14bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.521+0000 7fb0daff5700 0 client.16875 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 shutdown_connections 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb110fa4b80 1 -- 172.21.15.31:0/44880863 wait complete. 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb110fa4b80 20 client.16875 trim_cache size 0 max 16384 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:29:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:29:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:29:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 msgr2=0x7fb0e00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 0x7fb0e00a3b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009fd0 tx=0x7fb0f0009920 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 msgr2=0x7fb0f8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.522+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9c0 tx=0x7fb0e800ecd0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 shutdown_connections 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f81a31c0 0x7fb0f81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f81010d0 0x7fb0f8073390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0e00a1670 0x7fb0e00a3b30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 --2- 172.21.15.31:0/136402180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f81061f0 0x7fb0f80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 >> 172.21.15.31:0/136402180 conn(0x7fb0f80fc930 msgr2=0x7fb0f80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 shutdown_connections 2023-10-06T21:29:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:09.523+0000 7fb0fd5ff700 1 -- 172.21.15.31:0/136402180 wait complete. 2023-10-06T21:29:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:09 smithi031 ceph-mon[98539]: pgmap v255: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:09 smithi146 ceph-mon[102809]: pgmap v255: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:10.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 138879 1.77 MB/sec execute 300 sec latency 19.529 ms 2023-10-06T21:29:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:09 smithi142 ceph-mon[102951]: pgmap v255: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:11.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 139158 1.76 MB/sec execute 301 sec latency 25.786 ms 2023-10-06T21:29:12.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 139510 1.76 MB/sec execute 302 sec latency 19.002 ms 2023-10-06T21:29:12.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:12 smithi142 ceph-mon[102951]: pgmap v256: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:12 smithi031 ceph-mon[98539]: pgmap v256: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:12 smithi146 ceph-mon[102809]: pgmap v256: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2023-10-06T21:29:13.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 139864 1.77 MB/sec execute 303 sec latency 26.415 ms 2023-10-06T21:29:14.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 140142 1.76 MB/sec execute 304 sec latency 195.815 ms 2023-10-06T21:29:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:14 smithi142 ceph-mon[102951]: pgmap v257: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:14 smithi142 ceph-mon[102951]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:29:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:14 smithi031 ceph-mon[98539]: pgmap v257: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:14 smithi031 ceph-mon[98539]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:29:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:14 smithi146 ceph-mon[102809]: pgmap v257: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:14 smithi146 ceph-mon[102809]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:29:15.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 140449 1.76 MB/sec execute 305 sec latency 152.416 ms 2023-10-06T21:29:16.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 140693 1.76 MB/sec execute 306 sec latency 370.491 ms 2023-10-06T21:29:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:16 smithi142 ceph-mon[102951]: pgmap v259: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:16.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:16 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:16 smithi031 ceph-mon[98539]: pgmap v259: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:16 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:16 smithi146 ceph-mon[102809]: pgmap v259: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:16 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:17.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 140969 1.75 MB/sec execute 307 sec latency 18.610 ms 2023-10-06T21:29:17.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:17 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:17.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:17 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:17.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:17 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:29:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:17 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:17 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:17 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:29:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:17 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:17 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:17 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:29:18.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 141570 1.77 MB/sec execute 308 sec latency 21.865 ms 2023-10-06T21:29:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:18 smithi142 ceph-mon[102951]: pgmap v260: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:18 smithi031 ceph-mon[98539]: pgmap v260: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:18 smithi146 ceph-mon[102809]: pgmap v260: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-10-06T21:29:19.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 141756 1.76 MB/sec execute 309 sec latency 22.737 ms 2023-10-06T21:29:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:19 smithi031 ceph-mon[98539]: pgmap v261: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:19 smithi146 ceph-mon[102809]: pgmap v261: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:20.348 INFO:tasks.workunit.client.0.smithi031.stdout: 1 142145 1.77 MB/sec execute 310 sec latency 18.770 ms 2023-10-06T21:29:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:19 smithi142 ceph-mon[102951]: pgmap v261: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:21.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 142293 1.77 MB/sec execute 311 sec latency 153.604 ms 2023-10-06T21:29:22.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 142452 1.76 MB/sec execute 312 sec latency 293.540 ms 2023-10-06T21:29:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:22 smithi142 ceph-mon[102951]: pgmap v262: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:22 smithi031 ceph-mon[98539]: pgmap v262: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:22 smithi146 ceph-mon[102809]: pgmap v262: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2023-10-06T21:29:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 -- 172.21.15.31:0/3570797733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81640732e0 msgr2=0x7f81640736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 --2- 172.21.15.31:0/3570797733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81640732e0 0x7f81640736c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8154009990 tx=0x7f8154009ca0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 -- 172.21.15.31:0/3570797733 shutdown_connections 2023-10-06T21:29:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 --2- 172.21.15.31:0/3570797733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f816410d6a0 0x7f816410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 --2- 172.21.15.31:0/3570797733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81640732e0 0x7f81640736c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 --2- 172.21.15.31:0/3570797733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8164073c00 0x7f816410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 -- 172.21.15.31:0/3570797733 >> 172.21.15.31:0/3570797733 conn(0x7f81640fc950 msgr2=0x7f81640fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 -- 172.21.15.31:0/3570797733 shutdown_connections 2023-10-06T21:29:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.684+0000 7f8169bff700 1 -- 172.21.15.31:0/3570797733 wait complete. 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.685+0000 7f8169bff700 1 Processor -- start 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.685+0000 7f8169bff700 1 -- start start 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8164073c00 0x7f8164199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e300 con 0x7f816410d6a0 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e470 con 0x7f8164073c00 2023-10-06T21:29:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8169bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e630 con 0x7f81640732e0 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8163fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43078/0 (socket says 172.21.15.31:43078) 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 -- 172.21.15.31:0/4229995035 learned_addr learned my addr 172.21.15.31:0/4229995035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8163fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33680/0 (socket says 172.21.15.31:33680) 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f8162ffd700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8164073c00 0x7f8164199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 -- 172.21.15.31:0/4229995035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8164073c00 msgr2=0x7f8164199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8164073c00 0x7f8164199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.686+0000 7f81637fe700 1 -- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 msgr2=0x7f816419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f81637fe700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f81637fe700 1 -- 172.21.15.31:0/4229995035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540095f0 con 0x7f81640732e0 2023-10-06T21:29:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8163fff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f81637fe700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8154005740 tx=0x7f8154005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8160ff9700 1 -- 172.21.15.31:0/4229995035 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815401d070 con 0x7f81640732e0 2023-10-06T21:29:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816419e8b0 con 0x7f81640732e0 2023-10-06T21:29:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8160ff9700 1 -- 172.21.15.31:0/4229995035 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8154022470 con 0x7f81640732e0 2023-10-06T21:29:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8160ff9700 1 -- 172.21.15.31:0/4229995035 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154017650 con 0x7f81640732e0 2023-10-06T21:29:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.687+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81641a6150 con 0x7f81640732e0 2023-10-06T21:29:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.691+0000 7f8160ff9700 1 -- 172.21.15.31:0/4229995035 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f81540225e0 con 0x7f81640732e0 2023-10-06T21:29:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 Processor -- start 2023-10-06T21:29:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 -- start start 2023-10-06T21:29:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f8160ff9700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 0x7f814c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f8160ff9700 1 -- 172.21.15.31:0/4229995035 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f81540ce4a0 con 0x7f81640732e0 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf733400 con 0x55fabf7ca520 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf724bc0 con 0x55fabf7c9c00 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f817d52eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7224e0 con 0x55fabf7d53d0 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.692+0000 7f8163fff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 0x7f814c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f81637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f81637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:38658/0 (socket says 172.21.15.31:38658) 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f81637fe700 1 -- 172.21.15.31:0/2302075083 learned_addr learned my addr 172.21.15.31:0/2302075083 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8163fff700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 -- 172.21.15.31:0/2302075083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 msgr2=0x55fabf7d77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 -- 172.21.15.31:0/2302075083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 msgr2=0x55fabf7c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 -- 172.21.15.31:0/2302075083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabf548d80 con 0x55fabf7ca520 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f81637fe700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8163fff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 0x7f814c0a3b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f815c009ec0 tx=0x7f815c003cf0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.693+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d4e90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8158009c80 tx=0x7f8158009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148ff2089a76a6b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.694+0000 7f8149ffb700 1 -- 172.21.15.31:0/2302075083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158011cb0 con 0x55fabf7ca520 2023-10-06T21:29:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.694+0000 7f8149ffb700 1 -- 172.21.15.31:0/2302075083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8158011e10 con 0x55fabf7ca520 2023-10-06T21:29:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.694+0000 7f8149ffb700 1 -- 172.21.15.31:0/2302075083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158003df0 con 0x55fabf7ca520 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.694+0000 7f817d52eb80 1 -- 172.21.15.31:0/2302075083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 msgr2=0x55fabf7d4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.694+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d4e90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8158009c80 tx=0x7f8158009f90 comp rx=0 tx=0).stop 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 -- 172.21.15.31:0/2302075083 shutdown_connections 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2302075083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d4e90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 -- 172.21.15.31:0/2302075083 >> 172.21.15.31:0/2302075083 conn(0x55fabf73f200 msgr2=0x55fabf7c6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 -- 172.21.15.31:0/2302075083 shutdown_connections 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 -- 172.21.15.31:0/2302075083 wait complete. 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.695+0000 7f817d52eb80 1 -- start start 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 Processor -- start 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf71ddd0 con 0x55fabf7ca520 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf6ba6b0 con 0x55fabf7c9c00 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf6ba350 con 0x55fabf7d53d0 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.696+0000 7f817d52eb80 10 client.0 _ll_register_callbacks cb 0x55fabf5177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33696/0 (socket says 172.21.15.31:33696) 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 -- 172.21.15.31:0/2778396312 learned_addr learned my addr 172.21.15.31:0/2778396312 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f81637fe700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8163fff700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 -- 172.21.15.31:0/2778396312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 msgr2=0x55fabf7d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 -- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 msgr2=0x55fabf7d4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7d4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 -- 172.21.15.31:0/2778396312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81641a6150 con 0x55fabf7ca520 2023-10-06T21:29:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.697+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8158005830 tx=0x7f8158005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914757da00965394 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f8142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815801a6d0 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fabf7bdd50 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f815801a830 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815801a9f0 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f817d52eb80 10 client.16887 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f81580098e0 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.698+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fabf7bdd50 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8158023c00 con 0x55fabf7ca520 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 10 client.16887 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 20 client.16887 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf7e4ca0 0x55fabf7e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 4 client.16887 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fabf716db0 con 0x55fabf7e4ca0 2023-10-06T21:29:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.699+0000 7f81637fe700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf7e4ca0 0x55fabf7e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.700+0000 7f8142ffd700 10 client.16887 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.700+0000 7f81637fe700 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf7e4ca0 0x55fabf7e70c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e3759e6ef178fd server_cookie=5423c78fa1777ae2 in_seq=0 out_seq=0 2023-10-06T21:29:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.700+0000 7f8142ffd700 1 -- 172.21.15.31:0/2778396312 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fabf716db0 con 0x55fabf7e4ca0 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.700+0000 7f8142ffd700 10 client.16887 handle_command_reply: tid=0 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.700+0000 7f817d52eb80 1 client.16887 shutdown 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 msgr2=0x55fabf7d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8158005830 tx=0x7f8158005ce0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 shutdown_connections 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7d53d0 0x55fabf7d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7c9c00 0x55fabf7d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf7e4ca0 0x55fabf7e70c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2778396312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7ca520 0x55fabf7d2000 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f8142ffd700 0 client.16887 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 >> 172.21.15.31:0/2778396312 conn(0x55fabf73f200 msgr2=0x55fabf7c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 shutdown_connections 2023-10-06T21:29:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 1 -- 172.21.15.31:0/2778396312 wait complete. 2023-10-06T21:29:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.701+0000 7f817d52eb80 20 client.16887 trim_cache size 0 max 16384 2023-10-06T21:29:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 Processor -- start 2023-10-06T21:29:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 -- start start 2023-10-06T21:29:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf59e8c0 con 0x55fabf7c9c00 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7c6dc0 con 0x55fabf7d53d0 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f817d52eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7dc480 con 0x55fabf7ca520 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f81637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f81637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33704/0 (socket says 172.21.15.31:33704) 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f81637fe700 1 -- 172.21.15.31:0/2678848630 learned_addr learned my addr 172.21.15.31:0/2678848630 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.718+0000 7f8162ffd700 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f8163fff700 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 -- 172.21.15.31:0/2678848630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 msgr2=0x55fabf7d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 -- 172.21.15.31:0/2678848630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 msgr2=0x55fabf7cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 -- 172.21.15.31:0/2678848630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabf531c00 con 0x55fabf7c9c00 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f81637fe700 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8154006010 tx=0x7f8154009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2637dfdfa3b6bf9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f814a7fc700 1 -- 172.21.15.31:0/2678848630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815409ada0 con 0x55fabf7c9c00 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f814a7fc700 1 -- 172.21.15.31:0/2678848630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f815409bb40 con 0x55fabf7c9c00 2023-10-06T21:29:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.719+0000 7f814a7fc700 1 -- 172.21.15.31:0/2678848630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540998c0 con 0x55fabf7c9c00 2023-10-06T21:29:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 -- 172.21.15.31:0/2678848630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 msgr2=0x55fabf7dbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8154006010 tx=0x7f8154009ca0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 -- 172.21.15.31:0/2678848630 shutdown_connections 2023-10-06T21:29:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7dbf40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 --2- 172.21.15.31:0/2678848630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 -- 172.21.15.31:0/2678848630 >> 172.21.15.31:0/2678848630 conn(0x55fabf73f200 msgr2=0x55fabf7e10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 -- 172.21.15.31:0/2678848630 shutdown_connections 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.720+0000 7f817d52eb80 1 -- 172.21.15.31:0/2678848630 wait complete. 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.721+0000 7f817d52eb80 1 -- start start 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.721+0000 7f817d52eb80 1 Processor -- start 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.721+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.721+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.721+0000 7f817d52eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f817d52eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7cf450 con 0x55fabf7c9c00 2023-10-06T21:29:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f817d52eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7cf5c0 con 0x55fabf7d53d0 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f817d52eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabf7cf780 con 0x55fabf7ca520 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f817d52eb80 10 client.0 _ll_register_callbacks cb 0x7f8154005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33708/0 (socket says 172.21.15.31:33708) 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 -- 172.21.15.31:0/3558954670 learned_addr learned my addr 172.21.15.31:0/3558954670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f8162ffd700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f8163fff700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 -- 172.21.15.31:0/3558954670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 msgr2=0x55fabf7ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 -- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 msgr2=0x55fabf7ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.722+0000 7f81637fe700 1 -- 172.21.15.31:0/3558954670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81540b3e70 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f81637fe700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8154086050 tx=0x7f815409b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ed59482b4f0253 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f8142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154022ac0 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f81540b0070 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8158011e10 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815409f160 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f817d52eb80 10 client.16893 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fabf56f1f0 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.723+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8158011e10 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f815409aa50 con 0x55fabf7c9c00 2023-10-06T21:29:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 10 client.16893 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 20 client.16893 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf839280 0x55fabf83b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 4 client.16893 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fabf716ad0 con 0x55fabf839280 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.724+0000 7f8162ffd700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf839280 0x55fabf83b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.725+0000 7f8142ffd700 10 client.16893 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.725+0000 7f8162ffd700 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf839280 0x55fabf83b6a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c7a49932f1307a server_cookie=14fffec8916096a6 in_seq=0 out_seq=0 2023-10-06T21:29:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f8142ffd700 1 -- 172.21.15.31:0/3558954670 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fabf716ad0 con 0x55fabf839280 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f8142ffd700 10 client.16893 handle_command_reply: tid=0 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 client.16893 shutdown 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 msgr2=0x55fabf7ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8154086050 tx=0x7f815409b4c0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 shutdown_connections 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fabf7d53d0 0x55fabf7ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fabf839280 0x55fabf83b6a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fabf7c9c00 0x55fabf7ce280 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 --2- 172.21.15.31:0/3558954670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabf7ca520 0x55fabf7ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 >> 172.21.15.31:0/3558954670 conn(0x55fabf73f200 msgr2=0x55fabf7cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f8142ffd700 0 client.16893 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 shutdown_connections 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.748+0000 7f817d52eb80 1 -- 172.21.15.31:0/3558954670 wait complete. 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f817d52eb80 20 client.16893 trim_cache size 0 max 16384 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:29:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 msgr2=0x7f814c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 0x7f814c0a3b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f815c009ec0 tx=0x7f815c003cf0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 msgr2=0x7f8164199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8154005740 tx=0x7f8154005770 comp rx=0 tx=0).stop 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 shutdown_connections 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f814c0a1670 0x7f814c0a3b30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f816410d6a0 0x7f816419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81640732e0 0x7f8164199110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 --2- 172.21.15.31:0/4229995035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8164073c00 0x7f8164199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.749+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 >> 172.21.15.31:0/4229995035 conn(0x7f81640fc950 msgr2=0x7f816410f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.750+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 shutdown_connections 2023-10-06T21:29:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:22.750+0000 7f8169bff700 1 -- 172.21.15.31:0/4229995035 wait complete. 2023-10-06T21:29:23.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 142739 1.76 MB/sec execute 313 sec latency 20.739 ms 2023-10-06T21:29:24.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 143094 1.76 MB/sec execute 314 sec latency 16.212 ms 2023-10-06T21:29:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:24 smithi142 ceph-mon[102951]: pgmap v263: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 93 op/s 2023-10-06T21:29:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:24 smithi031 ceph-mon[98539]: pgmap v263: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 93 op/s 2023-10-06T21:29:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:24 smithi146 ceph-mon[102809]: pgmap v263: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 93 op/s 2023-10-06T21:29:25.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 143448 1.76 MB/sec execute 315 sec latency 25.544 ms 2023-10-06T21:29:26.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 143786 1.76 MB/sec execute 316 sec latency 25.538 ms 2023-10-06T21:29:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:26 smithi142 ceph-mon[102951]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 904 KiB/s wr, 84 op/s 2023-10-06T21:29:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:26 smithi031 ceph-mon[98539]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 904 KiB/s wr, 84 op/s 2023-10-06T21:29:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:26 smithi146 ceph-mon[102809]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 904 KiB/s wr, 84 op/s 2023-10-06T21:29:27.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 144215 1.75 MB/sec execute 317 sec latency 27.949 ms 2023-10-06T21:29:28.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 144491 1.75 MB/sec execute 318 sec latency 18.433 ms 2023-10-06T21:29:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:28 smithi142 ceph-mon[102951]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 834 KiB/s wr, 77 op/s 2023-10-06T21:29:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:28 smithi031 ceph-mon[98539]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 834 KiB/s wr, 77 op/s 2023-10-06T21:29:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:28 smithi146 ceph-mon[102809]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 834 KiB/s wr, 77 op/s 2023-10-06T21:29:29.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 145082 1.76 MB/sec execute 319 sec latency 18.625 ms 2023-10-06T21:29:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:29 smithi031 ceph-mon[98539]: pgmap v266: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-10-06T21:29:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:29 smithi146 ceph-mon[102809]: pgmap v266: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-10-06T21:29:30.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 145280 1.76 MB/sec execute 320 sec latency 17.825 ms 2023-10-06T21:29:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:29 smithi142 ceph-mon[102951]: pgmap v266: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-10-06T21:29:31.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 145678 1.77 MB/sec execute 321 sec latency 18.844 ms 2023-10-06T21:29:32.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 145844 1.77 MB/sec execute 322 sec latency 19.623 ms 2023-10-06T21:29:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:32 smithi142 ceph-mon[102951]: pgmap v267: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 811 KiB/s wr, 87 op/s 2023-10-06T21:29:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:32 smithi031 ceph-mon[98539]: pgmap v267: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 811 KiB/s wr, 87 op/s 2023-10-06T21:29:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:32 smithi146 ceph-mon[102809]: pgmap v267: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 811 KiB/s wr, 87 op/s 2023-10-06T21:29:33.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 146029 1.76 MB/sec execute 323 sec latency 16.992 ms 2023-10-06T21:29:34.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 146350 1.76 MB/sec execute 324 sec latency 23.701 ms 2023-10-06T21:29:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:34 smithi142 ceph-mon[102951]: pgmap v268: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:29:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:34 smithi031 ceph-mon[98539]: pgmap v268: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:29:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:34 smithi146 ceph-mon[102809]: pgmap v268: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2023-10-06T21:29:35.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 146670 1.76 MB/sec execute 325 sec latency 19.662 ms 2023-10-06T21:29:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 -- 172.21.15.31:0/1000854318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04681061f0 msgr2=0x7f046810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1000854318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04681061f0 0x7f046810dd70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f046000b0a0 tx=0x7f046000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 -- 172.21.15.31:0/1000854318 shutdown_connections 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1000854318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04681061f0 0x7f046810dd70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1000854318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681010d0 0x7f04681014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1000854318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0468101a80 0x7f0468105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 -- 172.21.15.31:0/1000854318 >> 172.21.15.31:0/1000854318 conn(0x7f04680fc930 msgr2=0x7f04680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 -- 172.21.15.31:0/1000854318 shutdown_connections 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.914+0000 7f046d7ff700 1 -- 172.21.15.31:0/1000854318 wait complete. 2023-10-06T21:29:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.915+0000 7f046d7ff700 1 Processor -- start 2023-10-06T21:29:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 -- start start 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681061f0 0x7f0468197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 0x7f046819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819c620 con 0x7f046819bba0 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819c790 con 0x7f04681010d0 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.916+0000 7f046d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819c950 con 0x7f04681061f0 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33224/0 (socket says 172.21.15.31:33224) 2023-10-06T21:29:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 0x7f046819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f0466ffd700 1 -- 172.21.15.31:0/1288084447 learned_addr learned my addr 172.21.15.31:0/1288084447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04667fc700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681061f0 0x7f0468197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 -- 172.21.15.31:0/1288084447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681061f0 msgr2=0x7f0468197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681061f0 0x7f0468197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 -- 172.21.15.31:0/1288084447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 msgr2=0x7f0468196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 -- 172.21.15.31:0/1288084447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c009520 con 0x7f046819bba0 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.917+0000 7f04677fe700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 0x7f046819c020 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f0460008ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.918+0000 7f044ffff700 1 -- 172.21.15.31:0/1288084447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000e040 con 0x7f046819bba0 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.918+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460009d00 con 0x7f046819bba0 2023-10-06T21:29:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.918+0000 7f044ffff700 1 -- 172.21.15.31:0/1288084447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f046001d470 con 0x7f046819bba0 2023-10-06T21:29:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.918+0000 7f044ffff700 1 -- 172.21.15.31:0/1288084447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600052a0 con 0x7f046819bba0 2023-10-06T21:29:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.918+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04681a4140 con 0x7f046819bba0 2023-10-06T21:29:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.921+0000 7f044ffff700 1 -- 172.21.15.31:0/1288084447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f046001d5e0 con 0x7f046819bba0 2023-10-06T21:29:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.923+0000 7f04810a0b80 1 Processor -- start 2023-10-06T21:29:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.923+0000 7f044ffff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 0x7f04500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.923+0000 7f04810a0b80 1 -- start start 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.923+0000 7f044ffff700 1 -- 172.21.15.31:0/1288084447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f04600ce3f0 con 0x7f046819bba0 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.923+0000 7f04667fc700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 0x7f04500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07aa97e0 0x55ea07aabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07a03400 con 0x55ea07aa97e0 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea079f4bc0 con 0x55ea07a99c00 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04810a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea079f24e0 con 0x55ea07a9a520 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33232/0 (socket says 172.21.15.31:33232) 2023-10-06T21:29:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f0466ffd700 1 -- 172.21.15.31:0/1145523893 learned_addr learned my addr 172.21.15.31:0/1145523893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04667fc700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04677fe700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07aa97e0 0x55ea07aabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f04667fc700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 0x7f04500a3ab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0468198810 tx=0x7f045c011720 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.924+0000 7f0466ffd700 1 -- 172.21.15.31:0/1145523893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 msgr2=0x55ea07aa92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f0466ffd700 1 -- 172.21.15.31:0/1145523893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07aa97e0 msgr2=0x55ea07aabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07aa97e0 0x55ea07aabbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f0466ffd700 1 -- 172.21.15.31:0/1145523893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea07818d80 con 0x55ea07a99c00 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f04667fc700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04580117e0 tx=0x7f0458011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c64c7aa4771fac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f044d7fa700 1 -- 172.21.15.31:0/1145523893 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800f480 con 0x55ea07a99c00 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f044d7fa700 1 -- 172.21.15.31:0/1145523893 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f045800fa70 con 0x55ea07a99c00 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.925+0000 7f044d7fa700 1 -- 172.21.15.31:0/1145523893 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458019950 con 0x55ea07a99c00 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 -- 172.21.15.31:0/1145523893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 msgr2=0x55ea07a99fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04580117e0 tx=0x7f0458011af0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 -- 172.21.15.31:0/1145523893 shutdown_connections 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07aa97e0 0x55ea07aabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07a99c00 0x55ea07a99fe0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1145523893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 -- 172.21.15.31:0/1145523893 >> 172.21.15.31:0/1145523893 conn(0x55ea07a0f200 msgr2=0x55ea07a96080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 -- 172.21.15.31:0/1145523893 shutdown_connections 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.926+0000 7f04810a0b80 1 -- 172.21.15.31:0/1145523893 wait complete. 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.927+0000 7f04810a0b80 1 -- start start 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.927+0000 7f04810a0b80 1 Processor -- start 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea079eddd0 con 0x55ea07a99c00 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea0798a6b0 con 0x55ea07aa97e0 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea0798a350 con 0x55ea07a9a520 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04810a0b80 10 client.0 _ll_register_callbacks cb 0x55ea077e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60294/0 (socket says 172.21.15.31:60294) 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04667fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 -- 172.21.15.31:0/3901161401 learned_addr learned my addr 172.21.15.31:0/3901161401 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04677fe700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 -- 172.21.15.31:0/3901161401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 msgr2=0x55ea07aa6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 -- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 msgr2=0x55ea07aa6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f0466ffd700 1 -- 172.21.15.31:0/3901161401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c011ca0 con 0x55ea07a99c00 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.928+0000 7f04677fe700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:29:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f0466ffd700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f045801c040 tx=0x7f045800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef6ed9d2ef95dfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f044aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458005ce0 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea07a8dd50 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0458005250 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0458005410 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f04810a0b80 10 client.16905 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.929+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f0458011440 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ea07a8dd50 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0458019590 con 0x55ea07a99c00 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 20 client.16905 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07ab9270 0x55ea07abb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 4 client.16905 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.930+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea079e6db0 con 0x55ea07ab9270 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.931+0000 7f04667fc700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07ab9270 0x55ea07abb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.931+0000 7f04667fc700 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07ab9270 0x55ea07abb690 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a2c7e5655ba05d server_cookie=5dbb25cedc734194 in_seq=0 out_seq=0 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.931+0000 7f044aff5700 10 client.16905 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f044aff5700 1 -- 172.21.15.31:0/3901161401 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea079e6db0 con 0x55ea07ab9270 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f044aff5700 10 client.16905 handle_command_reply: tid=0 2023-10-06T21:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 client.16905 shutdown 2023-10-06T21:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 msgr2=0x55ea07aa9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f045801c040 tx=0x7f045800f730 comp rx=0 tx=0).stop 2023-10-06T21:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 shutdown_connections 2023-10-06T21:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07ab9270 0x55ea07abb690 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07aa9270 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 --2- 172.21.15.31:0/3901161401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07aa6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 >> 172.21.15.31:0/3901161401 conn(0x55ea07a0f200 msgr2=0x55ea07a95c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f044aff5700 0 client.16905 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 shutdown_connections 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.932+0000 7f04810a0b80 1 -- 172.21.15.31:0/3901161401 wait complete. 2023-10-06T21:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.933+0000 7f04810a0b80 20 client.16905 trim_cache size 0 max 16384 2023-10-06T21:29:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.953+0000 7f04810a0b80 1 Processor -- start 2023-10-06T21:29:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.953+0000 7f04810a0b80 1 -- start start 2023-10-06T21:29:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea0786e8c0 con 0x55ea07a9a520 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07a94e30 con 0x55ea07aa97e0 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04810a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07aa22d0 con 0x55ea07a99c00 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52364/0 (socket says 172.21.15.31:52364) 2023-10-06T21:29:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60298/0 (socket says 172.21.15.31:60298) 2023-10-06T21:29:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f0466ffd700 1 -- 172.21.15.31:0/1246371663 learned_addr learned my addr 172.21.15.31:0/1246371663 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.954+0000 7f04677fe700 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 -- 172.21.15.31:0/1246371663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 msgr2=0x55ea07aa6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 -- 172.21.15.31:0/1246371663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 msgr2=0x55ea07aa5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 -- 172.21.15.31:0/1246371663 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea07801c00 con 0x55ea07a99c00 2023-10-06T21:29:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f04667fc700 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0458000c00 tx=0x7f045801bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a6b4beceb118df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f044dffb700 1 -- 172.21.15.31:0/1246371663 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458015030 con 0x55ea07a99c00 2023-10-06T21:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f044dffb700 1 -- 172.21.15.31:0/1246371663 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04580056e0 con 0x55ea07a99c00 2023-10-06T21:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.955+0000 7f044dffb700 1 -- 172.21.15.31:0/1246371663 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04580038e0 con 0x55ea07a99c00 2023-10-06T21:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 -- 172.21.15.31:0/1246371663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 msgr2=0x55ea07aa8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0458000c00 tx=0x7f045801bfc0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 -- 172.21.15.31:0/1246371663 shutdown_connections 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aa6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a99c00 0x55ea07aa8da0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1246371663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a9a520 0x55ea07aa5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 -- 172.21.15.31:0/1246371663 >> 172.21.15.31:0/1246371663 conn(0x55ea07a0f200 msgr2=0x55ea07aa1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 -- 172.21.15.31:0/1246371663 shutdown_connections 2023-10-06T21:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.956+0000 7f04810a0b80 1 -- 172.21.15.31:0/1246371663 wait complete. 2023-10-06T21:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 -- start start 2023-10-06T21:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 Processor -- start 2023-10-06T21:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07b06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07aac840 con 0x55ea07a99c00 2023-10-06T21:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07aac980 con 0x55ea07aa97e0 2023-10-06T21:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea07aacb10 con 0x55ea07a9a520 2023-10-06T21:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.957+0000 7f04810a0b80 10 client.0 _ll_register_callbacks cb 0x7f04600074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60310/0 (socket says 172.21.15.31:60310) 2023-10-06T21:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 -- 172.21.15.31:0/1912153879 learned_addr learned my addr 172.21.15.31:0/1912153879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f04667fc700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07b06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f04677fe700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 -- 172.21.15.31:0/1912153879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 msgr2=0x55ea07b06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07b06830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 -- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 msgr2=0x55ea07aac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 -- 172.21.15.31:0/1912153879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c00f830 con 0x55ea07a99c00 2023-10-06T21:29:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f0466ffd700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f045801c040 tx=0x7f0458019e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afea0ccb9e5185 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.958+0000 7f044aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458015040 con 0x55ea07a99c00 2023-10-06T21:29:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f045800fa70 con 0x55ea07a99c00 2023-10-06T21:29:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0458016070 con 0x55ea07a99c00 2023-10-06T21:29:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0458019760 con 0x55ea07a99c00 2023-10-06T21:29:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f04810a0b80 10 client.16911 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f0458011440 con 0x55ea07a99c00 2023-10-06T21:29:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.959+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f045800fa70 con 0x55ea07a99c00 2023-10-06T21:29:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0458036480 con 0x55ea07a99c00 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 10 client.16911 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 20 client.16911 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07b0dbd0 0x55ea07b0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 4 client.16911 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:29:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea079e6ad0 con 0x55ea07b0dbd0 2023-10-06T21:29:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.960+0000 7f04667fc700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07b0dbd0 0x55ea07b0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.961+0000 7f04667fc700 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07b0dbd0 0x55ea07b0fff0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994d869c7c4d6788 server_cookie=a04556a3cf636759 in_seq=0 out_seq=0 2023-10-06T21:29:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.961+0000 7f044aff5700 10 client.16911 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.984+0000 7f044aff5700 1 -- 172.21.15.31:0/1912153879 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea079e6ad0 con 0x55ea07b0dbd0 2023-10-06T21:29:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.984+0000 7f044aff5700 10 client.16911 handle_command_reply: tid=0 2023-10-06T21:29:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.984+0000 7f04810a0b80 1 client.16911 shutdown 2023-10-06T21:29:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 msgr2=0x55ea07b062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f045801c040 tx=0x7f0458019e40 comp rx=0 tx=0).stop 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 shutdown_connections 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea07aa97e0 0x55ea07aac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ea07b0dbd0 0x55ea07b0fff0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ea07a99c00 0x55ea07b062f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 --2- 172.21.15.31:0/1912153879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea07a9a520 0x55ea07b06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f044aff5700 0 client.16911 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 >> 172.21.15.31:0/1912153879 conn(0x55ea07a0f200 msgr2=0x55ea07aa5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 shutdown_connections 2023-10-06T21:29:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 1 -- 172.21.15.31:0/1912153879 wait complete. 2023-10-06T21:29:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.985+0000 7f04810a0b80 20 client.16911 trim_cache size 0 max 16384 2023-10-06T21:29:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:29:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:29:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:29:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:29:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 msgr2=0x7f04500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 0x7f04500a3ab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0468198810 tx=0x7f045c011720 comp rx=0 tx=0).stop 2023-10-06T21:29:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 msgr2=0x7f046819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 0x7f046819c020 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f0460008ea0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 shutdown_connections 2023-10-06T21:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04500a15f0 0x7f04500a3ab0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f046819bba0 0x7f046819c020 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04681010d0 0x7f0468196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 --2- 172.21.15.31:0/1288084447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04681061f0 0x7f0468197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 >> 172.21.15.31:0/1288084447 conn(0x7f04680fc930 msgr2=0x7f04680fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 shutdown_connections 2023-10-06T21:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:35.986+0000 7f046d7ff700 1 -- 172.21.15.31:0/1288084447 wait complete. 2023-10-06T21:29:36.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 147021 1.76 MB/sec execute 326 sec latency 28.120 ms 2023-10-06T21:29:36.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:36 smithi142 ceph-mon[102951]: pgmap v269: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:36 smithi031 ceph-mon[98539]: pgmap v269: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:36 smithi146 ceph-mon[102809]: pgmap v269: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:37.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 147364 1.76 MB/sec execute 327 sec latency 30.930 ms 2023-10-06T21:29:38.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 147795 1.75 MB/sec execute 328 sec latency 26.479 ms 2023-10-06T21:29:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:38 smithi142 ceph-mon[102951]: pgmap v270: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:38 smithi031 ceph-mon[98539]: pgmap v270: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:38 smithi146 ceph-mon[102809]: pgmap v270: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 948 KiB/s wr, 85 op/s 2023-10-06T21:29:39.349 INFO:tasks.workunit.client.0.smithi031.stdout: 1 148067 1.75 MB/sec execute 329 sec latency 21.445 ms 2023-10-06T21:29:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:39 smithi031 ceph-mon[98539]: pgmap v271: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:29:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:39 smithi146 ceph-mon[102809]: pgmap v271: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:29:40.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 148681 1.76 MB/sec execute 330 sec latency 19.923 ms 2023-10-06T21:29:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:39 smithi142 ceph-mon[102951]: pgmap v271: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-10-06T21:29:41.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 148957 1.77 MB/sec execute 331 sec latency 18.832 ms 2023-10-06T21:29:42.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 149253 1.77 MB/sec execute 332 sec latency 19.094 ms 2023-10-06T21:29:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:42 smithi142 ceph-mon[102951]: pgmap v272: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 898 KiB/s wr, 89 op/s 2023-10-06T21:29:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:42 smithi031 ceph-mon[98539]: pgmap v272: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 898 KiB/s wr, 89 op/s 2023-10-06T21:29:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:42 smithi146 ceph-mon[102809]: pgmap v272: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 898 KiB/s wr, 89 op/s 2023-10-06T21:29:43.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 149428 1.76 MB/sec execute 333 sec latency 19.174 ms 2023-10-06T21:29:44.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 149614 1.76 MB/sec execute 334 sec latency 23.277 ms 2023-10-06T21:29:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:44 smithi142 ceph-mon[102951]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-10-06T21:29:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:44 smithi031 ceph-mon[98539]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-10-06T21:29:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:44 smithi146 ceph-mon[102809]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-10-06T21:29:45.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 149974 1.76 MB/sec execute 335 sec latency 22.791 ms 2023-10-06T21:29:46.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 150383 1.76 MB/sec execute 336 sec latency 18.892 ms 2023-10-06T21:29:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:46 smithi142 ceph-mon[102951]: pgmap v274: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:46 smithi031 ceph-mon[98539]: pgmap v274: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:46 smithi146 ceph-mon[102809]: pgmap v274: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:47.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 150657 1.76 MB/sec execute 337 sec latency 26.330 ms 2023-10-06T21:29:48.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 151038 1.76 MB/sec execute 338 sec latency 27.054 ms 2023-10-06T21:29:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:48 smithi142 ceph-mon[102951]: pgmap v275: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:48 smithi031 ceph-mon[98539]: pgmap v275: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:48 smithi146 ceph-mon[102809]: pgmap v275: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 979 KiB/s wr, 88 op/s 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.155+0000 7f9567fff700 1 -- 172.21.15.31:0/856145861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 msgr2=0x7f956010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.155+0000 7f9567fff700 1 --2- 172.21.15.31:0/856145861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f956010dd70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f955800b0a0 tx=0x7f955800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 -- 172.21.15.31:0/856145861 shutdown_connections 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 --2- 172.21.15.31:0/856145861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f956010dd70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 --2- 172.21.15.31:0/856145861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601010d0 0x7f95601014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 --2- 172.21.15.31:0/856145861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9560101a80 0x7f9560105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 -- 172.21.15.31:0/856145861 >> 172.21.15.31:0/856145861 conn(0x7f95600fc950 msgr2=0x7f95600fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 -- 172.21.15.31:0/856145861 shutdown_connections 2023-10-06T21:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.156+0000 7f9567fff700 1 -- 172.21.15.31:0/856145861 wait complete. 2023-10-06T21:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 Processor -- start 2023-10-06T21:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 -- start start 2023-10-06T21:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f95600738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 0x7f95601a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95601a5af0 con 0x7f95601061f0 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95601a5c30 con 0x7f95601a31c0 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.157+0000 7f9567fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95601a5da0 con 0x7f95601010d0 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f956559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f95600738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47764/0 (socket says 172.21.15.31:47764) 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 -- 172.21.15.31:0/3989191143 learned_addr learned my addr 172.21.15.31:0/3989191143 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f956659c700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 0x7f95601a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 -- 172.21.15.31:0/3989191143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 msgr2=0x7f95601a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 0x7f95601a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 -- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 msgr2=0x7f95600738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f95600738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f9565d9b700 1 -- 172.21.15.31:0/3989191143 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009d00 con 0x7f95601010d0 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.158+0000 7f956659c700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 0x7f95601a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e9a0 tx=0x7f955c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9556ffd700 1 -- 172.21.15.31:0/3989191143 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c009790 con 0x7f95601010d0 2023-10-06T21:29:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95601a6080 con 0x7f95601010d0 2023-10-06T21:29:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9556ffd700 1 -- 172.21.15.31:0/3989191143 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f955c010430 con 0x7f95601010d0 2023-10-06T21:29:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9556ffd700 1 -- 172.21.15.31:0/3989191143 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00cea0 con 0x7f95601010d0 2023-10-06T21:29:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.159+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95601a6590 con 0x7f95601010d0 2023-10-06T21:29:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.163+0000 7f9556ffd700 1 -- 172.21.15.31:0/3989191143 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f955c0105a0 con 0x7f95601010d0 2023-10-06T21:29:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.164+0000 7f957ba10b80 1 Processor -- start 2023-10-06T21:29:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.164+0000 7f957ba10b80 1 -- start start 2023-10-06T21:29:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.164+0000 7f9556ffd700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 0x7f954c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.164+0000 7f9556ffd700 1 -- 172.21.15.31:0/3989191143 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f955c014070 con 0x7f95601010d0 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f956659c700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 0x7f954c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4bc00 0x5576ddd4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576ddd597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddcb5400 con 0x5576ddd4c520 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddca6bc0 con 0x5576ddd4bc00 2023-10-06T21:29:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f957ba10b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddca44e0 con 0x5576ddd573d0 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f956559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f956559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36068/0 (socket says 172.21.15.31:36068) 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f956559a700 1 -- 172.21.15.31:0/3546926807 learned_addr learned my addr 172.21.15.31:0/3546926807 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f956659c700 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576ddd597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.165+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4bc00 0x5576ddd4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 -- 172.21.15.31:0/3546926807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 msgr2=0x5576ddd597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576ddd597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 -- 172.21.15.31:0/3546926807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4bc00 msgr2=0x5576ddd4bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4bc00 0x5576ddd4bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 -- 172.21.15.31:0/3546926807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576ddacad80 con 0x5576ddd4c520 2023-10-06T21:29:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956659c700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 0x7f954c0a3ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f955800b780 tx=0x7f9558009c90 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f956559a700 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9550009cb0 tx=0x7f9550011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8805478d05262cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f954bfff700 1 -- 172.21.15.31:0/3546926807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011d20 con 0x5576ddd4c520 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f954bfff700 1 -- 172.21.15.31:0/3546926807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9550011e80 con 0x5576ddd4c520 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.166+0000 7f954bfff700 1 -- 172.21.15.31:0/3546926807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550005020 con 0x5576ddd4c520 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 -- 172.21.15.31:0/3546926807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 msgr2=0x5576ddd56e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9550009cb0 tx=0x7f9550011440 comp rx=0 tx=0).stop 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 -- 172.21.15.31:0/3546926807 shutdown_connections 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576ddd597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4bc00 0x5576ddd4bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3546926807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd56e90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 -- 172.21.15.31:0/3546926807 >> 172.21.15.31:0/3546926807 conn(0x5576ddcc1200 msgr2=0x5576ddd48080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 -- 172.21.15.31:0/3546926807 shutdown_connections 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.167+0000 7f957ba10b80 1 -- 172.21.15.31:0/3546926807 wait complete. 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.168+0000 7f957ba10b80 1 -- start start 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.168+0000 7f957ba10b80 1 Processor -- start 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd573d0 0x5576ddd54620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddc9fdd0 con 0x5576ddd4c520 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddc3c6b0 con 0x5576ddd573d0 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddc3c350 con 0x5576ddd4bc00 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f957ba10b80 10 client.0 _ll_register_callbacks cb 0x5576dda997a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f956559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47776/0 (socket says 172.21.15.31:47776) 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f956559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36078/0 (socket says 172.21.15.31:36078) 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f9565d9b700 1 -- 172.21.15.31:0/2583469739 learned_addr learned my addr 172.21.15.31:0/2583469739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.169+0000 7f956659c700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd573d0 0x5576ddd54620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 -- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd573d0 msgr2=0x5576ddd54620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd573d0 0x5576ddd54620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 -- 172.21.15.31:0/2583469739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 msgr2=0x5576ddd53ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 -- 172.21.15.31:0/2583469739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500098e0 con 0x5576ddd4bc00 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f956559a700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f9565d9b700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f955c099fb0 tx=0x7f955c010950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adc9c9047e4a77b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f95497fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09a2b0 con 0x5576ddd4bc00 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f955c0b1020 con 0x5576ddd4bc00 2023-10-06T21:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.170+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576ddc98db0 con 0x5576ddd4bc00 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c063020 con 0x5576ddd4bc00 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 10 client.26596 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5576ddab3c00 con 0x5576ddd4bc00 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5576ddc98db0 con 0x5576ddd4bc00 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f955c0997a0 con 0x5576ddd4bc00 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 10 client.26596 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 10 client.26596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 20 client.26596 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576ddd66ec0 0x5576ddd692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 4 client.26596 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.171+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576ddc98ad0 con 0x5576ddd66ec0 2023-10-06T21:29:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.172+0000 7f956559a700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576ddd66ec0 0x5576ddd692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.172+0000 7f956559a700 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576ddd66ec0 0x5576ddd692b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105a86048cbf5d3a server_cookie=6bd57a85696971f2 in_seq=0 out_seq=0 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.172+0000 7f95497fa700 10 client.26596 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f95497fa700 1 -- 172.21.15.31:0/2583469739 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5576ddc98ad0 con 0x5576ddd66ec0 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f95497fa700 10 client.26596 handle_command_reply: tid=0 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 client.26596 shutdown 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 msgr2=0x5576ddd56d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f955c099fb0 tx=0x7f955c010950 comp rx=0 tx=0).stop 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 shutdown_connections 2023-10-06T21:29:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd573d0 0x5576ddd54620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd56d70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576ddd66ec0 0x5576ddd692b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 --2- 172.21.15.31:0/2583469739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4c520 0x5576ddd53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 >> 172.21.15.31:0/2583469739 conn(0x5576ddcc1200 msgr2=0x5576ddd47c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f95497fa700 0 client.26596 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.173+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 shutdown_connections 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.174+0000 7f957ba10b80 1 -- 172.21.15.31:0/2583469739 wait complete. 2023-10-06T21:29:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.174+0000 7f957ba10b80 20 client.26596 trim_cache size 0 max 16384 2023-10-06T21:29:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 Processor -- start 2023-10-06T21:29:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 -- start start 2023-10-06T21:29:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd56910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 0x5576ddd512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddb208c0 con 0x5576ddd573d0 2023-10-06T21:29:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddd496a0 con 0x5576ddd4c520 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.192+0000 7f957ba10b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddd48dc0 con 0x5576ddd4bc00 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 0x5576ddd512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 0x5576ddd512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36084/0 (socket says 172.21.15.31:36084) 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956659c700 1 -- 172.21.15.31:0/491722529 learned_addr learned my addr 172.21.15.31:0/491722529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd56910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f9565d9b700 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 -- 172.21.15.31:0/491722529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 msgr2=0x5576ddd537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 -- 172.21.15.31:0/491722529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 msgr2=0x5576ddd512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 0x5576ddd512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.193+0000 7f956559a700 1 -- 172.21.15.31:0/491722529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576ddaf11f0 con 0x5576ddd4c520 2023-10-06T21:29:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.194+0000 7f956559a700 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd56910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9550009230 tx=0x7f95500096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108ba36abcfd0ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.194+0000 7f9554ff9700 1 -- 172.21.15.31:0/491722529 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550013ea0 con 0x5576ddd4c520 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.194+0000 7f9554ff9700 1 -- 172.21.15.31:0/491722529 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9550005ce0 con 0x5576ddd4c520 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.194+0000 7f9554ff9700 1 -- 172.21.15.31:0/491722529 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550003680 con 0x5576ddd4c520 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- 172.21.15.31:0/491722529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 msgr2=0x5576ddd56910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd56910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9550009230 tx=0x7f95500096d0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- 172.21.15.31:0/491722529 shutdown_connections 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd573d0 0x5576ddd512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd4bc00 0x5576ddd537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 --2- 172.21.15.31:0/491722529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd56910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- 172.21.15.31:0/491722529 >> 172.21.15.31:0/491722529 conn(0x5576ddcc1200 msgr2=0x5576ddd4f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- 172.21.15.31:0/491722529 shutdown_connections 2023-10-06T21:29:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- 172.21.15.31:0/491722529 wait complete. 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 -- start start 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.195+0000 7f957ba10b80 1 Processor -- start 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576dddb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ddd5ad50 con 0x5576ddd4bc00 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576dddba200 con 0x5576ddd4c520 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576dddba340 con 0x5576ddd573d0 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f957ba10b80 10 client.0 _ll_register_callbacks cb 0x7f955c099ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f9565d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36092/0 (socket says 172.21.15.31:36092) 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f9565d9b700 1 -- 172.21.15.31:0/3053656543 learned_addr learned my addr 172.21.15.31:0/3053656543 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.196+0000 7f956559a700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f956659c700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576dddb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 -- 172.21.15.31:0/3053656543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 msgr2=0x5576dddb9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576dddb9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 -- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 msgr2=0x5576ddd5a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd5a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 -- 172.21.15.31:0/3053656543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955001d040 con 0x5576ddd4bc00 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f956659c700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576dddb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:29:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f956559a700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd5a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f9565d9b700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f955c0967b0 tx=0x7f955c010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57229f4de7885686 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f95497fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c09a2b0 con 0x5576ddd4bc00 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576ddd3fd50 con 0x5576ddd4bc00 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.197+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f955c0b1070 con 0x5576ddd4bc00 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c09e400 con 0x5576ddd4bc00 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 10 client.16929 fetch_fsmap learned FSMap version 34 2023-10-06T21:29:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5576ddac9250 con 0x5576ddd4bc00 2023-10-06T21:29:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5576ddd3fd50 con 0x5576ddd4bc00 2023-10-06T21:29:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f955c0997a0 con 0x5576ddd4bc00 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 10 client.16929 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 20 client.16929 populate_metadata read hostname 'smithi031' 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576dddbb660 0x5576dddbda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 4 client.16929 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576ddaebf10 con 0x5576dddbb660 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.198+0000 7f956559a700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576dddbb660 0x5576dddbda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.199+0000 7f956559a700 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576dddbb660 0x5576dddbda80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3d540d29c005c2 server_cookie=befb15fbb0d71fef in_seq=0 out_seq=0 2023-10-06T21:29:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.199+0000 7f95497fa700 10 client.16929 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.225+0000 7f95497fa700 1 -- 172.21.15.31:0/3053656543 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576ddaebf10 con 0x5576dddbb660 2023-10-06T21:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.225+0000 7f95497fa700 10 client.16929 handle_command_reply: tid=0 2023-10-06T21:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.225+0000 7f957ba10b80 1 client.16929 shutdown 2023-10-06T21:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 msgr2=0x5576ddd59ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f955c0967b0 tx=0x7f955c010a70 comp rx=0 tx=0).stop 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 shutdown_connections 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576ddd573d0 0x5576dddb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5576dddbb660 0x5576dddbda80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5576ddd4bc00 0x5576ddd59ef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f95497fa700 0 client.16929 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 --2- 172.21.15.31:0/3053656543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576ddd4c520 0x5576ddd5a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 >> 172.21.15.31:0/3053656543 conn(0x5576ddcc1200 msgr2=0x5576ddd4db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 shutdown_connections 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 1 -- 172.21.15.31:0/3053656543 wait complete. 2023-10-06T21:29:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.226+0000 7f957ba10b80 20 client.16929 trim_cache size 0 max 16384 2023-10-06T21:29:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:29:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:29:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:29:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 msgr2=0x7f954c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 0x7f954c0a3ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f955800b780 tx=0x7f9558009c90 comp rx=0 tx=0).stop 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 msgr2=0x7f9560073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e9a0 tx=0x7f955c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 shutdown_connections 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95601a31c0 0x7f95601a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95601010d0 0x7f9560073390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f954c0a15f0 0x7f954c0a3ab0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 --2- 172.21.15.31:0/3989191143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95601061f0 0x7f95600738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 >> 172.21.15.31:0/3989191143 conn(0x7f95600fc950 msgr2=0x7f95600fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 shutdown_connections 2023-10-06T21:29:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:29:49.227+0000 7f9567fff700 1 -- 172.21.15.31:0/3989191143 wait complete. 2023-10-06T21:29:49.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 151391 1.75 MB/sec execute 339 sec latency 27.962 ms 2023-10-06T21:29:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:49 smithi031 ceph-mon[98539]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:29:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:49 smithi146 ceph-mon[102809]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:29:50.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 151775 1.76 MB/sec execute 340 sec latency 23.821 ms 2023-10-06T21:29:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:49 smithi142 ceph-mon[102951]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-10-06T21:29:51.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 152187 1.76 MB/sec execute 341 sec latency 341.478 ms 2023-10-06T21:29:52.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 152365 1.76 MB/sec execute 342 sec latency 21.611 ms 2023-10-06T21:29:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:52 smithi142 ceph-mon[102951]: pgmap v277: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 981 KiB/s wr, 87 op/s 2023-10-06T21:29:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:52 smithi031 ceph-mon[98539]: pgmap v277: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 981 KiB/s wr, 87 op/s 2023-10-06T21:29:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:52 smithi146 ceph-mon[102809]: pgmap v277: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 981 KiB/s wr, 87 op/s 2023-10-06T21:29:53.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 152770 1.77 MB/sec execute 343 sec latency 20.031 ms 2023-10-06T21:29:54.350 INFO:tasks.workunit.client.0.smithi031.stdout: 1 152939 1.76 MB/sec execute 344 sec latency 18.706 ms 2023-10-06T21:29:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:54 smithi142 ceph-mon[102951]: pgmap v278: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-10-06T21:29:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:54 smithi031 ceph-mon[98539]: pgmap v278: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-10-06T21:29:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:54 smithi146 ceph-mon[102809]: pgmap v278: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-10-06T21:29:55.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 153125 1.76 MB/sec execute 345 sec latency 22.049 ms 2023-10-06T21:29:56.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 153449 1.76 MB/sec execute 346 sec latency 19.807 ms 2023-10-06T21:29:56.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:56 smithi142 ceph-mon[102951]: pgmap v279: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:56 smithi031 ceph-mon[98539]: pgmap v279: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:56 smithi146 ceph-mon[102809]: pgmap v279: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:57.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 153768 1.75 MB/sec execute 347 sec latency 19.902 ms 2023-10-06T21:29:58.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 154111 1.76 MB/sec execute 348 sec latency 28.557 ms 2023-10-06T21:29:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:58 smithi142 ceph-mon[102951]: pgmap v280: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:58 smithi031 ceph-mon[98539]: pgmap v280: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:58 smithi146 ceph-mon[102809]: pgmap v280: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 865 KiB/s wr, 84 op/s 2023-10-06T21:29:59.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 154459 1.75 MB/sec execute 349 sec latency 24.617 ms 2023-10-06T21:30:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:29:59 smithi031 ceph-mon[98539]: pgmap v281: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:30:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:29:59.999+0000 7f51d5f22700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:30:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:29:59.999+0000 7f51d5f22700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-06T21:30:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:29:59.999+0000 7f51d5f22700 -1 log_channel(cluster) log [ERR] : mds.f(mds.2): Metadata damage detected 2023-10-06T21:30:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:29:59 smithi146 ceph-mon[102809]: pgmap v281: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:30:00.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 154878 1.75 MB/sec execute 350 sec latency 25.103 ms 2023-10-06T21:30:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:29:59 smithi142 ceph-mon[102951]: pgmap v281: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-10-06T21:30:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-mon[98539]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:30:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-mon[98539]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-06T21:30:01.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:00 smithi031 ceph-mon[98539]: mds.f(mds.2): Metadata damage detected 2023-10-06T21:30:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:00 smithi146 ceph-mon[102809]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:30:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:00 smithi146 ceph-mon[102809]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-06T21:30:01.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:00 smithi146 ceph-mon[102809]: mds.f(mds.2): Metadata damage detected 2023-10-06T21:30:01.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 155129 1.75 MB/sec execute 351 sec latency 82.377 ms 2023-10-06T21:30:01.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:00 smithi142 ceph-mon[102951]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:30:01.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:00 smithi142 ceph-mon[102951]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-06T21:30:01.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:00 smithi142 ceph-mon[102951]: mds.f(mds.2): Metadata damage detected 2023-10-06T21:30:02.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:01 smithi031 ceph-mon[98539]: pgmap v282: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 988 KiB/s wr, 87 op/s 2023-10-06T21:30:02.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:01 smithi146 ceph-mon[102809]: pgmap v282: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 988 KiB/s wr, 87 op/s 2023-10-06T21:30:02.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 155713 1.76 MB/sec execute 352 sec latency 18.905 ms 2023-10-06T21:30:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/1811707732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8101100 msgr2=0x7f3fd81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/1811707732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8101100 0x7f3fd81014e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009990 tx=0x7f3fd4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/1811707732 shutdown_connections 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/1811707732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8106220 0x7f3fd810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/1811707732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8101100 0x7f3fd81014e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/1811707732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd8101ab0 0x7f3fd8105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/1811707732 >> 172.21.15.31:0/1811707732 conn(0x7f3fd80fc9a0 msgr2=0x7f3fd80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/1811707732 shutdown_connections 2023-10-06T21:30:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.384+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/1811707732 wait complete. 2023-10-06T21:30:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.386+0000 7f3fdf7ff700 1 Processor -- start 2023-10-06T21:30:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.386+0000 7f3fdf7ff700 1 -- start start 2023-10-06T21:30:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8101100 0x7f3fd81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd819dcd0 0x7f3fd819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819e750 con 0x7f3fd8106220 2023-10-06T21:30:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819e8c0 con 0x7f3fd819dcd0 2023-10-06T21:30:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdf7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819ea80 con 0x7f3fd8101100 2023-10-06T21:30:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48958/0 (socket says 172.21.15.31:48958) 2023-10-06T21:30:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4257640602 learned_addr learned my addr 172.21.15.31:0/4257640602 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8101100 0x7f3fd81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fddd9c700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd819dcd0 0x7f3fd819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4257640602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8101100 msgr2=0x7f3fd81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8101100 0x7f3fd81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4257640602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd819dcd0 msgr2=0x7f3fd819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd819dcd0 0x7f3fd819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.387+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4257640602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009520 con 0x7f3fd8106220 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800ea30 tx=0x7f3fc800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fce7fc700 1 -- 172.21.15.31:0/4257640602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800cbd0 con 0x7f3fd8106220 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fce7fc700 1 -- 172.21.15.31:0/4257640602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc800eed0 con 0x7f3fd8106220 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fce7fc700 1 -- 172.21.15.31:0/4257640602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8003c40 con 0x7f3fd8106220 2023-10-06T21:30:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd40095f0 con 0x7f3fd8106220 2023-10-06T21:30:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.388+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81a6280 con 0x7f3fd8106220 2023-10-06T21:30:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.391+0000 7f3fce7fc700 1 -- 172.21.15.31:0/4257640602 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3fc8003e00 con 0x7f3fd8106220 2023-10-06T21:30:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.393+0000 7f3fce7fc700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 0x7f3fc40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.393+0000 7f3fce7fc700 1 -- 172.21.15.31:0/4257640602 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f3fc8014070 con 0x7f3fd8106220 2023-10-06T21:30:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.393+0000 7f3ff3127b80 1 Processor -- start 2023-10-06T21:30:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.393+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 0x7f3fc40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.393+0000 7f3ff3127b80 1 -- start start 2023-10-06T21:30:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 0x5588d96a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96ac3d0 0x5588d96ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d960a400 con 0x5588d96a0c00 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d95fbbc0 con 0x5588d96a1520 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d95f94e0 con 0x5588d96ac3d0 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 0x5588d96a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42394/0 (socket says 172.21.15.31:42394) 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 0x5588d96a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48966/0 (socket says 172.21.15.31:48966) 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1062999739 learned_addr learned my addr 172.21.15.31:0/1062999739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fddd9c700 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96ac3d0 0x5588d96ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.394+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 0x7f3fc40a3ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006010 tx=0x7f3fd400b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1062999739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96ac3d0 msgr2=0x5588d96ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96ac3d0 0x5588d96ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1062999739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 msgr2=0x5588d96a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 0x5588d96a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1062999739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588d941fd80 con 0x5588d96a1520 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80a2190 tx=0x7f3fc8099d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786e48333ed603d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fc37fe700 1 -- 172.21.15.31:0/1062999739 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc809a330 con 0x5588d96a1520 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fc37fe700 1 -- 172.21.15.31:0/1062999739 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc809e480 con 0x5588d96a1520 2023-10-06T21:30:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.395+0000 7f3fc37fe700 1 -- 172.21.15.31:0/1062999739 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8096a40 con 0x5588d96a1520 2023-10-06T21:30:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1062999739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 msgr2=0x5588d96abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80a2190 tx=0x7f3fc8099d90 comp rx=0 tx=0).stop 2023-10-06T21:30:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1062999739 shutdown_connections 2023-10-06T21:30:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96ac3d0 0x5588d96ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a0c00 0x5588d96a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1062999739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96abe90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1062999739 >> 172.21.15.31:0/1062999739 conn(0x5588d9616200 msgr2=0x5588d969d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1062999739 shutdown_connections 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1062999739 wait complete. 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.396+0000 7f3ff3127b80 1 -- start start 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 Processor -- start 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d96a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d96a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d95f4dd0 con 0x5588d96a1520 2023-10-06T21:30:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d95916b0 con 0x5588d96ac3d0 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d9591350 con 0x5588d96a0c00 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.397+0000 7f3ff3127b80 10 client.0 _ll_register_callbacks cb 0x5588d93ee7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57858/0 (socket says 172.21.15.31:57858) 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdd59b700 1 -- 172.21.15.31:0/4179543047 learned_addr learned my addr 172.21.15.31:0/4179543047 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fddd9c700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d96a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d96a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4179543047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 msgr2=0x5588d96a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 msgr2=0x5588d96a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d96a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/4179543047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81a6280 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fddd9c700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d96a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d96a9670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8098680 tx=0x7f3fc80cd310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb56ae176262116 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.398+0000 7f3fc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc809a330 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f3fc800cd30 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588d9694d50 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc809e170 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3ff3127b80 10 client.16941 fetch_fsmap learned FSMap version 34 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5588d9408c00 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.399+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5588d9694d50 con 0x5588d96a1520 2023-10-06T21:30:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fc80b2020 con 0x5588d96a1520 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 10 client.16941 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 20 client.16941 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d96bbac0 0x5588d96bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 4 client.16941 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588d95eddb0 con 0x5588d96bbac0 2023-10-06T21:30:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.400+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d96bbac0 0x5588d96bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:01 smithi142 ceph-mon[102951]: pgmap v282: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 988 KiB/s wr, 87 op/s 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.401+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d96bbac0 0x5588d96bdeb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168c606959ea33c7 server_cookie=dfec0358fff2f8e7 in_seq=0 out_seq=0 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.401+0000 7f3fc0ff9700 10 client.16941 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/4179543047 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5588d95eddb0 con 0x5588d96bbac0 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3fc0ff9700 10 client.16941 handle_command_reply: tid=0 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 client.16941 shutdown 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 msgr2=0x5588d96a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d96a9670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8098680 tx=0x7f3fc80cd310 comp rx=0 tx=0).stop 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 shutdown_connections 2023-10-06T21:30:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d96a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d96bbac0 0x5588d96bdeb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/4179543047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d96a9670 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 >> 172.21.15.31:0/4179543047 conn(0x5588d9616200 msgr2=0x5588d969cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.402+0000 7f3fc0ff9700 0 client.16941 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.403+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 shutdown_connections 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.403+0000 7f3ff3127b80 1 -- 172.21.15.31:0/4179543047 wait complete. 2023-10-06T21:30:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.403+0000 7f3ff3127b80 20 client.16941 trim_cache size 0 max 16384 2023-10-06T21:30:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 Processor -- start 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 -- start start 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96ac3d0 0x5588d96a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d94758c0 con 0x5588d96ac3d0 2023-10-06T21:30:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d96a1e20 con 0x5588d96a1520 2023-10-06T21:30:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.422+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d969ddc0 con 0x5588d96a0c00 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fddd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96ac3d0 0x5588d96a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57864/0 (socket says 172.21.15.31:57864) 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 -- 172.21.15.31:0/3574188439 learned_addr learned my addr 172.21.15.31:0/3574188439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 -- 172.21.15.31:0/3574188439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 msgr2=0x5588d96b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 -- 172.21.15.31:0/3574188439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96ac3d0 msgr2=0x5588d96a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96ac3d0 0x5588d96a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 -- 172.21.15.31:0/3574188439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588d94461f0 con 0x5588d96a0c00 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.423+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005910 tx=0x7f3fd4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cc0cd8d58f7f49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.424+0000 7f3fc3fff700 1 -- 172.21.15.31:0/3574188439 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4035a90 con 0x5588d96a0c00 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.424+0000 7f3fc3fff700 1 -- 172.21.15.31:0/3574188439 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd401e040 con 0x5588d96a0c00 2023-10-06T21:30:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.424+0000 7f3fc3fff700 1 -- 172.21.15.31:0/3574188439 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004c60 con 0x5588d96a0c00 2023-10-06T21:30:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- 172.21.15.31:0/3574188439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 msgr2=0x5588d96a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005910 tx=0x7f3fd4020000 comp rx=0 tx=0).stop 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- 172.21.15.31:0/3574188439 shutdown_connections 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96ac3d0 0x5588d96a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d96a3aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/3574188439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96a1520 0x5588d96b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- 172.21.15.31:0/3574188439 >> 172.21.15.31:0/3574188439 conn(0x5588d9616200 msgr2=0x5588d96a4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- 172.21.15.31:0/3574188439 shutdown_connections 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- 172.21.15.31:0/3574188439 wait complete. 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 -- start start 2023-10-06T21:30:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.425+0000 7f3ff3127b80 1 Processor -- start 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d9703a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d970e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d9704700 con 0x5588d96a1520 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d970ee40 con 0x5588d96ac3d0 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d970f000 con 0x5588d96a0c00 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3ff3127b80 10 client.0 _ll_register_callbacks cb 0x7f3fd40050e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48986/0 (socket says 172.21.15.31:48986) 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1127745081 learned_addr learned my addr 172.21.15.31:0/1127745081 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3fddd9c700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d970e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.426+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d9703a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1127745081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 msgr2=0x5588d9703a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d9703a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 msgr2=0x5588d970e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d970e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1127745081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd402c040 con 0x5588d96a1520 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8096da0 tx=0x7f3fc80988e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f38db14deb93b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80a0f80 con 0x5588d96a1520 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f3fc809a330 con 0x5588d96a1520 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.427+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc809e480 con 0x5588d96a1520 2023-10-06T21:30:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc80a26c0 con 0x5588d96a1520 2023-10-06T21:30:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 10 client.16947 fetch_fsmap learned FSMap version 34 2023-10-06T21:30:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5588d941e250 con 0x5588d96a1520 2023-10-06T21:30:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fc809e480 con 0x5588d96a1520 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fc80b2070 con 0x5588d96a1520 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 20 client.16947 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d97101e0 0x5588d97125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 4 client.16947 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588d95edad0 con 0x5588d97101e0 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.428+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d97101e0 0x5588d97125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.429+0000 7f3fdd59b700 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d97101e0 0x5588d97125d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e75bbe24d628c5 server_cookie=72b656fff73be3e9 in_seq=0 out_seq=0 2023-10-06T21:30:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.429+0000 7f3fc0ff9700 10 client.16947 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.453+0000 7f3fc0ff9700 1 -- 172.21.15.31:0/1127745081 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588d95edad0 con 0x5588d97101e0 2023-10-06T21:30:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.453+0000 7f3fc0ff9700 10 client.16947 handle_command_reply: tid=0 2023-10-06T21:30:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.453+0000 7f3ff3127b80 1 client.16947 shutdown 2023-10-06T21:30:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 msgr2=0x5588d9703f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8096da0 tx=0x7f3fc80988e0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 shutdown_connections 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588d96ac3d0 0x5588d970e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5588d96a0c00 0x5588d9703a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5588d97101e0 0x5588d97125d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 --2- 172.21.15.31:0/1127745081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d96a1520 0x5588d9703f80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3fc0ff9700 0 client.16947 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 >> 172.21.15.31:0/1127745081 conn(0x5588d9616200 msgr2=0x5588d96a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 shutdown_connections 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 1 -- 172.21.15.31:0/1127745081 wait complete. 2023-10-06T21:30:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.454+0000 7f3ff3127b80 20 client.16947 trim_cache size 0 max 16384 2023-10-06T21:30:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:30:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:30:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:30:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:30:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 msgr2=0x7f3fc40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 0x7f3fc40a3ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006010 tx=0x7f3fd400b260 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 msgr2=0x7f3fd8199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800ea30 tx=0x7f3fc800ed40 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 shutdown_connections 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd819dcd0 0x7f3fd819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd8101100 0x7f3fd81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3fc40a15f0 0x7f3fc40a3ab0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 --2- 172.21.15.31:0/4257640602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd8106220 0x7f3fd8199700 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.455+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 >> 172.21.15.31:0/4257640602 conn(0x7f3fd80fc9a0 msgr2=0x7f3fd80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.456+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 shutdown_connections 2023-10-06T21:30:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:02.456+0000 7f3fdf7ff700 1 -- 172.21.15.31:0/4257640602 wait complete. 2023-10-06T21:30:03.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 155904 1.76 MB/sec execute 353 sec latency 27.893 ms 2023-10-06T21:30:04.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 156302 1.77 MB/sec execute 354 sec latency 17.969 ms 2023-10-06T21:30:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:04 smithi142 ceph-mon[102951]: pgmap v283: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-10-06T21:30:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:04 smithi031 ceph-mon[98539]: pgmap v283: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-10-06T21:30:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:04 smithi146 ceph-mon[102809]: pgmap v283: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-10-06T21:30:05.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 156471 1.76 MB/sec execute 355 sec latency 18.552 ms 2023-10-06T21:30:05.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:05 smithi142 ceph-mon[102951]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:30:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:05 smithi031 ceph-mon[98539]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:30:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:05 smithi146 ceph-mon[102809]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:30:06.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 156648 1.76 MB/sec execute 356 sec latency 18.624 ms 2023-10-06T21:30:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:06 smithi142 ceph-mon[102951]: pgmap v285: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:06 smithi031 ceph-mon[98539]: pgmap v285: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:06 smithi146 ceph-mon[102809]: pgmap v285: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:07.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 156957 1.76 MB/sec execute 357 sec latency 23.595 ms 2023-10-06T21:30:08.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 157277 1.75 MB/sec execute 358 sec latency 17.615 ms 2023-10-06T21:30:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:08 smithi142 ceph-mon[102951]: pgmap v286: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:08 smithi031 ceph-mon[98539]: pgmap v286: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:08 smithi146 ceph-mon[102809]: pgmap v286: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-10-06T21:30:09.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 157641 1.76 MB/sec execute 359 sec latency 24.105 ms 2023-10-06T21:30:10.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:09 smithi031 ceph-mon[98539]: pgmap v287: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:10.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:09 smithi146 ceph-mon[102809]: pgmap v287: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:10.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 157968 1.75 MB/sec execute 360 sec latency 26.312 ms 2023-10-06T21:30:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:09 smithi142 ceph-mon[102951]: pgmap v287: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:11.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:10 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:11.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:10 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:11.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 158401 1.75 MB/sec execute 361 sec latency 27.463 ms 2023-10-06T21:30:11.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:10 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:12.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:11 smithi031 ceph-mon[98539]: pgmap v288: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:12.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:11 smithi146 ceph-mon[102809]: pgmap v288: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:12.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 158675 1.75 MB/sec execute 362 sec latency 18.573 ms 2023-10-06T21:30:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:11 smithi142 ceph-mon[102951]: pgmap v288: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-10-06T21:30:13.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159264 1.76 MB/sec execute 363 sec latency 16.662 ms 2023-10-06T21:30:14.351 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159455 1.76 MB/sec execute 364 sec latency 20.835 ms 2023-10-06T21:30:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:14 smithi142 ceph-mon[102951]: pgmap v289: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:14 smithi142 ceph-mon[102951]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:30:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:14 smithi031 ceph-mon[98539]: pgmap v289: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:14 smithi031 ceph-mon[98539]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:30:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:14 smithi146 ceph-mon[102809]: pgmap v289: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:14 smithi146 ceph-mon[102809]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:30:15.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159709 1.76 MB/sec execute 365 sec latency 428.035 ms 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 -- 172.21.15.31:0/2381951396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390101a80 msgr2=0x7fd390105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 --2- 172.21.15.31:0/2381951396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390101a80 0x7fd390105ac0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd384009990 tx=0x7fd384009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 -- 172.21.15.31:0/2381951396 shutdown_connections 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 --2- 172.21.15.31:0/2381951396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3901061f0 0x7fd39010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 --2- 172.21.15.31:0/2381951396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 0x7fd3901014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 --2- 172.21.15.31:0/2381951396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390101a80 0x7fd390105ac0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 -- 172.21.15.31:0/2381951396 >> 172.21.15.31:0/2381951396 conn(0x7fd3900fc950 msgr2=0x7fd3900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 -- 172.21.15.31:0/2381951396 shutdown_connections 2023-10-06T21:30:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.613+0000 7fd3953ff700 1 -- 172.21.15.31:0/2381951396 wait complete. 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.614+0000 7fd3953ff700 1 Processor -- start 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 -- start start 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 0x7fd390196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39019bab0 0x7fd39019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019c530 con 0x7fd3901061f0 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019c6a0 con 0x7fd39019bab0 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd3953ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019c860 con 0x7fd3901010d0 2023-10-06T21:30:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56256/0 (socket says 172.21.15.31:56256) 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd38e7fc700 1 -- 172.21.15.31:0/4091968555 learned_addr learned my addr 172.21.15.31:0/4091968555 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.615+0000 7fd38f7fe700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39019bab0 0x7fd39019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38effd700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 0x7fd390196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 -- 172.21.15.31:0/4091968555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 msgr2=0x7fd390196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 0x7fd390196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 -- 172.21.15.31:0/4091968555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39019bab0 msgr2=0x7fd39019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39019bab0 0x7fd39019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 -- 172.21.15.31:0/4091968555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840095f0 con 0x7fd3901061f0 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098e0 tx=0x7fd384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd377fff700 1 -- 172.21.15.31:0/4091968555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38401d070 con 0x7fd3901061f0 2023-10-06T21:30:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd377fff700 1 -- 172.21.15.31:0/4091968555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd384022470 con 0x7fd3901061f0 2023-10-06T21:30:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3901a3d60 con 0x7fd3901061f0 2023-10-06T21:30:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd377fff700 1 -- 172.21.15.31:0/4091968555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384017650 con 0x7fd3901061f0 2023-10-06T21:30:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.616+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3901a4180 con 0x7fd3901061f0 2023-10-06T21:30:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.620+0000 7fd377fff700 1 -- 172.21.15.31:0/4091968555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd3840225e0 con 0x7fd3901061f0 2023-10-06T21:30:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 Processor -- start 2023-10-06T21:30:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 -- start start 2023-10-06T21:30:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd377fff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 0x7fd3780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd377fff700 1 -- 172.21.15.31:0/4091968555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd3840ce6d0 con 0x7fd3901061f0 2023-10-06T21:30:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd38f7fe700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 0x7fd3780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9c13d0 0x56170f9c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f91f400 con 0x56170f9c13d0 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f910bc0 con 0x56170f9b6520 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.621+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f90e4e0 con 0x56170f9b5c00 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9c13d0 0x56170f9c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:40034/0 (socket says 172.21.15.31:40034) 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 -- 172.21.15.31:0/597612511 learned_addr learned my addr 172.21.15.31:0/597612511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 -- 172.21.15.31:0/597612511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 msgr2=0x56170f9c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 -- 172.21.15.31:0/597612511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9c13d0 msgr2=0x56170f9c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9c13d0 0x56170f9c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 -- 172.21.15.31:0/597612511 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170f734d80 con 0x56170f9b5c00 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38effd700 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d10 tx=0x7fd3800116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eab048840c23e9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.622+0000 7fd38f7fe700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 0x7fd3780a3ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd388014d50 tx=0x7fd388014350 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.623+0000 7fd3757fa700 1 -- 172.21.15.31:0/597612511 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800097e0 con 0x56170f9b5c00 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.623+0000 7fd3757fa700 1 -- 172.21.15.31:0/597612511 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd380009e60 con 0x56170f9b5c00 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.623+0000 7fd3757fa700 1 -- 172.21.15.31:0/597612511 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000f680 con 0x56170f9b5c00 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.623+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/597612511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 msgr2=0x56170f9b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.623+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d10 tx=0x7fd3800116c0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/597612511 shutdown_connections 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9c13d0 0x56170f9c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b5c00 0x56170f9b5fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/597612511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/597612511 >> 172.21.15.31:0/597612511 conn(0x56170f92b200 msgr2=0x56170f9b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/597612511 shutdown_connections 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/597612511 wait complete. 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 -- start start 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.624+0000 7fd3a8d12b80 1 Processor -- start 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b5c00 0x56170f9c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f909dd0 con 0x56170f9b6520 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f8a66b0 con 0x56170f9b5c00 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f8a6350 con 0x56170f9c13d0 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd3a8d12b80 10 client.0 _ll_register_callbacks cb 0x56170f7037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56286/0 (socket says 172.21.15.31:56286) 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 -- 172.21.15.31:0/1358714170 learned_addr learned my addr 172.21.15.31:0/1358714170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38effd700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b5c00 0x56170f9c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 -- 172.21.15.31:0/1358714170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 msgr2=0x56170f9be5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 -- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b5c00 msgr2=0x56170f9c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b5c00 0x56170f9c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.625+0000 7fd38e7fc700 1 -- 172.21.15.31:0/1358714170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd380009940 con 0x56170f9b6520 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd3840980e0 tx=0x7fd38409afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9204aa0699061b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd372ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38409b670 con 0x56170f9b6520 2023-10-06T21:30:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56170f9a9d50 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd38409f7c0 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3840a3970 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd3a8d12b80 10 client.16959 fetch_fsmap learned FSMap version 35 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56170f71dc00 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.626+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56170f9a9d50 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 35) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd384097460 con 0x56170f9b6520 2023-10-06T21:30:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 20 client.16959 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170f9d4e80 0x56170f9d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 4 client.16959 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56170f902db0 con 0x56170f9d4e80 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.627+0000 7fd38effd700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170f9d4e80 0x56170f9d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd38effd700 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170f9d4e80 0x56170f9d72a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101ab71cd36d178e server_cookie=387c4bf76ece5948 in_seq=0 out_seq=0 2023-10-06T21:30:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd372ff5700 10 client.16959 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd372ff5700 1 -- 172.21.15.31:0/1358714170 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56170f902db0 con 0x56170f9d4e80 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd372ff5700 10 client.16959 handle_command_reply: tid=0 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd3a8d12b80 1 client.16959 shutdown 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 msgr2=0x56170f9be010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.628+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd3840980e0 tx=0x7fd38409afd0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 shutdown_connections 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9be5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b5c00 0x56170f9c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170f9d4e80 0x56170f9d72a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1358714170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b6520 0x56170f9be010 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd372ff5700 0 client.16959 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 >> 172.21.15.31:0/1358714170 conn(0x56170f92b200 msgr2=0x56170f9b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 shutdown_connections 2023-10-06T21:30:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1358714170 wait complete. 2023-10-06T21:30:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.629+0000 7fd3a8d12b80 20 client.16959 trim_cache size 0 max 16384 2023-10-06T21:30:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.644+0000 7fd3a8d12b80 1 Processor -- start 2023-10-06T21:30:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.644+0000 7fd3a8d12b80 1 -- start start 2023-10-06T21:30:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.644+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.644+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.644+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f78a8c0 con 0x56170f9b5c00 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f9a9890 con 0x56170f9b6520 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f9b73f0 con 0x56170f9c13d0 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56296/0 (socket says 172.21.15.31:56296) 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 -- 172.21.15.31:0/697775702 learned_addr learned my addr 172.21.15.31:0/697775702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38f7fe700 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 -- 172.21.15.31:0/697775702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 msgr2=0x56170f9bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9bdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 -- 172.21.15.31:0/697775702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 msgr2=0x56170f9bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9bd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 -- 172.21.15.31:0/697775702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170f75b1f0 con 0x56170f9b5c00 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.645+0000 7fd38effd700 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fd0 tx=0x7fd380009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd05e2dae6cd1782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd375ffb700 1 -- 172.21.15.31:0/697775702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380005ce0 con 0x56170f9b5c00 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd375ffb700 1 -- 172.21.15.31:0/697775702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd380027580 con 0x56170f9b5c00 2023-10-06T21:30:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd375ffb700 1 -- 172.21.15.31:0/697775702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800097e0 con 0x56170f9b5c00 2023-10-06T21:30:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/697775702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 msgr2=0x56170f9c08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fd0 tx=0x7fd380009260 comp rx=0 tx=0).stop 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/697775702 shutdown_connections 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9c13d0 0x56170f9bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9c08c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/697775702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9b6520 0x56170f9bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/697775702 >> 172.21.15.31:0/697775702 conn(0x56170f92b200 msgr2=0x56170f9b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.646+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/697775702 shutdown_connections 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.647+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/697775702 wait complete. 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.647+0000 7fd3a8d12b80 1 -- start start 2023-10-06T21:30:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.647+0000 7fd3a8d12b80 1 Processor -- start 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.647+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.647+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 0x56170f9c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd3a8d12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9c13d0 0x56170f9c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f9c9000 con 0x56170f9b5c00 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f9c9140 con 0x56170f9c13d0 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd3a8d12b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170f9c9300 con 0x56170f9b6520 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd3a8d12b80 10 client.0 _ll_register_callbacks cb 0x56170f7037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56298/0 (socket says 172.21.15.31:56298) 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 -- 172.21.15.31:0/1756463274 learned_addr learned my addr 172.21.15.31:0/1756463274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 0x56170f9c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38f7fe700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9c13d0 0x56170f9c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 -- 172.21.15.31:0/1756463274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 msgr2=0x56170f9c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 0x56170f9c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 -- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9c13d0 msgr2=0x56170f9c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9c13d0 0x56170f9c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38effd700 1 -- 172.21.15.31:0/1756463274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56170f734d80 con 0x56170f9b5c00 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.648+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 0x56170f9c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd38effd700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd380009640 tx=0x7fd380011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71711eac2d669e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd372ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380005ce0 con 0x56170f9b5c00 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd380009e60 con 0x56170f9b5c00 2023-10-06T21:30:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd380036030 con 0x56170f9b5c00 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd380005880 con 0x56170f9b5c00 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd3a8d12b80 10 client.16965 fetch_fsmap learned FSMap version 35 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56170f733250 con 0x56170f9b5c00 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.649+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd380009e60 con 0x56170f9b5c00 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 35) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd3800113e0 con 0x56170f9b5c00 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 20 client.16965 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170fa29d90 0x56170fa2c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 4 client.16965 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56170f902ad0 con 0x56170fa29d90 2023-10-06T21:30:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.650+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170fa29d90 0x56170fa2c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.651+0000 7fd38e7fc700 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170fa29d90 0x56170fa2c1b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04e99cbaf6e595c server_cookie=cdb9229434391e4a in_seq=0 out_seq=0 2023-10-06T21:30:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.651+0000 7fd372ff5700 10 client.16965 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd372ff5700 1 -- 172.21.15.31:0/1756463274 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56170f902ad0 con 0x56170fa29d90 2023-10-06T21:30:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd372ff5700 10 client.16965 handle_command_reply: tid=0 2023-10-06T21:30:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 client.16965 shutdown 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 msgr2=0x56170f9cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd380009640 tx=0x7fd380011b50 comp rx=0 tx=0).stop 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 shutdown_connections 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56170f9c13d0 0x56170f9c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56170fa29d90 0x56170fa2c1b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56170f9b5c00 0x56170f9cc230 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 --2- 172.21.15.31:0/1756463274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56170f9b6520 0x56170f9c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 >> 172.21.15.31:0/1756463274 conn(0x56170f92b200 msgr2=0x56170f9b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd372ff5700 0 client.16965 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 shutdown_connections 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 1 -- 172.21.15.31:0/1756463274 wait complete. 2023-10-06T21:30:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.682+0000 7fd3a8d12b80 20 client.16965 trim_cache size 0 max 16384 2023-10-06T21:30:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:30:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:30:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:30:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 msgr2=0x7fd3780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 0x7fd3780a3ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd388014d50 tx=0x7fd388014350 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 msgr2=0x7fd390197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098e0 tx=0x7fd384004730 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 shutdown_connections 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39019bab0 0x7fd39019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3901010d0 0x7fd390196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd3780a15f0 0x7fd3780a3ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 --2- 172.21.15.31:0/4091968555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901061f0 0x7fd390197440 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 >> 172.21.15.31:0/4091968555 conn(0x7fd3900fc950 msgr2=0x7fd3900fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 shutdown_connections 2023-10-06T21:30:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:15.683+0000 7fd3953ff700 1 -- 172.21.15.31:0/4091968555 wait complete. 2023-10-06T21:30:16.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159709 1.76 MB/sec execute 366 sec latency 1428.164 ms 2023-10-06T21:30:16.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:16 smithi142 ceph-mon[102951]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:16 smithi031 ceph-mon[98539]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:16 smithi146 ceph-mon[102809]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:17.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159709 1.75 MB/sec execute 367 sec latency 2428.243 ms 2023-10-06T21:30:17.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:17 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:17 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:17 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:18.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159709 1.75 MB/sec execute 368 sec latency 3428.346 ms 2023-10-06T21:30:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:18 smithi142 ceph-mon[102951]: pgmap v292: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:18 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:18 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:18 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:30:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:18 smithi031 ceph-mon[98539]: pgmap v292: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:18 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:18 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:18 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:30:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:18 smithi146 ceph-mon[102809]: pgmap v292: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-06T21:30:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:18 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:18 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:18 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:30:19.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159709 1.74 MB/sec execute 369 sec latency 4428.452 ms 2023-10-06T21:30:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:19 smithi031 ceph-mon[98539]: pgmap v293: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:19 smithi146 ceph-mon[102809]: pgmap v293: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:20.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159776 1.74 MB/sec execute 370 sec latency 4540.001 ms 2023-10-06T21:30:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:19 smithi142 ceph-mon[102951]: pgmap v293: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:21.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159776 1.74 MB/sec execute 371 sec latency 1516.969 ms 2023-10-06T21:30:22.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159776 1.73 MB/sec execute 372 sec latency 2517.112 ms 2023-10-06T21:30:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:22 smithi142 ceph-mon[102951]: pgmap v294: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:22 smithi031 ceph-mon[98539]: pgmap v294: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:22 smithi146 ceph-mon[102809]: pgmap v294: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 752 KiB/s wr, 83 op/s 2023-10-06T21:30:23.352 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159776 1.73 MB/sec execute 373 sec latency 3517.228 ms 2023-10-06T21:30:24.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159776 1.72 MB/sec execute 374 sec latency 4517.298 ms 2023-10-06T21:30:24.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:24 smithi142 ceph-mon[102951]: pgmap v295: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 258 KiB/s wr, 41 op/s 2023-10-06T21:30:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:24 smithi031 ceph-mon[98539]: pgmap v295: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 258 KiB/s wr, 41 op/s 2023-10-06T21:30:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:24 smithi146 ceph-mon[102809]: pgmap v295: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 258 KiB/s wr, 41 op/s 2023-10-06T21:30:25.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 159947 1.72 MB/sec execute 375 sec latency 4622.840 ms 2023-10-06T21:30:26.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 160095 1.72 MB/sec execute 376 sec latency 135.427 ms 2023-10-06T21:30:26.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:26 smithi142 ceph-mon[102951]: pgmap v296: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 233 KiB/s wr, 37 op/s 2023-10-06T21:30:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:26 smithi031 ceph-mon[98539]: pgmap v296: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 233 KiB/s wr, 37 op/s 2023-10-06T21:30:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:26 smithi146 ceph-mon[102809]: pgmap v296: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 233 KiB/s wr, 37 op/s 2023-10-06T21:30:27.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 160253 1.71 MB/sec execute 377 sec latency 291.606 ms 2023-10-06T21:30:28.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 160608 1.71 MB/sec execute 378 sec latency 17.728 ms 2023-10-06T21:30:28.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:28 smithi142 ceph-mon[102951]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 215 KiB/s wr, 34 op/s 2023-10-06T21:30:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:28 smithi031 ceph-mon[98539]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 215 KiB/s wr, 34 op/s 2023-10-06T21:30:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:28 smithi146 ceph-mon[102809]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 215 KiB/s wr, 34 op/s 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.848+0000 7fb0f9dff700 1 -- 172.21.15.31:0/2775043202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41061f0 msgr2=0x7fb0f410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.848+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/2775043202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41061f0 0x7fb0f410dd70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b0a0 tx=0x7fb0ec00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 -- 172.21.15.31:0/2775043202 shutdown_connections 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/2775043202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41061f0 0x7fb0f410dd70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/2775043202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41010d0 0x7fb0f41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/2775043202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f4101a80 0x7fb0f4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 -- 172.21.15.31:0/2775043202 >> 172.21.15.31:0/2775043202 conn(0x7fb0f40fc950 msgr2=0x7fb0f40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 -- 172.21.15.31:0/2775043202 shutdown_connections 2023-10-06T21:30:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.849+0000 7fb0f9dff700 1 -- 172.21.15.31:0/2775043202 wait complete. 2023-10-06T21:30:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.850+0000 7fb0f9dff700 1 Processor -- start 2023-10-06T21:30:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.850+0000 7fb0f9dff700 1 -- start start 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 0x7fb0f41970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 0x7fb0f419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419c710 con 0x7fb0f41010d0 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419c880 con 0x7fb0f41061f0 2023-10-06T21:30:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f9dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419ca40 con 0x7fb0f419bc90 2023-10-06T21:30:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 0x7fb0f419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43250/0 (socket says 172.21.15.31:43250) 2023-10-06T21:30:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/1042669264 learned_addr learned my addr 172.21.15.31:0/1042669264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.851+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 0x7fb0f41970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/1042669264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 msgr2=0x7fb0f419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 0x7fb0f419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 msgr2=0x7fb0f41970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 0x7fb0f41970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/1042669264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009520 con 0x7fb0f41061f0 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 0x7fb0f41970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 0x7fb0f419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e970 tx=0x7fb0e800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f0ff9700 1 -- 172.21.15.31:0/1042669264 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8009790 con 0x7fb0f41061f0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec009d00 con 0x7fb0f41061f0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.852+0000 7fb0f0ff9700 1 -- 172.21.15.31:0/1042669264 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0e800eec0 con 0x7fb0f41061f0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.853+0000 7fb0f0ff9700 1 -- 172.21.15.31:0/1042669264 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80054b0 con 0x7fb0f41061f0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.853+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4073510 con 0x7fb0f41061f0 2023-10-06T21:30:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.856+0000 7fb0f0ff9700 1 -- 172.21.15.31:0/1042669264 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb0e8004750 con 0x7fb0f41061f0 2023-10-06T21:30:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.857+0000 7fb0f0ff9700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.857+0000 7fb0f0ff9700 1 -- 172.21.15.31:0/1042669264 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb0e8014070 con 0x7fb0f41061f0 2023-10-06T21:30:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.858+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.858+0000 7fb10d816b80 1 Processor -- start 2023-10-06T21:30:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.858+0000 7fb10d816b80 1 -- start start 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.858+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e40 tx=0x7fb0e4009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b3687bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36886e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 0x556b368897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b367e5400 con 0x556b3687bc00 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b367d6bc0 con 0x556b368873d0 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb10d816b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b367d44e0 con 0x556b3687c520 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b3687bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b3687bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56134/0 (socket says 172.21.15.31:56134) 2023-10-06T21:30:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 0x556b368897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb0f37fe700 1 -- 172.21.15.31:0/668390982 learned_addr learned my addr 172.21.15.31:0/668390982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.859+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36886e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 -- 172.21.15.31:0/668390982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 msgr2=0x556b36886e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36886e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 -- 172.21.15.31:0/668390982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 msgr2=0x556b3687bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b3687bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 -- 172.21.15.31:0/668390982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b365fad80 con 0x556b368873d0 2023-10-06T21:30:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 0x556b368897c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b780 tx=0x7fb0ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315bd38841ce0a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0d9ffb700 1 -- 172.21.15.31:0/668390982 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e040 con 0x556b368873d0 2023-10-06T21:30:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0d9ffb700 1 -- 172.21.15.31:0/668390982 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0ec006060 con 0x556b368873d0 2023-10-06T21:30:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.860+0000 7fb0d9ffb700 1 -- 172.21.15.31:0/668390982 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec01c440 con 0x556b368873d0 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 -- 172.21.15.31:0/668390982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 msgr2=0x556b368897c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 0x556b368897c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b780 tx=0x7fb0ec009c00 comp rx=0 tx=0).stop 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 -- 172.21.15.31:0/668390982 shutdown_connections 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b368873d0 0x556b368897c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b3687bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 --2- 172.21.15.31:0/668390982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36886e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 -- 172.21.15.31:0/668390982 >> 172.21.15.31:0/668390982 conn(0x556b367f1200 msgr2=0x556b36878080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.861+0000 7fb10d816b80 1 -- 172.21.15.31:0/668390982 shutdown_connections 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.862+0000 7fb10d816b80 1 -- 172.21.15.31:0/668390982 wait complete. 2023-10-06T21:30:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.862+0000 7fb10d816b80 1 -- start start 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 Processor -- start 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687bc00 0x556b36886df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b36884710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b367cfdd0 con 0x556b368873d0 2023-10-06T21:30:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3676c6b0 con 0x556b3687bc00 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3676c350 con 0x556b3687c520 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.863+0000 7fb10d816b80 10 client.0 _ll_register_callbacks cb 0x556b365c97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b36884710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687bc00 0x556b36886df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60202/0 (socket says 172.21.15.31:60202) 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/3282080971 learned_addr learned my addr 172.21.15.31:0/3282080971 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687bc00 msgr2=0x556b36886df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687bc00 0x556b36886df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/3282080971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 msgr2=0x556b36884710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b36884710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 -- 172.21.15.31:0/3282080971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f410e5e0 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c5b0 tx=0x7fb0e8005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e321d308f6c856b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.864+0000 7fb0d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8016540 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b3686fd50 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb0e809a900 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e809a200 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 10 client.26644 fetch_fsmap learned FSMap version 35 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556b365e3c00 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556b3686fd50 con 0x556b3687c520 2023-10-06T21:30:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 35) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb0e80996f0 con 0x556b3687c520 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 10 client.26644 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 10 client.26644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 20 client.26644 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b3689f640 0x556b368a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.865+0000 7fb10d816b80 4 client.26644 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.866+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b367c8db0 con 0x556b3689f640 2023-10-06T21:30:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.866+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b3689f640 0x556b368a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.867+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b3689f640 0x556b368a1a60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fccc35f59501ab server_cookie=c0a232dcaeecdf52 in_seq=0 out_seq=0 2023-10-06T21:30:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.867+0000 7fb0d2ffd700 10 client.26644 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.867+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/3282080971 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b367c8db0 con 0x556b3689f640 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.867+0000 7fb0d2ffd700 10 client.26644 handle_command_reply: tid=0 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.867+0000 7fb10d816b80 1 client.26644 shutdown 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 msgr2=0x556b36884070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c5b0 tx=0x7fb0e8005710 comp rx=0 tx=0).stop 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 shutdown_connections 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b3689f640 0x556b368a1a60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b36884710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb0d2ffd700 0 client.26644 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687bc00 0x556b36886df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3282080971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687c520 0x556b36884070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 >> 172.21.15.31:0/3282080971 conn(0x556b367f1200 msgr2=0x556b36877c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 shutdown_connections 2023-10-06T21:30:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 1 -- 172.21.15.31:0/3282080971 wait complete. 2023-10-06T21:30:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.868+0000 7fb10d816b80 20 client.26644 trim_cache size 0 max 16384 2023-10-06T21:30:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 Processor -- start 2023-10-06T21:30:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 -- start start 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687bc00 0x556b36884690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36896710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b366508c0 con 0x556b368873d0 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3689cd00 con 0x556b3687c520 2023-10-06T21:30:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.885+0000 7fb10d816b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b36878dc0 con 0x556b3687bc00 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56140/0 (socket says 172.21.15.31:56140) 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 -- 172.21.15.31:0/3163151834 learned_addr learned my addr 172.21.15.31:0/3163151834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36896710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687bc00 0x556b36884690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 -- 172.21.15.31:0/3163151834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687bc00 msgr2=0x556b36884690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687bc00 0x556b36884690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 -- 172.21.15.31:0/3163151834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 msgr2=0x556b36896710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36896710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f3fff700 1 -- 172.21.15.31:0/3163151834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b366211f0 con 0x556b368873d0 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.886+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36896710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.887+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b0a0 tx=0x7fb0ec009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29efbb5ccc2d2c8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.887+0000 7fb0da7fc700 1 -- 172.21.15.31:0/3163151834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e040 con 0x556b368873d0 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.887+0000 7fb0da7fc700 1 -- 172.21.15.31:0/3163151834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0ec007de0 con 0x556b368873d0 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.887+0000 7fb0da7fc700 1 -- 172.21.15.31:0/3163151834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec005340 con 0x556b368873d0 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- 172.21.15.31:0/3163151834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 msgr2=0x556b3689b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b0a0 tx=0x7fb0ec009420 comp rx=0 tx=0).stop 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- 172.21.15.31:0/3163151834 shutdown_connections 2023-10-06T21:30:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b368873d0 0x556b3689b8c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b3687bc00 0x556b36884690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 --2- 172.21.15.31:0/3163151834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36896710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- 172.21.15.31:0/3163151834 >> 172.21.15.31:0/3163151834 conn(0x556b367f1200 msgr2=0x556b36877250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- 172.21.15.31:0/3163151834 shutdown_connections 2023-10-06T21:30:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- 172.21.15.31:0/3163151834 wait complete. 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.888+0000 7fb10d816b80 1 -- start start 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 Processor -- start 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36894110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b368873d0 0x556b36894650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b368f2470 con 0x556b3687bc00 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b368f25b0 con 0x556b3687c520 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b368f2740 con 0x556b368873d0 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.889+0000 7fb10d816b80 10 client.0 _ll_register_callbacks cb 0x7fb0e8099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56152/0 (socket says 172.21.15.31:56152) 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 -- 172.21.15.31:0/2695042322 learned_addr learned my addr 172.21.15.31:0/2695042322 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36894110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f3fff700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b368873d0 0x556b36894650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 -- 172.21.15.31:0/2695042322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b368873d0 msgr2=0x556b36894650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b368873d0 0x556b36894650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 -- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 msgr2=0x556b36894110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36894110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 -- 172.21.15.31:0/2695042322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b365e3c00 con 0x556b3687bc00 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0f37fe700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009c20 tx=0x7fb0e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0268f634a6960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.890+0000 7fb0d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4024040 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0ec006060 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb0e4018070 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e4012950 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb10d816b80 10 client.16983 fetch_fsmap learned FSMap version 35 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556b365fad80 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.891+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0ec006060 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 35) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb0e4039aa0 con 0x556b3687bc00 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 10 client.16983 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 20 client.16983 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b368f3f10 0x556b368f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 4 client.16983 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:30:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b367c8ad0 con 0x556b368f3f10 2023-10-06T21:30:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b368f3f10 0x556b368f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb0f2ffd700 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b368f3f10 0x556b368f6330 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23ff7f8c264dd02 server_cookie=446d3552a9dbee22 in_seq=0 out_seq=0 2023-10-06T21:30:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.892+0000 7fb0d2ffd700 10 client.16983 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.923+0000 7fb0d2ffd700 1 -- 172.21.15.31:0/2695042322 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b367c8ad0 con 0x556b368f3f10 2023-10-06T21:30:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb0d2ffd700 10 client.16983 handle_command_reply: tid=0 2023-10-06T21:30:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 client.16983 shutdown 2023-10-06T21:30:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 msgr2=0x556b36893bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009c20 tx=0x7fb0e40047c0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 shutdown_connections 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b368873d0 0x556b36894650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b368f3f10 0x556b368f6330 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b3687bc00 0x556b36893bd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 --2- 172.21.15.31:0/2695042322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b3687c520 0x556b36894110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb0d2ffd700 0 client.16983 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 >> 172.21.15.31:0/2695042322 conn(0x556b367f1200 msgr2=0x556b3689b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 shutdown_connections 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 1 -- 172.21.15.31:0/2695042322 wait complete. 2023-10-06T21:30:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.924+0000 7fb10d816b80 20 client.16983 trim_cache size 0 max 16384 2023-10-06T21:30:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:30:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:30:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:30:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 msgr2=0x7fb0dc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e40 tx=0x7fb0e4009260 comp rx=0 tx=0).stop 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 msgr2=0x7fb0f4197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e970 tx=0x7fb0e800ec80 comp rx=0 tx=0).stop 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 shutdown_connections 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0f419bc90 0x7fb0f419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0f41010d0 0x7fb0f41970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 --2- 172.21.15.31:0/1042669264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41061f0 0x7fb0f4197620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 >> 172.21.15.31:0/1042669264 conn(0x7fb0f40fc950 msgr2=0x7fb0f40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 shutdown_connections 2023-10-06T21:30:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:28.925+0000 7fb0f9dff700 1 -- 172.21.15.31:0/1042669264 wait complete. 2023-10-06T21:30:29.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 160926 1.71 MB/sec execute 379 sec latency 22.794 ms 2023-10-06T21:30:30.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 161286 1.71 MB/sec execute 380 sec latency 26.479 ms 2023-10-06T21:30:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:30 smithi142 ceph-mon[102951]: pgmap v298: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 497 KiB/s wr, 66 op/s 2023-10-06T21:30:30.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:30 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:30 smithi031 ceph-mon[98539]: pgmap v298: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 497 KiB/s wr, 66 op/s 2023-10-06T21:30:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:30 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:30 smithi146 ceph-mon[102809]: pgmap v298: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 497 KiB/s wr, 66 op/s 2023-10-06T21:30:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:30 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:31.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 161641 1.71 MB/sec execute 381 sec latency 25.974 ms 2023-10-06T21:30:32.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 162012 1.71 MB/sec execute 382 sec latency 30.741 ms 2023-10-06T21:30:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:32 smithi142 ceph-mon[102951]: pgmap v299: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 314 KiB/s wr, 40 op/s 2023-10-06T21:30:32.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:32 smithi031 ceph-mon[98539]: pgmap v299: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 314 KiB/s wr, 40 op/s 2023-10-06T21:30:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:32 smithi146 ceph-mon[102809]: pgmap v299: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 314 KiB/s wr, 40 op/s 2023-10-06T21:30:33.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 162386 1.71 MB/sec execute 383 sec latency 18.051 ms 2023-10-06T21:30:34.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 162918 1.72 MB/sec execute 384 sec latency 20.241 ms 2023-10-06T21:30:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:34 smithi142 ceph-mon[102951]: pgmap v300: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 740 KiB/s wr, 86 op/s 2023-10-06T21:30:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:34 smithi031 ceph-mon[98539]: pgmap v300: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 740 KiB/s wr, 86 op/s 2023-10-06T21:30:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:34 smithi146 ceph-mon[102809]: pgmap v300: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 740 KiB/s wr, 86 op/s 2023-10-06T21:30:35.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163152 1.72 MB/sec execute 385 sec latency 23.120 ms 2023-10-06T21:30:35.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:35 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:35 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:35 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:36.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163256 1.72 MB/sec execute 386 sec latency 698.357 ms 2023-10-06T21:30:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:36 smithi142 ceph-mon[102951]: pgmap v301: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:36 smithi031 ceph-mon[98539]: pgmap v301: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:36 smithi146 ceph-mon[102809]: pgmap v301: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:37.353 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163256 1.71 MB/sec execute 387 sec latency 1698.480 ms 2023-10-06T21:30:38.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163256 1.71 MB/sec execute 388 sec latency 2698.638 ms 2023-10-06T21:30:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:38 smithi142 ceph-mon[102951]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:38 smithi031 ceph-mon[98539]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:38 smithi146 ceph-mon[102809]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 708 KiB/s wr, 76 op/s 2023-10-06T21:30:39.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163256 1.70 MB/sec execute 389 sec latency 3698.770 ms 2023-10-06T21:30:40.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163274 1.70 MB/sec execute 390 sec latency 3801.990 ms 2023-10-06T21:30:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:40 smithi142 ceph-mon[102951]: pgmap v303: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1018 KiB/s wr, 101 op/s 2023-10-06T21:30:40.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:40 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:40 smithi031 ceph-mon[98539]: pgmap v303: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1018 KiB/s wr, 101 op/s 2023-10-06T21:30:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:40 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:40 smithi146 ceph-mon[102809]: pgmap v303: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 1018 KiB/s wr, 101 op/s 2023-10-06T21:30:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:40 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:41.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163274 1.70 MB/sec execute 391 sec latency 1738.513 ms 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.088+0000 7fe5345ff700 1 -- 172.21.15.31:0/1359459151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c101a80 msgr2=0x7fe52c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.088+0000 7fe5345ff700 1 --2- 172.21.15.31:0/1359459151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c101a80 0x7fe52c105ac0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009990 tx=0x7fe51c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 -- 172.21.15.31:0/1359459151 shutdown_connections 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 --2- 172.21.15.31:0/1359459151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c1061f0 0x7fe52c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 --2- 172.21.15.31:0/1359459151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1010d0 0x7fe52c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 --2- 172.21.15.31:0/1359459151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c101a80 0x7fe52c105ac0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 -- 172.21.15.31:0/1359459151 >> 172.21.15.31:0/1359459151 conn(0x7fe52c0fc950 msgr2=0x7fe52c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 -- 172.21.15.31:0/1359459151 shutdown_connections 2023-10-06T21:30:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.089+0000 7fe5345ff700 1 -- 172.21.15.31:0/1359459151 wait complete. 2023-10-06T21:30:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 Processor -- start 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 -- start start 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 0x7fe52c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c19dcc0 0x7fe52c19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19e740 con 0x7fe52c1010d0 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19e8b0 con 0x7fe52c19dcc0 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe5345ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19ea70 con 0x7fe52c1061f0 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.090+0000 7fe53239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41994/0 (socket says 172.21.15.31:41994) 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 -- 172.21.15.31:0/625198396 learned_addr learned my addr 172.21.15.31:0/625198396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe532b9c700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c19dcc0 0x7fe52c19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 -- 172.21.15.31:0/625198396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 msgr2=0x7fe52c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe531b9a700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 0x7fe52c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 0x7fe52c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 -- 172.21.15.31:0/625198396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c19dcc0 msgr2=0x7fe52c19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c19dcc0 0x7fe52c19e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 -- 172.21.15.31:0/625198396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0095f0 con 0x7fe52c1010d0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe531b9a700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 0x7fe52c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe53239b700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d950 tx=0x7fe52800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.091+0000 7fe5237fe700 1 -- 172.21.15.31:0/625198396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528009b40 con 0x7fe52c1010d0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.092+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c1a5fd0 con 0x7fe52c1010d0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.092+0000 7fe5237fe700 1 -- 172.21.15.31:0/625198396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe528009ca0 con 0x7fe52c1010d0 2023-10-06T21:30:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.092+0000 7fe5237fe700 1 -- 172.21.15.31:0/625198396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52800f620 con 0x7fe52c1010d0 2023-10-06T21:30:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.092+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c1a6450 con 0x7fe52c1010d0 2023-10-06T21:30:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.095+0000 7fe5237fe700 1 -- 172.21.15.31:0/625198396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe528010460 con 0x7fe52c1010d0 2023-10-06T21:30:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.096+0000 7fe547f48b80 1 Processor -- start 2023-10-06T21:30:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe5237fe700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 0x7fe5180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 -- start start 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe5237fe700 1 -- 172.21.15.31:0/625198396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe5280cd530 con 0x7fe52c1010d0 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe532b9c700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 0x7fe5180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3954ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd3955d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd394b9400 con 0x55fd39550520 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd394aabc0 con 0x55fd3955b3d0 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.097+0000 7fe547f48b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd394a84e0 con 0x55fd3954fc00 2023-10-06T21:30:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42002/0 (socket says 172.21.15.31:42002) 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 -- 172.21.15.31:0/1113828240 learned_addr learned my addr 172.21.15.31:0/1113828240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe53239b700 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3954ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe532b9c700 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd3955d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe532b9c700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 0x7fe5180a3ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5240093c0 tx=0x7fe524009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 -- 172.21.15.31:0/1113828240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 msgr2=0x55fd3954ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3954ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 -- 172.21.15.31:0/1113828240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 msgr2=0x55fd3955d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd3955d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 -- 172.21.15.31:0/1113828240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd392ced80 con 0x55fd39550520 2023-10-06T21:30:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.098+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005e80 tx=0x7fe51c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b251ada8577c304f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe520bfe700 1 -- 172.21.15.31:0/1113828240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01e070 con 0x55fd39550520 2023-10-06T21:30:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe520bfe700 1 -- 172.21.15.31:0/1113828240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe51c0054d0 con 0x55fd39550520 2023-10-06T21:30:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe520bfe700 1 -- 172.21.15.31:0/1113828240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004740 con 0x55fd39550520 2023-10-06T21:30:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 -- 172.21.15.31:0/1113828240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 msgr2=0x55fd3955ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005e80 tx=0x7fe51c005eb0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 -- 172.21.15.31:0/1113828240 shutdown_connections 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd3955d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3954ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1113828240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ae90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 -- 172.21.15.31:0/1113828240 >> 172.21.15.31:0/1113828240 conn(0x55fd394c5200 msgr2=0x55fd3954c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.099+0000 7fe547f48b80 1 -- 172.21.15.31:0/1113828240 shutdown_connections 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.100+0000 7fe547f48b80 1 -- 172.21.15.31:0/1113828240 wait complete. 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.100+0000 7fe547f48b80 1 -- start start 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.100+0000 7fe547f48b80 1 Processor -- start 2023-10-06T21:30:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3955ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd39557fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd394a3dd0 con 0x55fd3955b3d0 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd394406b0 con 0x55fd39550520 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39440350 con 0x55fd3954fc00 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe547f48b80 10 client.0 _ll_register_callbacks cb 0x55fd3929d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe532b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe531b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd39557fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe532b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42010/0 (socket says 172.21.15.31:42010) 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe532b9c700 1 -- 172.21.15.31:0/1382799420 learned_addr learned my addr 172.21.15.31:0/1382799420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.101+0000 7fe53239b700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3955ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 -- 172.21.15.31:0/1382799420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 msgr2=0x55fd3955ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3955ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 -- 172.21.15.31:0/1382799420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 msgr2=0x55fd39558600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 -- 172.21.15.31:0/1382799420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c10e5e0 con 0x55fd39550520 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe532b9c700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe531b9a700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd39557fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe51c024040 tx=0x7fe51c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cafe5aa2d89c4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01e040 con 0x55fd39550520 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd39543d50 con 0x55fd39550520 2023-10-06T21:30:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.102+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe51c005640 con 0x55fd39550520 2023-10-06T21:30:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51c004910 con 0x55fd39550520 2023-10-06T21:30:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 10 client.26711 fetch_fsmap learned FSMap version 38 2023-10-06T21:30:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fd392b7c00 con 0x55fd39550520 2023-10-06T21:30:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd39543d50 con 0x55fd39550520 2023-10-06T21:30:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fe51c02bb50 con 0x55fd39550520 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 10 client.26711 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 20 client.26711 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd3956aa40 0x55fd3956ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 4 client.26711 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.103+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd3949cdb0 con 0x55fd3956aa40 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.104+0000 7fe53239b700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd3956aa40 0x55fd3956ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.104+0000 7fe515ffb700 10 client.26711 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.104+0000 7fe53239b700 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd3956aa40 0x55fd3956ce60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556744b208024fa7 server_cookie=63829326150ca94d in_seq=0 out_seq=0 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.105+0000 7fe515ffb700 1 -- 172.21.15.31:0/1382799420 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd3949cdb0 con 0x55fd3956aa40 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.105+0000 7fe515ffb700 10 client.26711 handle_command_reply: tid=0 2023-10-06T21:30:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.105+0000 7fe547f48b80 1 client.26711 shutdown 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 msgr2=0x55fd39557fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd39557fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe51c024040 tx=0x7fe51c00bd30 comp rx=0 tx=0).stop 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 shutdown_connections 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd3956aa40 0x55fd3956ce60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3955b3d0 0x55fd39558600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd3955ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe515ffb700 0 client.26711 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 --2- 172.21.15.31:0/1382799420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd39557fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 >> 172.21.15.31:0/1382799420 conn(0x55fd394c5200 msgr2=0x55fd3954bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 shutdown_connections 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 1 -- 172.21.15.31:0/1382799420 wait complete. 2023-10-06T21:30:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.106+0000 7fe547f48b80 20 client.26711 trim_cache size 0 max 16384 2023-10-06T21:30:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.121+0000 7fe547f48b80 1 Processor -- start 2023-10-06T21:30:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.121+0000 7fe547f48b80 1 -- start start 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd395532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3955b3d0 0x55fd39553830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39550c60 con 0x55fd3954fc00 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd393248c0 con 0x55fd39550520 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe547f48b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39571bc0 con 0x55fd3955b3d0 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42014/0 (socket says 172.21.15.31:42014) 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 -- 172.21.15.31:0/3070490536 learned_addr learned my addr 172.21.15.31:0/3070490536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe531b9a700 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd395532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 -- 172.21.15.31:0/3070490536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3955b3d0 msgr2=0x55fd39553830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3955b3d0 0x55fd39553830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 -- 172.21.15.31:0/3070490536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 msgr2=0x55fd395532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd395532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe53239b700 1 -- 172.21.15.31:0/3070490536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd392f51f0 con 0x55fd3954fc00 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.122+0000 7fe531b9a700 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd395532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe53239b700 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe528009520 tx=0x7fe52800dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481a1e75b5243f05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe5213ff700 1 -- 172.21.15.31:0/3070490536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52809e590 con 0x55fd3954fc00 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe5213ff700 1 -- 172.21.15.31:0/3070490536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5280a27d0 con 0x55fd3954fc00 2023-10-06T21:30:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe5213ff700 1 -- 172.21.15.31:0/3070490536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52809e150 con 0x55fd3954fc00 2023-10-06T21:30:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe547f48b80 1 -- 172.21.15.31:0/3070490536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 msgr2=0x55fd395665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe547f48b80 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe528009520 tx=0x7fe52800dfa0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.123+0000 7fe547f48b80 1 -- 172.21.15.31:0/3070490536 shutdown_connections 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3955b3d0 0x55fd39553830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd3954fc00 0x55fd395665b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 --2- 172.21.15.31:0/3070490536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd39550520 0x55fd395532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 -- 172.21.15.31:0/3070490536 >> 172.21.15.31:0/3070490536 conn(0x55fd394c5200 msgr2=0x55fd3954a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 -- 172.21.15.31:0/3070490536 shutdown_connections 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 -- 172.21.15.31:0/3070490536 wait complete. 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 -- start start 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.124+0000 7fe547f48b80 1 Processor -- start 2023-10-06T21:30:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd39561c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd39560440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39560980 con 0x55fd39550520 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39560af0 con 0x55fd3955b3d0 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39560cb0 con 0x55fd3954fc00 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe547f48b80 10 client.0 _ll_register_callbacks cb 0x7fe5240091b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42020/0 (socket says 172.21.15.31:42020) 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 -- 172.21.15.31:0/515116113 learned_addr learned my addr 172.21.15.31:0/515116113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe53239b700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd39561c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe532b9c700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd39560440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 -- 172.21.15.31:0/515116113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 msgr2=0x55fd39561c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd39561c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 -- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 msgr2=0x55fd39560440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd39560440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe531b9a700 1 -- 172.21.15.31:0/515116113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c1a6450 con 0x55fd39550520 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.125+0000 7fe532b9c700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd39560440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe531b9a700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009990 tx=0x7fe51c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0048fe7b70acb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01e030 con 0x55fd39550520 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe51c0054d0 con 0x55fd39550520 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe51c000e40 con 0x55fd39550520 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51c00e3e0 con 0x55fd39550520 2023-10-06T21:30:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe547f48b80 10 client.17001 fetch_fsmap learned FSMap version 38 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fd392cd250 con 0x55fd39550520 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.126+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe51c049020 con 0x55fd39550520 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fe51c00fc20 con 0x55fd39550520 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 20 client.17001 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd395bf9b0 0x55fd395c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 4 client.17001 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd3949cad0 con 0x55fd395bf9b0 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.127+0000 7fe53239b700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd395bf9b0 0x55fd395c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.128+0000 7fe53239b700 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd395bf9b0 0x55fd395c1dd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af0237bcb93238a server_cookie=81b22cbdaa6837db in_seq=0 out_seq=0 2023-10-06T21:30:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.128+0000 7fe515ffb700 10 client.17001 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.163+0000 7fe515ffb700 1 -- 172.21.15.31:0/515116113 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd3949cad0 con 0x55fd395bf9b0 2023-10-06T21:30:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.163+0000 7fe515ffb700 10 client.17001 handle_command_reply: tid=0 2023-10-06T21:30:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.163+0000 7fe547f48b80 1 client.17001 shutdown 2023-10-06T21:30:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 msgr2=0x55fd3955ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009990 tx=0x7fe51c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 shutdown_connections 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd3955b3d0 0x55fd39560440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd3954fc00 0x55fd39561c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55fd395bf9b0 0x55fd395c1dd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 --2- 172.21.15.31:0/515116113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd39550520 0x55fd3955ff00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe515ffb700 0 client.17001 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 >> 172.21.15.31:0/515116113 conn(0x55fd394c5200 msgr2=0x55fd3954b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 shutdown_connections 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 1 -- 172.21.15.31:0/515116113 wait complete. 2023-10-06T21:30:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.164+0000 7fe547f48b80 20 client.17001 trim_cache size 0 max 16384 2023-10-06T21:30:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:30:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:30:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:30:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 msgr2=0x7fe5180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 0x7fe5180a3ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5240093c0 tx=0x7fe524009920 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 msgr2=0x7fe52c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d950 tx=0x7fe52800dd10 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 shutdown_connections 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c19dcc0 0x7fe52c19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe5180a15f0 0x7fe5180a3ab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe52c1010d0 0x7fe52c199110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 --2- 172.21.15.31:0/625198396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe52c1061f0 0x7fe52c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 >> 172.21.15.31:0/625198396 conn(0x7fe52c0fc950 msgr2=0x7fe52c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 shutdown_connections 2023-10-06T21:30:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:42.165+0000 7fe5345ff700 1 -- 172.21.15.31:0/625198396 wait complete. 2023-10-06T21:30:42.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163274 1.69 MB/sec execute 392 sec latency 2738.604 ms 2023-10-06T21:30:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:42 smithi142 ceph-mon[102951]: pgmap v304: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 736 KiB/s wr, 69 op/s 2023-10-06T21:30:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:42 smithi031 ceph-mon[98539]: pgmap v304: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 736 KiB/s wr, 69 op/s 2023-10-06T21:30:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:42 smithi146 ceph-mon[102809]: pgmap v304: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 736 KiB/s wr, 69 op/s 2023-10-06T21:30:43.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163274 1.69 MB/sec execute 393 sec latency 3738.733 ms 2023-10-06T21:30:44.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163323 1.68 MB/sec execute 394 sec latency 4256.926 ms 2023-10-06T21:30:44.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:44 smithi142 ceph-mon[102951]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 782 KiB/s wr, 81 op/s 2023-10-06T21:30:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:44 smithi031 ceph-mon[98539]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 782 KiB/s wr, 81 op/s 2023-10-06T21:30:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:44 smithi146 ceph-mon[102809]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 782 KiB/s wr, 81 op/s 2023-10-06T21:30:45.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163495 1.68 MB/sec execute 395 sec latency 348.681 ms 2023-10-06T21:30:46.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163665 1.68 MB/sec execute 396 sec latency 17.707 ms 2023-10-06T21:30:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:46 smithi031 ceph-mon[98539]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:46 smithi146 ceph-mon[102809]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:46.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:46 smithi142 ceph-mon[102951]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:47.354 INFO:tasks.workunit.client.0.smithi031.stdout: 1 163939 1.67 MB/sec execute 397 sec latency 22.936 ms 2023-10-06T21:30:48.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 164292 1.67 MB/sec execute 398 sec latency 19.951 ms 2023-10-06T21:30:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:48 smithi031 ceph-mon[98539]: pgmap v307: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:48 smithi146 ceph-mon[102809]: pgmap v307: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:48.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:48 smithi142 ceph-mon[102951]: pgmap v307: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 356 KiB/s wr, 36 op/s 2023-10-06T21:30:49.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 164658 1.68 MB/sec execute 399 sec latency 24.574 ms 2023-10-06T21:30:50.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:49 smithi031 ceph-mon[98539]: pgmap v308: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 679 KiB/s wr, 73 op/s 2023-10-06T21:30:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:49 smithi146 ceph-mon[102809]: pgmap v308: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 679 KiB/s wr, 73 op/s 2023-10-06T21:30:50.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 164990 1.67 MB/sec execute 400 sec latency 28.389 ms 2023-10-06T21:30:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:49 smithi142 ceph-mon[102951]: pgmap v308: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 679 KiB/s wr, 73 op/s 2023-10-06T21:30:51.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 165354 1.67 MB/sec execute 401 sec latency 26.712 ms 2023-10-06T21:30:52.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 165651 1.67 MB/sec execute 402 sec latency 20.847 ms 2023-10-06T21:30:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:52 smithi142 ceph-mon[102951]: pgmap v309: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 368 KiB/s wr, 49 op/s 2023-10-06T21:30:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:52 smithi031 ceph-mon[98539]: pgmap v309: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 368 KiB/s wr, 49 op/s 2023-10-06T21:30:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:52 smithi146 ceph-mon[102809]: pgmap v309: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 368 KiB/s wr, 49 op/s 2023-10-06T21:30:53.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166145 1.68 MB/sec execute 403 sec latency 20.106 ms 2023-10-06T21:30:54.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166503 1.68 MB/sec execute 404 sec latency 17.573 ms 2023-10-06T21:30:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:54 smithi142 ceph-mon[102951]: pgmap v310: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 770 KiB/s wr, 89 op/s 2023-10-06T21:30:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:54 smithi031 ceph-mon[98539]: pgmap v310: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 770 KiB/s wr, 89 op/s 2023-10-06T21:30:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:54 smithi146 ceph-mon[102809]: pgmap v310: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 770 KiB/s wr, 89 op/s 2023-10-06T21:30:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 -- 172.21.15.31:0/1839815276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c10db20 msgr2=0x7fd57c10ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 --2- 172.21.15.31:0/1839815276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c10db20 0x7fd57c10ff10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd57400b0a0 tx=0x7fd57400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 -- 172.21.15.31:0/1839815276 shutdown_connections 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 --2- 172.21.15.31:0/1839815276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c10db20 0x7fd57c10ff10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 --2- 172.21.15.31:0/1839815276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c0682d0 0x7fd57c0686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 --2- 172.21.15.31:0/1839815276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57c068bf0 0x7fd57c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 -- 172.21.15.31:0/1839815276 >> 172.21.15.31:0/1839815276 conn(0x7fd57c076b70 msgr2=0x7fd57c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 -- 172.21.15.31:0/1839815276 shutdown_connections 2023-10-06T21:30:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.315+0000 7fd5831ff700 1 -- 172.21.15.31:0/1839815276 wait complete. 2023-10-06T21:30:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.316+0000 7fd5831ff700 1 Processor -- start 2023-10-06T21:30:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.316+0000 7fd5831ff700 1 -- start start 2023-10-06T21:30:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c068bf0 0x7fd57c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57c10db20 0x7fd57c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19e3d0 con 0x7fd57c0682d0 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19e540 con 0x7fd57c068bf0 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd5831ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19e700 con 0x7fd57c10db20 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54256/0 (socket says 172.21.15.31:54256) 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 -- 172.21.15.31:0/3914539443 learned_addr learned my addr 172.21.15.31:0/3914539443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd57bfff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c068bf0 0x7fd57c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 -- 172.21.15.31:0/3914539443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57c10db20 msgr2=0x7fd57c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57c10db20 0x7fd57c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 -- 172.21.15.31:0/3914539443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c068bf0 msgr2=0x7fd57c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c068bf0 0x7fd57c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.317+0000 7fd580f9b700 1 -- 172.21.15.31:0/3914539443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574009d00 con 0x7fd57c0682d0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd580f9b700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9a0 tx=0x7fd57000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd579ffb700 1 -- 172.21.15.31:0/3914539443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570009790 con 0x7fd57c0682d0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd579ffb700 1 -- 172.21.15.31:0/3914539443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd57000cbd0 con 0x7fd57c0682d0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd579ffb700 1 -- 172.21.15.31:0/3914539443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570018680 con 0x7fd57c0682d0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c19e9e0 con 0x7fd57c0682d0 2023-10-06T21:30:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.318+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1a6220 con 0x7fd57c0682d0 2023-10-06T21:30:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.321+0000 7fd579ffb700 1 -- 172.21.15.31:0/3914539443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd57000cd40 con 0x7fd57c0682d0 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.322+0000 7fd596b38b80 1 Processor -- start 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.322+0000 7fd579ffb700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 0x7fd56c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 -- start start 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd579ffb700 1 -- 172.21.15.31:0/3914539443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd570014070 con 0x7fd57c0682d0 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd58179c700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 0x7fd56c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecb02e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecb057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eca61400 con 0x5574ecb033d0 2023-10-06T21:30:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eca52bc0 con 0x5574ecaf7c00 2023-10-06T21:30:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd596b38b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eca504e0 con 0x5574ecaf8520 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd580f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd580f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58404/0 (socket says 172.21.15.31:58404) 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd580f9b700 1 -- 172.21.15.31:0/382181960 learned_addr learned my addr 172.21.15.31:0/382181960 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.323+0000 7fd58179c700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecb057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd57bfff700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecb02e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 -- 172.21.15.31:0/382181960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 msgr2=0x5574ecb02e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecb02e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 -- 172.21.15.31:0/382181960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 msgr2=0x5574ecb057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecb057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 -- 172.21.15.31:0/382181960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ec876d80 con 0x5574ecaf7c00 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd57bfff700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecb02e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd580f9b700 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd570096de0 tx=0x7fd570099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cf9c0cb6de7b34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd58179c700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 0x7fd56c0a3ab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd57400d010 tx=0x7fd574007d80 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd566ffd700 1 -- 172.21.15.31:0/382181960 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57009a370 con 0x5574ecaf7c00 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd566ffd700 1 -- 172.21.15.31:0/382181960 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd57009e4c0 con 0x5574ecaf7c00 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.324+0000 7fd566ffd700 1 -- 172.21.15.31:0/382181960 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57009a880 con 0x5574ecaf7c00 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 -- 172.21.15.31:0/382181960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 msgr2=0x5574ecaf7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd570096de0 tx=0x7fd570099cd0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 -- 172.21.15.31:0/382181960 shutdown_connections 2023-10-06T21:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecb057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecaf7fe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 --2- 172.21.15.31:0/382181960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecb02e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 -- 172.21.15.31:0/382181960 >> 172.21.15.31:0/382181960 conn(0x5574eca6d200 msgr2=0x5574ecaf4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 -- 172.21.15.31:0/382181960 shutdown_connections 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.325+0000 7fd596b38b80 1 -- 172.21.15.31:0/382181960 wait complete. 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 -- start start 2023-10-06T21:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 Processor -- start 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb02de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecb00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eca4bdd0 con 0x5574ecaf8520 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ec9e86b0 con 0x5574ecaf7c00 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.326+0000 7fd596b38b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ec9e8350 con 0x5574ecb033d0 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd596b38b80 10 client.0 _ll_register_callbacks cb 0x5574ec8457a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54272/0 (socket says 172.21.15.31:54272) 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 -- 172.21.15.31:0/954459400 learned_addr learned my addr 172.21.15.31:0/954459400 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd580f9b700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb02de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd58179c700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecb00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 -- 172.21.15.31:0/954459400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 msgr2=0x5574ecb00700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecb00700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 -- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 msgr2=0x5574ecb02de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb02de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd57bfff700 1 -- 172.21.15.31:0/954459400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1a6220 con 0x5574ecaf8520 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.327+0000 7fd58179c700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecb00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd57bfff700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd5680126d0 tx=0x7fd5680129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b7238c5223f5c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd55bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568004460 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574ecaebd50 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd568003680 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd568003840 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd596b38b80 10 client.17013 fetch_fsmap learned FSMap version 39 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574ec85fc00 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.328+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5574ecaebd50 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd568010af0 con 0x5574ecaf8520 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:30:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 20 client.17013 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb12f60 0x5574ecb15380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 4 client.17013 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574eca44db0 con 0x5574ecb12f60 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.329+0000 7fd580f9b700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb12f60 0x5574ecb15380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.330+0000 7fd55bfff700 10 client.17013 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.330+0000 7fd580f9b700 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb12f60 0x5574ecb15380 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c2075681dab8b7 server_cookie=26105908de49c04c in_seq=0 out_seq=0 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.330+0000 7fd55bfff700 1 -- 172.21.15.31:0/954459400 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574eca44db0 con 0x5574ecb12f60 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.330+0000 7fd55bfff700 10 client.17013 handle_command_reply: tid=0 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.330+0000 7fd596b38b80 1 client.17013 shutdown 2023-10-06T21:30:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 msgr2=0x5574ecb00060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd5680126d0 tx=0x7fd5680129e0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 shutdown_connections 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecb00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb02de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb12f60 0x5574ecb15380 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 --2- 172.21.15.31:0/954459400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecb00060 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd55bfff700 0 client.17013 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 >> 172.21.15.31:0/954459400 conn(0x5574eca6d200 msgr2=0x5574ecaf3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 shutdown_connections 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 1 -- 172.21.15.31:0/954459400 wait complete. 2023-10-06T21:30:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.331+0000 7fd596b38b80 20 client.17013 trim_cache size 0 max 16384 2023-10-06T21:30:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.346+0000 7fd596b38b80 1 Processor -- start 2023-10-06T21:30:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.346+0000 7fd596b38b80 1 -- start start 2023-10-06T21:30:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecaff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ec8cc8c0 con 0x5574ecb033d0 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ecb0a450 con 0x5574ecaf7c00 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd596b38b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ecaf53c0 con 0x5574ecaf8520 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54276/0 (socket says 172.21.15.31:54276) 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 -- 172.21.15.31:0/1987380509 learned_addr learned my addr 172.21.15.31:0/1987380509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd580f9b700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd57bfff700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecaff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 -- 172.21.15.31:0/1987380509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 msgr2=0x5574ecaff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecaff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 -- 172.21.15.31:0/1987380509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 msgr2=0x5574ecb09f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb09f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd58179c700 1 -- 172.21.15.31:0/1987380509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ec89d1f0 con 0x5574ecb033d0 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd580f9b700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb09f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.347+0000 7fd57bfff700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecaff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd58179c700 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd57c073e30 tx=0x7fd574024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ce6e3ffc184d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd5677fe700 1 -- 172.21.15.31:0/1987380509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57402e070 con 0x5574ecb033d0 2023-10-06T21:30:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd5677fe700 1 -- 172.21.15.31:0/1987380509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd574003e90 con 0x5574ecb033d0 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd5677fe700 1 -- 172.21.15.31:0/1987380509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400e070 con 0x5574ecb033d0 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd596b38b80 1 -- 172.21.15.31:0/1987380509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 msgr2=0x5574ecaffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.348+0000 7fd596b38b80 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd57c073e30 tx=0x7fd574024000 comp rx=0 tx=0).stop 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 -- 172.21.15.31:0/1987380509 shutdown_connections 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecb033d0 0x5574ecaffd60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecb09f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 --2- 172.21.15.31:0/1987380509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecaf8520 0x5574ecaff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 -- 172.21.15.31:0/1987380509 >> 172.21.15.31:0/1987380509 conn(0x5574eca6d200 msgr2=0x5574ecafb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 -- 172.21.15.31:0/1987380509 shutdown_connections 2023-10-06T21:30:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 -- 172.21.15.31:0/1987380509 wait complete. 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 -- start start 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.349+0000 7fd596b38b80 1 Processor -- start 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecafd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ecafcd60 con 0x5574ecaf8520 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ecafced0 con 0x5574ecaf7c00 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ecafd090 con 0x5574ecb033d0 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd596b38b80 10 client.0 _ll_register_callbacks cb 0x7fd568009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36562/0 (socket says 172.21.15.31:36562) 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54290/0 (socket says 172.21.15.31:54290) 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 -- 172.21.15.31:0/630162329 learned_addr learned my addr 172.21.15.31:0/630162329 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd580f9b700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecafd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 -- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 msgr2=0x5574ecafd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecafd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 -- 172.21.15.31:0/630162329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 msgr2=0x5574ecafc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd58179c700 1 -- 172.21.15.31:0/630162329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1107d0 con 0x5574ecb033d0 2023-10-06T21:30:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.350+0000 7fd57bfff700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd58179c700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd57400b880 tx=0x7fd57401f6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ef85a57ea3acd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd55bfff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400e030 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd57009e4c0 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd57401fd90 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57402e070 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd596b38b80 10 client.26674 fetch_fsmap learned FSMap version 39 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.351+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574ec875250 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd57009e4c0 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd574019050 con 0x5574ecb033d0 2023-10-06T21:30:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 10 client.26674 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:30:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 20 client.26674 populate_metadata read hostname 'smithi031' 2023-10-06T21:30:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb67e00 0x5574ecb6a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 4 client.26674 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:30:55.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166799 1.68 MB/sec execute 405 sec latency 138.835 ms 2023-10-06T21:30:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5574eca44ad0 con 0x5574ecb67e00 2023-10-06T21:30:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.352+0000 7fd580f9b700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb67e00 0x5574ecb6a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.353+0000 7fd55bfff700 10 client.26674 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.353+0000 7fd580f9b700 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb67e00 0x5574ecb6a220 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea336ea1047e1fe8 server_cookie=523b3c981522a6e2 in_seq=0 out_seq=0 2023-10-06T21:30:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd55bfff700 1 -- 172.21.15.31:0/630162329 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5574eca44ad0 con 0x5574ecb67e00 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd55bfff700 10 client.26674 handle_command_reply: tid=0 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 client.26674 shutdown 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 msgr2=0x5574ecafc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd57400b880 tx=0x7fd57401f6d0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 shutdown_connections 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574ecb033d0 0x5574ecafc820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574ecaf7c00 0x5574ecafd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5574ecb67e00 0x5574ecb6a220 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 --2- 172.21.15.31:0/630162329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574ecaf8520 0x5574ecafc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd55bfff700 0 client.26674 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 >> 172.21.15.31:0/630162329 conn(0x5574eca6d200 msgr2=0x5574ecb01970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 shutdown_connections 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 1 -- 172.21.15.31:0/630162329 wait complete. 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd596b38b80 20 client.26674 trim_cache size 0 max 16384 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:30:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:30:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:30:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 msgr2=0x7fd56c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 0x7fd56c0a3ab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd57400d010 tx=0x7fd574007d80 comp rx=0 tx=0).stop 2023-10-06T21:30:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 msgr2=0x7fd57c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9a0 tx=0x7fd57000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 shutdown_connections 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57c10db20 0x7fd57c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd56c0a15f0 0x7fd56c0a3ab0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd57c0682d0 0x7fd57c1991e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 --2- 172.21.15.31:0/3914539443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd57c068bf0 0x7fd57c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 >> 172.21.15.31:0/3914539443 conn(0x7fd57c076b70 msgr2=0x7fd57c10ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.375+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 shutdown_connections 2023-10-06T21:30:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:30:55.376+0000 7fd5831ff700 1 -- 172.21.15.31:0/3914539443 wait complete. 2023-10-06T21:30:55.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:55 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:55.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:55 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:55 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:30:56.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166799 1.68 MB/sec execute 406 sec latency 1138.940 ms 2023-10-06T21:30:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:56 smithi142 ceph-mon[102951]: pgmap v311: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:56 smithi031 ceph-mon[98539]: pgmap v311: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:56 smithi146 ceph-mon[102809]: pgmap v311: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:57.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166799 1.68 MB/sec execute 407 sec latency 2139.049 ms 2023-10-06T21:30:58.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166799 1.67 MB/sec execute 408 sec latency 3139.160 ms 2023-10-06T21:30:58.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:58 smithi142 ceph-mon[102951]: pgmap v312: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:58.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:58 smithi031 ceph-mon[98539]: pgmap v312: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:58 smithi146 ceph-mon[102809]: pgmap v312: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 724 KiB/s wr, 77 op/s 2023-10-06T21:30:59.355 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166799 1.67 MB/sec execute 409 sec latency 4139.314 ms 2023-10-06T21:31:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:30:59 smithi031 ceph-mon[98539]: pgmap v313: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-10-06T21:31:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:30:59 smithi146 ceph-mon[102809]: pgmap v313: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-10-06T21:31:00.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166866 1.66 MB/sec execute 410 sec latency 4243.484 ms 2023-10-06T21:31:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:30:59 smithi142 ceph-mon[102951]: pgmap v313: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-10-06T21:31:01.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166866 1.66 MB/sec execute 411 sec latency 1537.312 ms 2023-10-06T21:31:02.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166866 1.66 MB/sec execute 412 sec latency 2537.429 ms 2023-10-06T21:31:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:02 smithi142 ceph-mon[102951]: pgmap v314: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 714 KiB/s wr, 68 op/s 2023-10-06T21:31:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:02 smithi031 ceph-mon[98539]: pgmap v314: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 714 KiB/s wr, 68 op/s 2023-10-06T21:31:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:02 smithi146 ceph-mon[102809]: pgmap v314: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 714 KiB/s wr, 68 op/s 2023-10-06T21:31:03.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166866 1.65 MB/sec execute 413 sec latency 3537.543 ms 2023-10-06T21:31:04.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 166866 1.65 MB/sec execute 414 sec latency 4537.702 ms 2023-10-06T21:31:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:04 smithi142 ceph-mon[102951]: pgmap v315: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 743 KiB/s wr, 77 op/s 2023-10-06T21:31:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:04 smithi031 ceph-mon[98539]: pgmap v315: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 743 KiB/s wr, 77 op/s 2023-10-06T21:31:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:04 smithi146 ceph-mon[102809]: pgmap v315: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 743 KiB/s wr, 77 op/s 2023-10-06T21:31:05.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 167034 1.65 MB/sec execute 415 sec latency 4636.507 ms 2023-10-06T21:31:06.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 167202 1.64 MB/sec execute 416 sec latency 19.034 ms 2023-10-06T21:31:06.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:06 smithi142 ceph-mon[102951]: pgmap v316: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:06 smithi031 ceph-mon[98539]: pgmap v316: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:06 smithi146 ceph-mon[102809]: pgmap v316: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:07.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 167454 1.64 MB/sec execute 417 sec latency 21.816 ms 2023-10-06T21:31:08.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 167804 1.64 MB/sec execute 418 sec latency 17.994 ms 2023-10-06T21:31:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:08 smithi142 ceph-mon[102951]: pgmap v317: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:08 smithi031 ceph-mon[98539]: pgmap v317: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:08 smithi146 ceph-mon[102809]: pgmap v317: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 341 KiB/s wr, 37 op/s 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 -- 172.21.15.31:0/1398237744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 msgr2=0x7fdf80107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 --2- 172.21.15.31:0/1398237744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf80107340 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0098f0 tx=0x7fdf7c009c00 comp rx=0 tx=0).stop 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 -- 172.21.15.31:0/1398237744 shutdown_connections 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 --2- 172.21.15.31:0/1398237744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80068c90 0x7fdf8010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 --2- 172.21.15.31:0/1398237744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf80107340 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 --2- 172.21.15.31:0/1398237744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf800682d0 0x7fdf80068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 -- 172.21.15.31:0/1398237744 >> 172.21.15.31:0/1398237744 conn(0x7fdf80076b70 msgr2=0x7fdf80076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 -- 172.21.15.31:0/1398237744 shutdown_connections 2023-10-06T21:31:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.552+0000 7fdf875ff700 1 -- 172.21.15.31:0/1398237744 wait complete. 2023-10-06T21:31:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 Processor -- start 2023-10-06T21:31:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 -- start start 2023-10-06T21:31:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 0x7fdf801991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 0x7fdf80199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019e3c0 con 0x7fdf80106f60 2023-10-06T21:31:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019e530 con 0x7fdf80068c90 2023-10-06T21:31:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.553+0000 7fdf875ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019e6f0 con 0x7fdf800682d0 2023-10-06T21:31:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf8539b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 0x7fdf801991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44982/0 (socket says 172.21.15.31:44982) 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf8539b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 0x7fdf801991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43092/0 (socket says 172.21.15.31:43092) 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 -- 172.21.15.31:0/2837682228 learned_addr learned my addr 172.21.15.31:0/2837682228 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 0x7fdf80199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 -- 172.21.15.31:0/2837682228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 msgr2=0x7fdf801991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 0x7fdf801991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 -- 172.21.15.31:0/2837682228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 msgr2=0x7fdf80199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 0x7fdf80199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 -- 172.21.15.31:0/2837682228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700095f0 con 0x7fdf80106f60 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 0x7fdf80199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.554+0000 7fdf85b9c700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800efc0 tx=0x7fdf7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.555+0000 7fdf767fc700 1 -- 172.21.15.31:0/2837682228 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78018070 con 0x7fdf80106f60 2023-10-06T21:31:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.555+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c009520 con 0x7fdf80106f60 2023-10-06T21:31:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.555+0000 7fdf767fc700 1 -- 172.21.15.31:0/2837682228 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf7800f040 con 0x7fdf80106f60 2023-10-06T21:31:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.555+0000 7fdf767fc700 1 -- 172.21.15.31:0/2837682228 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800cbb0 con 0x7fdf80106f60 2023-10-06T21:31:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.555+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf801a6030 con 0x7fdf80106f60 2023-10-06T21:31:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.559+0000 7fdf767fc700 1 -- 172.21.15.31:0/2837682228 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf7800cd90 con 0x7fdf80106f60 2023-10-06T21:31:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.560+0000 7fdf9af46b80 1 Processor -- start 2023-10-06T21:31:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.560+0000 7fdf9af46b80 1 -- start start 2023-10-06T21:31:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.560+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x56130421fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613042203d0 0x5613042227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130417e400 con 0x561304215520 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130416fbc0 con 0x5613042203d0 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130416d4e0 con 0x561304214c00 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43108/0 (socket says 172.21.15.31:43108) 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 -- 172.21.15.31:0/1289530126 learned_addr learned my addr 172.21.15.31:0/1289530126 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613042203d0 0x5613042227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf8539b700 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x56130421fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf767fc700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 -- 172.21.15.31:0/1289530126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613042203d0 msgr2=0x5613042227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613042203d0 0x5613042227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 -- 172.21.15.31:0/1289530126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 msgr2=0x56130421fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x56130421fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf84b9a700 1 -- 172.21.15.31:0/1289530126 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561303f93d80 con 0x561304214c00 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.561+0000 7fdf767fc700 1 -- 172.21.15.31:0/2837682228 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fdf780ccc80 con 0x7fdf80106f60 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.562+0000 7fdf85b9c700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.562+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf70012a40 tx=0x7fdf70009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ef5a78f27cbc30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.562+0000 7fdf6b7fe700 1 -- 172.21.15.31:0/1289530126 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70012d50 con 0x561304214c00 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.562+0000 7fdf6b7fe700 1 -- 172.21.15.31:0/1289530126 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf70004030 con 0x561304214c00 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.562+0000 7fdf6b7fe700 1 -- 172.21.15.31:0/1289530126 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70004bd0 con 0x561304214c00 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1289530126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 msgr2=0x561304214fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf85b9c700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf780090a0 tx=0x7fdf78021000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf70012a40 tx=0x7fdf70009a30 comp rx=0 tx=0).stop 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1289530126 shutdown_connections 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613042203d0 0x5613042227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304214c00 0x561304214fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1289530126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x56130421fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1289530126 >> 172.21.15.31:0/1289530126 conn(0x56130418a200 msgr2=0x561304211080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1289530126 shutdown_connections 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.563+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1289530126 wait complete. 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 -- start start 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 Processor -- start 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 0x56130421d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613042203d0 0x56130421d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561304168dd0 con 0x5613042203d0 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613041056b0 con 0x561304214c00 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561304105350 con 0x561304215520 2023-10-06T21:31:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf9af46b80 10 client.0 _ll_register_callbacks cb 0x561303f627a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.564+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42800/0 (socket says 172.21.15.31:42800) 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf84b9a700 1 -- 172.21.15.31:0/1675850229 learned_addr learned my addr 172.21.15.31:0/1675850229 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 0x56130421d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613042203d0 0x56130421d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 -- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 msgr2=0x56130421fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 -- 172.21.15.31:0/1675850229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613042203d0 msgr2=0x56130421d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613042203d0 0x56130421d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 -- 172.21.15.31:0/1675850229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7001a040 con 0x561304215520 2023-10-06T21:31:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf8539b700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 0x56130421d060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009810 tx=0x7fdf7c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d67248b378404ad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.565+0000 7fdf68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00b760 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561304208d50 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fdf7c00b8c0 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7c00ba80 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf9af46b80 10 client.26686 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561303f7cc00 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.566+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561304208d50 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdf7c015020 con 0x561304215520 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 10 client.26686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 20 client.26686 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561304230030 0x561304232450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 4 client.26686 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561304161db0 con 0x561304230030 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.567+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561304230030 0x561304232450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.568+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561304230030 0x561304232450 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1519db0b54908b1f server_cookie=d12643b309b791c4 in_seq=0 out_seq=0 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.568+0000 7fdf68ff9700 10 client.26686 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.568+0000 7fdf68ff9700 1 -- 172.21.15.31:0/1675850229 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561304161db0 con 0x561304230030 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.568+0000 7fdf68ff9700 10 client.26686 handle_command_reply: tid=0 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 client.26686 shutdown 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 msgr2=0x56130421d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 0x56130421d060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009810 tx=0x7fdf7c00f690 comp rx=0 tx=0).stop 2023-10-06T21:31:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 shutdown_connections 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561304230030 0x561304232450 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613042203d0 0x56130421d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf68ff9700 0 client.26686 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/1675850229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561304215520 0x56130421d060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 >> 172.21.15.31:0/1675850229 conn(0x56130418a200 msgr2=0x561304210c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 shutdown_connections 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 1 -- 172.21.15.31:0/1675850229 wait complete. 2023-10-06T21:31:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.569+0000 7fdf9af46b80 20 client.26686 trim_cache size 0 max 16384 2023-10-06T21:31:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 Processor -- start 2023-10-06T21:31:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 -- start start 2023-10-06T21:31:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x56130420f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561303fe98c0 con 0x561304215520 2023-10-06T21:31:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561304227480 con 0x561304214c00 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.589+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561304218ed0 con 0x5613042203d0 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45000/0 (socket says 172.21.15.31:45000) 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 -- 172.21.15.31:0/2029217347 learned_addr learned my addr 172.21.15.31:0/2029217347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x56130420f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 -- 172.21.15.31:0/2029217347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 msgr2=0x56130420f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x56130420f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 -- 172.21.15.31:0/2029217347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 msgr2=0x56130421c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 -- 172.21.15.31:0/2029217347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561303fba1f0 con 0x561304215520 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.590+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c003890 tx=0x7fdf7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a391a59acfdaa321 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.591+0000 7fdf6bfff700 1 -- 172.21.15.31:0/2029217347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005310 con 0x561304215520 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.591+0000 7fdf6bfff700 1 -- 172.21.15.31:0/2029217347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf7c01d070 con 0x561304215520 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.591+0000 7fdf6bfff700 1 -- 172.21.15.31:0/2029217347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005470 con 0x561304215520 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2029217347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 msgr2=0x561304226f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c003890 tx=0x7fdf7c009840 comp rx=0 tx=0).stop 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2029217347 shutdown_connections 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x56130420f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x56130421c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2029217347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304226f10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2029217347 >> 172.21.15.31:0/2029217347 conn(0x56130418a200 msgr2=0x561304236d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2029217347 shutdown_connections 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.592+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2029217347 wait complete. 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.593+0000 7fdf9af46b80 1 -- start start 2023-10-06T21:31:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.593+0000 7fdf9af46b80 1 Processor -- start 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304219820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x561304219d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130421a2a0 con 0x561304215520 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130421a3e0 con 0x561304214c00 2023-10-06T21:31:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130421a5a0 con 0x5613042203d0 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf9af46b80 10 client.0 _ll_register_callbacks cb 0x7fdf7c00feb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf84b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42818/0 (socket says 172.21.15.31:42818) 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.594+0000 7fdf84b9a700 1 -- 172.21.15.31:0/2183710791 learned_addr learned my addr 172.21.15.31:0/2183710791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x561304219d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304219820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 -- 172.21.15.31:0/2183710791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 msgr2=0x561304219d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x561304219d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 -- 172.21.15.31:0/2183710791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 msgr2=0x561304219820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304219820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 -- 172.21.15.31:0/2183710791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561303f7cc00 con 0x561304214c00 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.595+0000 7fdf84b9a700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf70012500 tx=0x7fdf70009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db25aec3a155beb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.596+0000 7fdf68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.596+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70012030 con 0x561304214c00 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.596+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fdf70012190 con 0x561304214c00 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.596+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf70004030 con 0x561304214c00 2023-10-06T21:31:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.597+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf70012350 con 0x561304214c00 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.597+0000 7fdf9af46b80 10 client.26747 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.597+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561303f92250 con 0x561304214c00 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.597+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf70004030 con 0x561304214c00 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.597+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdf7003b020 con 0x561304214c00 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 20 client.26747 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613042847f0 0x561304286c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 4 client.26747 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561304161ad0 con 0x5613042847f0 2023-10-06T21:31:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.598+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613042847f0 0x561304286c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.599+0000 7fdf8539b700 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613042847f0 0x561304286c10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c59fb72ddb03fa1 server_cookie=57fd8e582f26a22 in_seq=0 out_seq=0 2023-10-06T21:31:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.599+0000 7fdf68ff9700 10 client.26747 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.623+0000 7fdf68ff9700 1 -- 172.21.15.31:0/2183710791 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561304161ad0 con 0x5613042847f0 2023-10-06T21:31:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.623+0000 7fdf68ff9700 10 client.26747 handle_command_reply: tid=0 2023-10-06T21:31:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 client.26747 shutdown 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 msgr2=0x5613042192e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf70012500 tx=0x7fdf70009a30 comp rx=0 tx=0).stop 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 shutdown_connections 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613042203d0 0x561304219d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561304214c00 0x5613042192e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613042847f0 0x561304286c10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 --2- 172.21.15.31:0/2183710791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561304215520 0x561304219820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 >> 172.21.15.31:0/2183710791 conn(0x56130418a200 msgr2=0x56130422b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf68ff9700 0 client.26747 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 shutdown_connections 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 1 -- 172.21.15.31:0/2183710791 wait complete. 2023-10-06T21:31:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.624+0000 7fdf9af46b80 20 client.26747 trim_cache size 0 max 16384 2023-10-06T21:31:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:31:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:31:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:31:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 msgr2=0x7fdf6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf780090a0 tx=0x7fdf78021000 comp rx=0 tx=0).stop 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 msgr2=0x7fdf8019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800efc0 tx=0x7fdf7800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 shutdown_connections 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf80106f60 0x7fdf8019dd80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf800682d0 0x7fdf801991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 --2- 172.21.15.31:0/2837682228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80068c90 0x7fdf80199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 >> 172.21.15.31:0/2837682228 conn(0x7fdf80076b70 msgr2=0x7fdf8010bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 shutdown_connections 2023-10-06T21:31:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:08.625+0000 7fdf875ff700 1 -- 172.21.15.31:0/2837682228 wait complete. 2023-10-06T21:31:09.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 168165 1.64 MB/sec execute 419 sec latency 28.349 ms 2023-10-06T21:31:10.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 168460 1.64 MB/sec execute 420 sec latency 28.471 ms 2023-10-06T21:31:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:10 smithi142 ceph-mon[102951]: pgmap v318: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 723 KiB/s wr, 86 op/s 2023-10-06T21:31:10.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:10 smithi142 ceph-mon[102951]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:31:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:10 smithi031 ceph-mon[98539]: pgmap v318: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 723 KiB/s wr, 86 op/s 2023-10-06T21:31:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:10 smithi031 ceph-mon[98539]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:31:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:10 smithi146 ceph-mon[102809]: pgmap v318: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 723 KiB/s wr, 86 op/s 2023-10-06T21:31:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:10 smithi146 ceph-mon[102809]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:31:11.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 168839 1.64 MB/sec execute 421 sec latency 29.172 ms 2023-10-06T21:31:12.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 169173 1.64 MB/sec execute 422 sec latency 28.262 ms 2023-10-06T21:31:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:12 smithi142 ceph-mon[102951]: pgmap v320: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 492 KiB/s wr, 70 op/s 2023-10-06T21:31:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:12 smithi031 ceph-mon[98539]: pgmap v320: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 492 KiB/s wr, 70 op/s 2023-10-06T21:31:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:12 smithi146 ceph-mon[102809]: pgmap v320: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 492 KiB/s wr, 70 op/s 2023-10-06T21:31:13.356 INFO:tasks.workunit.client.0.smithi031.stdout: 1 169656 1.65 MB/sec execute 423 sec latency 18.332 ms 2023-10-06T21:31:14.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170045 1.65 MB/sec execute 424 sec latency 18.669 ms 2023-10-06T21:31:14.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:14 smithi142 ceph-mon[102951]: pgmap v321: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 967 KiB/s wr, 111 op/s 2023-10-06T21:31:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:14 smithi142 ceph-mon[102951]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:31:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:14 smithi031 ceph-mon[98539]: pgmap v321: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 967 KiB/s wr, 111 op/s 2023-10-06T21:31:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:14 smithi031 ceph-mon[98539]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:31:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:14 smithi146 ceph-mon[102809]: pgmap v321: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 967 KiB/s wr, 111 op/s 2023-10-06T21:31:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:14 smithi146 ceph-mon[102809]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:31:15.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170346 1.65 MB/sec execute 425 sec latency 84.478 ms 2023-10-06T21:31:16.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170346 1.65 MB/sec execute 426 sec latency 1084.544 ms 2023-10-06T21:31:16.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:16 smithi142 ceph-mon[102951]: pgmap v323: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-10-06T21:31:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:16 smithi031 ceph-mon[98539]: pgmap v323: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-10-06T21:31:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:16 smithi146 ceph-mon[102809]: pgmap v323: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-10-06T21:31:17.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170346 1.64 MB/sec execute 427 sec latency 2084.634 ms 2023-10-06T21:31:18.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170346 1.64 MB/sec execute 428 sec latency 3084.680 ms 2023-10-06T21:31:18.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:18 smithi142 ceph-mon[102951]: pgmap v324: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 637 KiB/s wr, 66 op/s 2023-10-06T21:31:18.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:18 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:18 smithi031 ceph-mon[98539]: pgmap v324: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 637 KiB/s wr, 66 op/s 2023-10-06T21:31:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:18 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:18.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:18 smithi146 ceph-mon[102809]: pgmap v324: 97 pgs: 97 active+clean; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 637 KiB/s wr, 66 op/s 2023-10-06T21:31:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:18 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:19.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170346 1.63 MB/sec execute 429 sec latency 4084.767 ms 2023-10-06T21:31:20.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:20 smithi031 ceph-mon[98539]: pgmap v325: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 861 KiB/s wr, 81 op/s 2023-10-06T21:31:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:20 smithi146 ceph-mon[102809]: pgmap v325: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 861 KiB/s wr, 81 op/s 2023-10-06T21:31:20.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170413 1.63 MB/sec execute 430 sec latency 4184.776 ms 2023-10-06T21:31:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:20 smithi142 ceph-mon[102951]: pgmap v325: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 861 KiB/s wr, 81 op/s 2023-10-06T21:31:21.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170413 1.63 MB/sec execute 431 sec latency 1577.515 ms 2023-10-06T21:31:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3256265424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4073b70 msgr2=0x7fe7c4111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3256265424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4073b70 0x7fe7c4111930 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80098f0 tx=0x7fe7b8009c00 comp rx=0 tx=0).stop 2023-10-06T21:31:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3256265424 shutdown_connections 2023-10-06T21:31:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3256265424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4111e70 0x7fe7c4114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3256265424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073250 0x7fe7c4073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3256265424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4073b70 0x7fe7c4111930 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3256265424 >> 172.21.15.31:0/3256265424 conn(0x7fe7c40fc950 msgr2=0x7fe7c40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3256265424 shutdown_connections 2023-10-06T21:31:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.777+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3256265424 wait complete. 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.778+0000 7fe7c8bff700 1 Processor -- start 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.778+0000 7fe7c8bff700 1 -- start start 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.778+0000 7fe7c8bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 0x7fe7c41a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.778+0000 7fe7c8bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c8bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4111e70 0x7fe7c419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c8bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419da50 con 0x7fe7c4111e70 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c8bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419dbc0 con 0x7fe7c4073250 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c8bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419dd80 con 0x7fe7c4073b70 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4111e70 0x7fe7c419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 0x7fe7c41a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56560/0 (socket says 172.21.15.31:56560) 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 -- 172.21.15.31:0/3537099197 learned_addr learned my addr 172.21.15.31:0/3537099197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 0x7fe7c41a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33184/0 (socket says 172.21.15.31:33184) 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 -- 172.21.15.31:0/3537099197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 msgr2=0x7fe7c41a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 0x7fe7c41a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 -- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4111e70 msgr2=0x7fe7c419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4111e70 0x7fe7c419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.779+0000 7fe7c1d9b700 1 -- 172.21.15.31:0/3537099197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8009520 con 0x7fe7c4073b70 2023-10-06T21:31:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009230 tx=0x7fe7b800b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7b37fe700 1 -- 172.21.15.31:0/3537099197 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b801c070 con 0x7fe7c4073b70 2023-10-06T21:31:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7b37fe700 1 -- 172.21.15.31:0/3537099197 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b8005040 con 0x7fe7c4073b70 2023-10-06T21:31:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c419e000 con 0x7fe7c4073b70 2023-10-06T21:31:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7b37fe700 1 -- 172.21.15.31:0/3537099197 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80038e0 con 0x7fe7c4073b70 2023-10-06T21:31:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.780+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41aa570 con 0x7fe7c4073b70 2023-10-06T21:31:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7dc52cb80 1 Processor -- start 2023-10-06T21:31:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7dc52cb80 1 -- start start 2023-10-06T21:31:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7b37fe700 1 -- 172.21.15.31:0/3537099197 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe7b8020030 con 0x7fe7c4073b70 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9bf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.785+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9c043d0 0x556ea9c067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9b62400 con 0x556ea9c043d0 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9b53bc0 con 0x556ea9bf8c00 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9b514e0 con 0x556ea9bf9520 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56572/0 (socket says 172.21.15.31:56572) 2023-10-06T21:31:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 -- 172.21.15.31:0/562910912 learned_addr learned my addr 172.21.15.31:0/562910912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9c043d0 0x556ea9c067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9bf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 -- 172.21.15.31:0/562910912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 msgr2=0x556ea9bf8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9bf8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 -- 172.21.15.31:0/562910912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9c043d0 msgr2=0x556ea9c067c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9c043d0 0x556ea9c067c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c259c700 1 -- 172.21.15.31:0/562910912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea9977d80 con 0x556ea9bf9520 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.786+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9bf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7c259c700 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009d60 tx=0x7fe7b4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc0f8bb991266f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7b0ff9700 1 -- 172.21.15.31:0/562910912 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4019630 con 0x556ea9bf9520 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7b0ff9700 1 -- 172.21.15.31:0/562910912 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b4019c20 con 0x556ea9bf9520 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7b0ff9700 1 -- 172.21.15.31:0/562910912 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400eb40 con 0x556ea9bf9520 2023-10-06T21:31:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7b37fe700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 0x7fe7ac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7b37fe700 1 -- 172.21.15.31:0/3537099197 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe7b80ce310 con 0x7fe7c4073b70 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.787+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 0x7fe7ac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/562910912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 msgr2=0x556ea9c03e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009d60 tx=0x7fe7b4011720 comp rx=0 tx=0).stop 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/562910912 shutdown_connections 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9c043d0 0x556ea9c067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9bf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/562910912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03e90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/562910912 >> 172.21.15.31:0/562910912 conn(0x556ea9b6e200 msgr2=0x556ea9bf5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/562910912 shutdown_connections 2023-10-06T21:31:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 0x7fe7ac0a3ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009fd0 tx=0x7fe7bc00d040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.788+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/562910912 wait complete. 2023-10-06T21:31:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.789+0000 7fe7dc52cb80 1 -- start start 2023-10-06T21:31:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.789+0000 7fe7dc52cb80 1 Processor -- start 2023-10-06T21:31:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 0x556ea9c01000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9c01630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9b4cdd0 con 0x556ea9bf9520 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9ae96b0 con 0x556ea9bf8c00 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9ae9350 con 0x556ea9c043d0 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7dc52cb80 10 client.0 _ll_register_callbacks cb 0x556ea99467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7c1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 0x556ea9c01000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33206/0 (socket says 172.21.15.31:33206) 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7c259c700 1 -- 172.21.15.31:0/2686144250 learned_addr learned my addr 172.21.15.31:0/2686144250 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.790+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9c01630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 -- 172.21.15.31:0/2686144250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 msgr2=0x556ea9c01630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9c01630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 -- 172.21.15.31:0/2686144250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 msgr2=0x556ea9c01000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 0x556ea9c01000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 -- 172.21.15.31:0/2686144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc009c20 con 0x556ea9bf8c00 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 0x556ea9c01000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000c00 tx=0x7fe7b4019a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3be1b54dcf37d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.791+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4005450 con 0x556ea9bf8c00 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea9becd50 con 0x556ea9bf8c00 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe7b400e4d0 con 0x556ea9bf8c00 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b40055b0 con 0x556ea9bf8c00 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7dc52cb80 10 client.26756 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fe7b40099c0 con 0x556ea9bf8c00 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.792+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ea9becd50 con 0x556ea9bf8c00 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe7b4004c40 con 0x556ea9bf8c00 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 10 client.26756 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 10 client.26756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 20 client.26756 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c13df0 0x556ea9c16210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 4 client.26756 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ea9b45db0 con 0x556ea9c13df0 2023-10-06T21:31:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.793+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c13df0 0x556ea9c16210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.794+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c13df0 0x556ea9c16210 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b875d86b9fe8282 server_cookie=5fab29d4bde37f4d in_seq=0 out_seq=0 2023-10-06T21:31:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.794+0000 7fe7a9ffb700 10 client.26756 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.794+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2686144250 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ea9b45db0 con 0x556ea9c13df0 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.794+0000 7fe7a9ffb700 10 client.26756 handle_command_reply: tid=0 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 client.26756 shutdown 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 msgr2=0x556ea9c03d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000c00 tx=0x7fe7b4019a90 comp rx=0 tx=0).stop 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 shutdown_connections 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9c01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf8c00 0x556ea9c03d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c13df0 0x556ea9c16210 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2686144250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf9520 0x556ea9c01000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7a9ffb700 0 client.26756 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 >> 172.21.15.31:0/2686144250 conn(0x556ea9b6e200 msgr2=0x556ea9bf4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 shutdown_connections 2023-10-06T21:31:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2686144250 wait complete. 2023-10-06T21:31:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.795+0000 7fe7dc52cb80 20 client.26756 trim_cache size 0 max 16384 2023-10-06T21:31:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 Processor -- start 2023-10-06T21:31:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 -- start start 2023-10-06T21:31:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9c043d0 0x556ea9c00820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea99cd8c0 con 0x556ea9bf8c00 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9bf5dc0 con 0x556ea9c043d0 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.813+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9c0b480 con 0x556ea9bf9520 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55054/0 (socket says 172.21.15.31:55054) 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 -- 172.21.15.31:0/369211148 learned_addr learned my addr 172.21.15.31:0/369211148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c2d9d700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9c043d0 0x556ea9c00820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 -- 172.21.15.31:0/369211148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 msgr2=0x556ea9c03920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 -- 172.21.15.31:0/369211148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9c043d0 msgr2=0x556ea9c00820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9c043d0 0x556ea9c00820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 -- 172.21.15.31:0/369211148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea9960c00 con 0x556ea9bf8c00 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.814+0000 7fe7c259c700 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4011a70 tx=0x7fe7b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3286c411914a58af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.815+0000 7fe7b17fa700 1 -- 172.21.15.31:0/369211148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4003cb0 con 0x556ea9bf8c00 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.815+0000 7fe7b17fa700 1 -- 172.21.15.31:0/369211148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b40056e0 con 0x556ea9bf8c00 2023-10-06T21:31:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.815+0000 7fe7b17fa700 1 -- 172.21.15.31:0/369211148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40327f0 con 0x556ea9bf8c00 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/369211148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 msgr2=0x556ea9c0af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4011a70 tx=0x7fe7b4011aa0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/369211148 shutdown_connections 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9c043d0 0x556ea9c00820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bf8c00 0x556ea9c0af40 secure :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4011a70 tx=0x7fe7b4011aa0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/369211148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9bf9520 0x556ea9c03920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/369211148 >> 172.21.15.31:0/369211148 conn(0x556ea9b6e200 msgr2=0x556ea9bfc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/369211148 shutdown_connections 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/369211148 wait complete. 2023-10-06T21:31:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.816+0000 7fe7dc52cb80 1 -- start start 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.817+0000 7fe7dc52cb80 1 Processor -- start 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.817+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 0x556ea9bfd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.817+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7dc52cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bfdd60 0x556ea9bfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9c0b280 con 0x556ea9bfdd60 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9bfe720 con 0x556ea9bf9520 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7dc52cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea9bfe8e0 con 0x556ea9c043d0 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7dc52cb80 10 client.0 _ll_register_callbacks cb 0x7fe7b80d1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bfdd60 0x556ea9bfe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 0x556ea9bfd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56606/0 (socket says 172.21.15.31:56606) 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c1d9b700 1 -- 172.21.15.31:0/2000010482 learned_addr learned my addr 172.21.15.31:0/2000010482 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 -- 172.21.15.31:0/2000010482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 msgr2=0x556ea9bfd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 -- 172.21.15.31:0/2000010482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bfdd60 msgr2=0x556ea9bfe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bfdd60 0x556ea9bfe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c259c700 1 -- 172.21.15.31:0/2000010482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41aa570 con 0x556ea9bf9520 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.818+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7c259c700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 0x556ea9bfd2e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x556ea9c115b0 tx=0x7fe7b4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c01f04b968eed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:31:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4003ab0 con 0x556ea9bf9520 2023-10-06T21:31:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7b4019c20 con 0x556ea9bf9520 2023-10-06T21:31:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe7b4000e40 con 0x556ea9bf9520 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b4003c10 con 0x556ea9bf9520 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7dc52cb80 10 client.26762 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.819+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fe7b40099c0 con 0x556ea9bf9520 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b4046020 con 0x556ea9bf9520 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe7b4042420 con 0x556ea9bf9520 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 10 client.26762 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 10 client.26762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 20 client.26762 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c683c0 0x556ea9c6a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 4 client.26762 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ea9b45ad0 con 0x556ea9c683c0 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.820+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c683c0 0x556ea9c6a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.821+0000 7fe7a9ffb700 10 client.26762 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.821+0000 7fe7c1d9b700 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c683c0 0x556ea9c6a7e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc8db8564662be3 server_cookie=692d9283b92758a4 in_seq=0 out_seq=0 2023-10-06T21:31:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7a9ffb700 1 -- 172.21.15.31:0/2000010482 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ea9b45ad0 con 0x556ea9c683c0 2023-10-06T21:31:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7a9ffb700 10 client.26762 handle_command_reply: tid=0 2023-10-06T21:31:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 client.26762 shutdown 2023-10-06T21:31:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 msgr2=0x556ea9bfd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 0x556ea9bfd2e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x556ea9c115b0 tx=0x7fe7b4009580 comp rx=0 tx=0).stop 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 shutdown_connections 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ea9c683c0 0x556ea9c6a7e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea9bfdd60 0x556ea9bfe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ea9bf9520 0x556ea9bfd2e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7a9ffb700 0 client.26762 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 --2- 172.21.15.31:0/2000010482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea9c043d0 0x556ea9bfd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 >> 172.21.15.31:0/2000010482 conn(0x556ea9b6e200 msgr2=0x556ea9bfab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 shutdown_connections 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 1 -- 172.21.15.31:0/2000010482 wait complete. 2023-10-06T21:31:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.846+0000 7fe7dc52cb80 20 client.26762 trim_cache size 0 max 16384 2023-10-06T21:31:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:31:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:31:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:31:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 msgr2=0x7fe7ac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 0x7fe7ac0a3ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009fd0 tx=0x7fe7bc00d040 comp rx=0 tx=0).stop 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 msgr2=0x7fe7c41a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009230 tx=0x7fe7b800b560 comp rx=0 tx=0).stop 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 shutdown_connections 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe7ac0a15f0 0x7fe7ac0a3ab0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c4111e70 0x7fe7c419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c4073250 0x7fe7c41a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 --2- 172.21.15.31:0/3537099197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c4073b70 0x7fe7c41a3ac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 >> 172.21.15.31:0/3537099197 conn(0x7fe7c40fc950 msgr2=0x7fe7c40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 shutdown_connections 2023-10-06T21:31:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:21.847+0000 7fe7c8bff700 1 -- 172.21.15.31:0/3537099197 wait complete. 2023-10-06T21:31:22.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170413 1.62 MB/sec execute 432 sec latency 2577.619 ms 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: pgmap v326: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 855 KiB/s wr, 80 op/s 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:22 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: pgmap v326: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 855 KiB/s wr, 80 op/s 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:22 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: pgmap v326: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 855 KiB/s wr, 80 op/s 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:22.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:22 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170413 1.62 MB/sec execute 433 sec latency 3577.755 ms 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:31:23.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:23.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:31:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:31:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:31:23.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:31:23.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:23.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:23.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:31:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:31:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:23.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:31:23.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:31:23.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:23.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:23.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:31:24.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170413 1.62 MB/sec execute 434 sec latency 4577.886 ms 2023-10-06T21:31:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:24 smithi031 ceph-mon[98539]: pgmap v327: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 410 KiB/s wr, 44 op/s 2023-10-06T21:31:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:24 smithi146 ceph-mon[102809]: pgmap v327: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 410 KiB/s wr, 44 op/s 2023-10-06T21:31:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:24 smithi142 ceph-mon[102951]: pgmap v327: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 410 KiB/s wr, 44 op/s 2023-10-06T21:31:25.357 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170590 1.61 MB/sec execute 435 sec latency 4678.384 ms 2023-10-06T21:31:26.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 170764 1.61 MB/sec execute 436 sec latency 18.837 ms 2023-10-06T21:31:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:26 smithi031 ceph-mon[98539]: pgmap v328: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 371 KiB/s wr, 40 op/s 2023-10-06T21:31:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:26 smithi146 ceph-mon[102809]: pgmap v328: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 371 KiB/s wr, 40 op/s 2023-10-06T21:31:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:26 smithi142 ceph-mon[102951]: pgmap v328: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 371 KiB/s wr, 40 op/s 2023-10-06T21:31:27.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 171030 1.61 MB/sec execute 437 sec latency 22.387 ms 2023-10-06T21:31:28.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 171369 1.61 MB/sec execute 438 sec latency 21.877 ms 2023-10-06T21:31:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:28 smithi031 ceph-mon[98539]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 342 KiB/s wr, 37 op/s 2023-10-06T21:31:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:28 smithi146 ceph-mon[102809]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 342 KiB/s wr, 37 op/s 2023-10-06T21:31:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:28 smithi142 ceph-mon[102951]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 342 KiB/s wr, 37 op/s 2023-10-06T21:31:29.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 171726 1.61 MB/sec execute 439 sec latency 31.751 ms 2023-10-06T21:31:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:30 smithi031 ceph-mon[98539]: pgmap v330: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 651 KiB/s wr, 71 op/s 2023-10-06T21:31:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:30 smithi146 ceph-mon[102809]: pgmap v330: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 651 KiB/s wr, 71 op/s 2023-10-06T21:31:30.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 171920 1.61 MB/sec execute 440 sec latency 403.551 ms 2023-10-06T21:31:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:30 smithi142 ceph-mon[102951]: pgmap v330: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 651 KiB/s wr, 71 op/s 2023-10-06T21:31:31.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 172285 1.61 MB/sec execute 441 sec latency 26.127 ms 2023-10-06T21:31:32.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 172664 1.61 MB/sec execute 442 sec latency 27.122 ms 2023-10-06T21:31:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:32 smithi142 ceph-mon[102951]: pgmap v331: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 363 KiB/s wr, 48 op/s 2023-10-06T21:31:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:32 smithi031 ceph-mon[98539]: pgmap v331: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 363 KiB/s wr, 48 op/s 2023-10-06T21:31:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:32 smithi146 ceph-mon[102809]: pgmap v331: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 363 KiB/s wr, 48 op/s 2023-10-06T21:31:33.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173033 1.61 MB/sec execute 443 sec latency 22.132 ms 2023-10-06T21:31:34.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173559 1.61 MB/sec execute 444 sec latency 18.390 ms 2023-10-06T21:31:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:34 smithi142 ceph-mon[102951]: pgmap v332: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 752 KiB/s wr, 87 op/s 2023-10-06T21:31:34.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:34 smithi031 ceph-mon[98539]: pgmap v332: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 752 KiB/s wr, 87 op/s 2023-10-06T21:31:34.567 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:34 smithi146 ceph-mon[102809]: pgmap v332: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 752 KiB/s wr, 87 op/s 2023-10-06T21:31:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 -- 172.21.15.31:0/449763494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d10106f60 msgr2=0x7f9d10107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 --2- 172.21.15.31:0/449763494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d10106f60 0x7f9d10107340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d00009ca0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 -- 172.21.15.31:0/449763494 shutdown_connections 2023-10-06T21:31:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 --2- 172.21.15.31:0/449763494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10068c90 0x7f9d1010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 --2- 172.21.15.31:0/449763494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d10106f60 0x7f9d10107340 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 --2- 172.21.15.31:0/449763494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d100682d0 0x7f9d10068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 -- 172.21.15.31:0/449763494 >> 172.21.15.31:0/449763494 conn(0x7f9d10076b70 msgr2=0x7f9d10076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 -- 172.21.15.31:0/449763494 shutdown_connections 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.019+0000 7f9d149ff700 1 -- 172.21.15.31:0/449763494 wait complete. 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.020+0000 7f9d149ff700 1 Processor -- start 2023-10-06T21:31:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 -- start start 2023-10-06T21:31:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 0x7f9d10100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d10068c90 0x7f9d10100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a5670 con 0x7f9d10068c90 2023-10-06T21:31:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a57b0 con 0x7f9d100682d0 2023-10-06T21:31:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.021+0000 7f9d149ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a5920 con 0x7f9d10106f60 2023-10-06T21:31:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 0x7f9d10100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d10068c90 0x7f9d10100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46196/0 (socket says 172.21.15.31:46196) 2023-10-06T21:31:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 0x7f9d10100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57268/0 (socket says 172.21.15.31:57268) 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 -- 172.21.15.31:0/2136716165 learned_addr learned my addr 172.21.15.31:0/2136716165 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 -- 172.21.15.31:0/2136716165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 msgr2=0x7f9d10100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 0x7f9d10100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 -- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d10068c90 msgr2=0x7f9d10100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d10068c90 0x7f9d10100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.022+0000 7f9d0ed9d700 1 -- 172.21.15.31:0/2136716165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000095f0 con 0x7f9d10106f60 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9d0ed9d700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007f10 tx=0x7f9d0800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9cff7fe700 1 -- 172.21.15.31:0/2136716165 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08009ea0 con 0x7f9d10106f60 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9cff7fe700 1 -- 172.21.15.31:0/2136716165 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d0800f040 con 0x7f9d10106f60 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d101a5c00 con 0x7f9d10106f60 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9cff7fe700 1 -- 172.21.15.31:0/2136716165 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08004640 con 0x7f9d10106f60 2023-10-06T21:31:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.023+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d101a60c0 con 0x7f9d10106f60 2023-10-06T21:31:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.026+0000 7f9cff7fe700 1 -- 172.21.15.31:0/2136716165 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9d080047a0 con 0x7f9d10106f60 2023-10-06T21:31:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 Processor -- start 2023-10-06T21:31:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 -- start start 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d840d520 0x5647d8417e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d84183d0 0x5647d841a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8376400 con 0x5647d84183d0 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8367bc0 con 0x5647d840cc00 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9d282dab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d83654e0 con 0x5647d840d520 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.028+0000 7f9cff7fe700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 0x7f9cf80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9cff7fe700 1 -- 172.21.15.31:0/2136716165 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9d080cc9d0 con 0x7f9d10106f60 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57270/0 (socket says 172.21.15.31:57270) 2023-10-06T21:31:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/2414645796 learned_addr learned my addr 172.21.15.31:0/2414645796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d84183d0 0x5647d841a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d840d520 0x5647d8417e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0ed9d700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 0x7f9cf80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/2414645796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d840d520 msgr2=0x5647d8417e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d840d520 0x5647d8417e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/2414645796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d84183d0 msgr2=0x5647d841a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d84183d0 0x5647d841a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.029+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/2414645796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d818bd80 con 0x5647d840cc00 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.030+0000 7f9d0ed9d700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 0x7f9cf80a3ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d08095da0 tx=0x7f9d0800df10 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.030+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d04011880 tx=0x7f9d040058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3375c6042492c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.030+0000 7f9cfcff9700 1 -- 172.21.15.31:0/2414645796 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04011ea0 con 0x5647d840cc00 2023-10-06T21:31:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.030+0000 7f9cfcff9700 1 -- 172.21.15.31:0/2414645796 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d040044c0 con 0x5647d840cc00 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.030+0000 7f9cfcff9700 1 -- 172.21.15.31:0/2414645796 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04005220 con 0x5647d840cc00 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 -- 172.21.15.31:0/2414645796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 msgr2=0x5647d840cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d04011880 tx=0x7f9d040058e0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 -- 172.21.15.31:0/2414645796 shutdown_connections 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d84183d0 0x5647d841a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d840cfe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 --2- 172.21.15.31:0/2414645796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d840d520 0x5647d8417e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 -- 172.21.15.31:0/2414645796 >> 172.21.15.31:0/2414645796 conn(0x5647d8382200 msgr2=0x5647d8409080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 -- 172.21.15.31:0/2414645796 shutdown_connections 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.031+0000 7f9d282dab80 1 -- 172.21.15.31:0/2414645796 wait complete. 2023-10-06T21:31:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 -- start start 2023-10-06T21:31:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 Processor -- start 2023-10-06T21:31:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8417dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8360dd0 con 0x5647d840d520 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d82fd6b0 con 0x5647d840cc00 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.032+0000 7f9d282dab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d82fd350 con 0x5647d84183d0 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d282dab80 10 client.0 _ll_register_callbacks cb 0x5647d815a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8417dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36558/0 (socket says 172.21.15.31:36558) 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 -- 172.21.15.31:0/300283809 learned_addr learned my addr 172.21.15.31:0/300283809 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 -- 172.21.15.31:0/300283809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 msgr2=0x5647d84156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 -- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 msgr2=0x5647d8417dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8417dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 -- 172.21.15.31:0/300283809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04019040 con 0x5647d840d520 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.033+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d00015000 tx=0x7f9d00005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2db6cf97c34f7de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9cf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0001d070 con 0x5647d840d520 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647d8400d50 con 0x5647d840d520 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9d00004500 con 0x5647d840d520 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d000046c0 con 0x5647d840d520 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9d282dab80 10 client.17067 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5647d8174c00 con 0x5647d840d520 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.034+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5647d8400d50 con 0x5647d840d520 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9d0000f750 con 0x5647d840d520 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 10 client.17067 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 20 client.17067 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d8427fc0 0x5647d842a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 4 client.17067 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647d8359db0 con 0x5647d8427fc0 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.035+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d8427fc0 0x5647d842a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.036+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d8427fc0 0x5647d842a3e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8467b8e159ccf61 server_cookie=135656a9454e1efc in_seq=0 out_seq=0 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.036+0000 7f9cf5ffb700 10 client.17067 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.036+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/300283809 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647d8359db0 con 0x5647d8427fc0 2023-10-06T21:31:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.036+0000 7f9cf5ffb700 10 client.17067 handle_command_reply: tid=0 2023-10-06T21:31:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.036+0000 7f9d282dab80 1 client.17067 shutdown 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 msgr2=0x5647d8415040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d00015000 tx=0x7f9d00005800 comp rx=0 tx=0).stop 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 shutdown_connections 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8417dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d8427fc0 0x5647d842a3e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 --2- 172.21.15.31:0/300283809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d8415040 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 >> 172.21.15.31:0/300283809 conn(0x5647d8382200 msgr2=0x5647d8408c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9cf5ffb700 0 client.17067 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 shutdown_connections 2023-10-06T21:31:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 1 -- 172.21.15.31:0/300283809 wait complete. 2023-10-06T21:31:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.037+0000 7f9d282dab80 20 client.17067 trim_cache size 0 max 16384 2023-10-06T21:31:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.055+0000 7f9d282dab80 1 Processor -- start 2023-10-06T21:31:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.055+0000 7f9d282dab80 1 -- start start 2023-10-06T21:31:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d84108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d81e18c0 con 0x5647d840d520 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8405890 con 0x5647d840cc00 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d282dab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8410f50 con 0x5647d84183d0 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d84108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57300/0 (socket says 172.21.15.31:57300) 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/3111594115 learned_addr learned my addr 172.21.15.31:0/3111594115 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.056+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/3111594115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 msgr2=0x5647d84197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/3111594115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 msgr2=0x5647d84108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d84108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/3111594115 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d81b21f0 con 0x5647d840cc00 2023-10-06T21:31:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d04013e90 tx=0x7f9d04009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9595282e91eab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9cfd7fa700 1 -- 172.21.15.31:0/3111594115 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040324a0 con 0x5647d840cc00 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9cfd7fa700 1 -- 172.21.15.31:0/3111594115 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d04038070 con 0x5647d840cc00 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.057+0000 7f9cfd7fa700 1 -- 172.21.15.31:0/3111594115 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04005270 con 0x5647d840cc00 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- 172.21.15.31:0/3111594115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 msgr2=0x5647d8410390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d04013e90 tx=0x7f9d04009d00 comp rx=0 tx=0).stop 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- 172.21.15.31:0/3111594115 shutdown_connections 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d84197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d8410390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3111594115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d84108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- 172.21.15.31:0/3111594115 >> 172.21.15.31:0/3111594115 conn(0x5647d8382200 msgr2=0x5647d84236a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- 172.21.15.31:0/3111594115 shutdown_connections 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- 172.21.15.31:0/3111594115 wait complete. 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.058+0000 7f9d282dab80 1 -- start start 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 Processor -- start 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d841d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d847adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d841ddd0 con 0x5647d840d520 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d847b330 con 0x5647d840cc00 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.059+0000 7f9d282dab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d847b4f0 con 0x5647d84183d0 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d282dab80 10 client.0 _ll_register_callbacks cb 0x7f9d00005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d841d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57310/0 (socket says 172.21.15.31:57310) 2023-10-06T21:31:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0dd9b700 1 -- 172.21.15.31:0/3254526542 learned_addr learned my addr 172.21.15.31:0/3254526542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d847adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 -- 172.21.15.31:0/3254526542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 msgr2=0x5647d847adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d847adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 -- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 msgr2=0x5647d841d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0e59c700 1 -- 172.21.15.31:0/3254526542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647d818bd80 con 0x5647d840d520 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.060+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9d0e59c700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d841d660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000fb30 tx=0x7f9d00009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761ad652df702e62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9cf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0001d070 con 0x5647d840d520 2023-10-06T21:31:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9d0003d020 con 0x5647d840d520 2023-10-06T21:31:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d040044c0 con 0x5647d840d520 2023-10-06T21:31:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0000fca0 con 0x5647d840d520 2023-10-06T21:31:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9d282dab80 10 client.17073 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.061+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5647d818a250 con 0x5647d840d520 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d040044c0 con 0x5647d840d520 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9d00024b90 con 0x5647d840d520 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 10 client.17073 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 20 client.17073 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d847c940 0x5647d847ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 4 client.17073 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:31:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647d8359ad0 con 0x5647d847c940 2023-10-06T21:31:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.062+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d847c940 0x5647d847ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.063+0000 7f9d0dd9b700 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d847c940 0x5647d847ed60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda31f538a327e1e server_cookie=688c2ea2f464979a in_seq=0 out_seq=0 2023-10-06T21:31:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.063+0000 7f9cf5ffb700 10 client.17073 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.089+0000 7f9cf5ffb700 1 -- 172.21.15.31:0/3254526542 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647d8359ad0 con 0x5647d847c940 2023-10-06T21:31:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.089+0000 7f9cf5ffb700 10 client.17073 handle_command_reply: tid=0 2023-10-06T21:31:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.089+0000 7f9d282dab80 1 client.17073 shutdown 2023-10-06T21:31:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 msgr2=0x5647d841d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d841d660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000fb30 tx=0x7f9d00009310 comp rx=0 tx=0).stop 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 shutdown_connections 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647d84183d0 0x5647d847adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647d840cc00 0x5647d841d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5647d847c940 0x5647d847ed60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 --2- 172.21.15.31:0/3254526542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5647d840d520 0x5647d841d660 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9cf5ffb700 0 client.17073 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 >> 172.21.15.31:0/3254526542 conn(0x5647d8382200 msgr2=0x5647d8410390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 shutdown_connections 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 1 -- 172.21.15.31:0/3254526542 wait complete. 2023-10-06T21:31:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.090+0000 7f9d282dab80 20 client.17073 trim_cache size 0 max 16384 2023-10-06T21:31:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:31:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:31:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:31:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 msgr2=0x7f9cf80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 0x7f9cf80a3ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d08095da0 tx=0x7f9d0800df10 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 msgr2=0x7f9d101a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007f10 tx=0x7f9d0800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 shutdown_connections 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d10106f60 0x7f9d101a5130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d100682d0 0x7f9d10100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9cf80a15f0 0x7f9cf80a3ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 --2- 172.21.15.31:0/2136716165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d10068c90 0x7f9d10100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 >> 172.21.15.31:0/2136716165 conn(0x7f9d10076b70 msgr2=0x7f9d100fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 shutdown_connections 2023-10-06T21:31:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:35.091+0000 7f9d149ff700 1 -- 172.21.15.31:0/2136716165 wait complete. 2023-10-06T21:31:35.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173803 1.62 MB/sec execute 445 sec latency 23.209 ms 2023-10-06T21:31:36.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173891 1.61 MB/sec execute 446 sec latency 747.557 ms 2023-10-06T21:31:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:36 smithi142 ceph-mon[102951]: pgmap v333: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:36 smithi031 ceph-mon[98539]: pgmap v333: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:36 smithi146 ceph-mon[102809]: pgmap v333: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:37.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173891 1.61 MB/sec execute 447 sec latency 1747.714 ms 2023-10-06T21:31:38.358 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173891 1.61 MB/sec execute 448 sec latency 2747.848 ms 2023-10-06T21:31:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:38 smithi142 ceph-mon[102951]: pgmap v334: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:38 smithi031 ceph-mon[98539]: pgmap v334: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:38 smithi146 ceph-mon[102809]: pgmap v334: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 698 KiB/s wr, 72 op/s 2023-10-06T21:31:39.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173891 1.60 MB/sec execute 449 sec latency 3748.001 ms 2023-10-06T21:31:40.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:40 smithi031 ceph-mon[98539]: pgmap v335: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1023 KiB/s wr, 100 op/s 2023-10-06T21:31:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:40 smithi146 ceph-mon[102809]: pgmap v335: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1023 KiB/s wr, 100 op/s 2023-10-06T21:31:40.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173958 1.60 MB/sec execute 450 sec latency 3846.608 ms 2023-10-06T21:31:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:40 smithi142 ceph-mon[102951]: pgmap v335: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1023 KiB/s wr, 100 op/s 2023-10-06T21:31:41.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173958 1.60 MB/sec execute 451 sec latency 1555.876 ms 2023-10-06T21:31:42.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173958 1.59 MB/sec execute 452 sec latency 2556.027 ms 2023-10-06T21:31:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:42 smithi142 ceph-mon[102951]: pgmap v336: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 714 KiB/s wr, 66 op/s 2023-10-06T21:31:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:42 smithi031 ceph-mon[98539]: pgmap v336: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 714 KiB/s wr, 66 op/s 2023-10-06T21:31:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:42 smithi146 ceph-mon[102809]: pgmap v336: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 714 KiB/s wr, 66 op/s 2023-10-06T21:31:43.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173958 1.59 MB/sec execute 453 sec latency 3556.181 ms 2023-10-06T21:31:44.359 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173958 1.59 MB/sec execute 454 sec latency 4556.312 ms 2023-10-06T21:31:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:44 smithi142 ceph-mon[102951]: pgmap v337: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 743 KiB/s wr, 75 op/s 2023-10-06T21:31:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:44 smithi031 ceph-mon[98539]: pgmap v337: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 743 KiB/s wr, 75 op/s 2023-10-06T21:31:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:44 smithi146 ceph-mon[102809]: pgmap v337: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 743 KiB/s wr, 75 op/s 2023-10-06T21:31:45.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173983 1.58 MB/sec execute 455 sec latency 4653.342 ms 2023-10-06T21:31:46.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173983 1.58 MB/sec execute 456 sec latency 1733.614 ms 2023-10-06T21:31:46.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:46 smithi142 ceph-mon[102951]: pgmap v338: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:46 smithi031 ceph-mon[98539]: pgmap v338: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:46 smithi146 ceph-mon[102809]: pgmap v338: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:47.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173983 1.58 MB/sec execute 457 sec latency 2733.743 ms 2023-10-06T21:31:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 -- 172.21.15.31:0/3008665499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c01061f0 msgr2=0x7ff8c010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/3008665499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c01061f0 0x7ff8c010dd70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800b0a0 tx=0x7ff8b800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 -- 172.21.15.31:0/3008665499 shutdown_connections 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/3008665499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c01061f0 0x7ff8c010dd70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/3008665499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01010d0 0x7ff8c01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/3008665499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c0101a80 0x7ff8c0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 -- 172.21.15.31:0/3008665499 >> 172.21.15.31:0/3008665499 conn(0x7ff8c00fc930 msgr2=0x7ff8c00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 -- 172.21.15.31:0/3008665499 shutdown_connections 2023-10-06T21:31:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.254+0000 7ff8c49ff700 1 -- 172.21.15.31:0/3008665499 wait complete. 2023-10-06T21:31:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.255+0000 7ff8c49ff700 1 Processor -- start 2023-10-06T21:31:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.255+0000 7ff8c49ff700 1 -- start start 2023-10-06T21:31:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c01010d0 0x7ff8c01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01061f0 0x7ff8c0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019e810 con 0x7ff8c019dd90 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019e980 con 0x7ff8c01010d0 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8c49ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019eb40 con 0x7ff8c01061f0 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43670/0 (socket says 172.21.15.31:43670) 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8bed9d700 1 -- 172.21.15.31:0/287765176 learned_addr learned my addr 172.21.15.31:0/287765176 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8be59c700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c01010d0 0x7ff8c01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.256+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01061f0 0x7ff8c0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 -- 172.21.15.31:0/287765176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01061f0 msgr2=0x7ff8c0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01061f0 0x7ff8c0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 -- 172.21.15.31:0/287765176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c01010d0 msgr2=0x7ff8c01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c01010d0 0x7ff8c01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 -- 172.21.15.31:0/287765176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009d00 con 0x7ff8c019dd90 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8007990 tx=0x7ff8b80079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8b77fe700 1 -- 172.21.15.31:0/287765176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800e040 con 0x7ff8c019dd90 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8b77fe700 1 -- 172.21.15.31:0/287765176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b8004dc0 con 0x7ff8c019dd90 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c01a6040 con 0x7ff8c019dd90 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8b77fe700 1 -- 172.21.15.31:0/287765176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b801b7c0 con 0x7ff8c019dd90 2023-10-06T21:31:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.257+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01a64c0 con 0x7ff8c019dd90 2023-10-06T21:31:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.260+0000 7ff8b77fe700 1 -- 172.21.15.31:0/287765176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff8b801b920 con 0x7ff8c019dd90 2023-10-06T21:31:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8d8416b80 1 Processor -- start 2023-10-06T21:31:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8b77fe700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 0x7ff8ac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8d8416b80 1 -- start start 2023-10-06T21:31:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8b77fe700 1 -- 172.21.15.31:0/287765176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff8b80ceba0 con 0x7ff8c019dd90 2023-10-06T21:31:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 0x7ff8ac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.262+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d903400 con 0x55ac0d999c00 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d8f4bc0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d8f24e0 con 0x55ac0d99a520 2023-10-06T21:31:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43686/0 (socket says 172.21.15.31:43686) 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 -- 172.21.15.31:0/1143161318 learned_addr learned my addr 172.21.15.31:0/1143161318 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 -- 172.21.15.31:0/1143161318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 msgr2=0x55ac0d9a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 -- 172.21.15.31:0/1143161318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d9a53d0 msgr2=0x55ac0d9a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8be59c700 1 -- 172.21.15.31:0/1143161318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac0d718d80 con 0x55ac0d999c00 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.263+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 0x7ff8ac0a3ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8c019aa00 tx=0x7ff8a80103d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.264+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bd70 tx=0x7ff8b00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f52373e2035f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.264+0000 7ff8b4ff9700 1 -- 172.21.15.31:0/1143161318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0010e10 con 0x55ac0d999c00 2023-10-06T21:31:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.264+0000 7ff8b4ff9700 1 -- 172.21.15.31:0/1143161318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b0015070 con 0x55ac0d999c00 2023-10-06T21:31:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.264+0000 7ff8b4ff9700 1 -- 172.21.15.31:0/1143161318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00116a0 con 0x55ac0d999c00 2023-10-06T21:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1143161318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 msgr2=0x55ac0d999fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bd70 tx=0x7ff8b00106b0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1143161318 shutdown_connections 2023-10-06T21:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d999c00 0x55ac0d999fe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1143161318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1143161318 >> 172.21.15.31:0/1143161318 conn(0x55ac0d90f200 msgr2=0x55ac0d996080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1143161318 shutdown_connections 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1143161318 wait complete. 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.265+0000 7ff8d8416b80 1 -- start start 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 Processor -- start 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d8eddd0 con 0x55ac0d99a520 2023-10-06T21:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d88a6b0 con 0x55ac0d999c00 2023-10-06T21:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d88a350 con 0x55ac0d9a53d0 2023-10-06T21:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8d8416b80 10 client.0 _ll_register_callbacks cb 0x55ac0d6e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43694/0 (socket says 172.21.15.31:43694) 2023-10-06T21:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8b7fff700 1 -- 172.21.15.31:0/1447852813 learned_addr learned my addr 172.21.15.31:0/1447852813 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.266+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 -- 172.21.15.31:0/1447852813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 msgr2=0x55ac0d9a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 -- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 msgr2=0x55ac0d9a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 -- 172.21.15.31:0/1447852813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b000b9a0 con 0x55ac0d99a520 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013710 tx=0x7ff8a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc095f32e6700fef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8004a30 con 0x55ac0d99a520 2023-10-06T21:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac0d98dd50 con 0x55ac0d99a520 2023-10-06T21:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.267+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff8a8019040 con 0x55ac0d99a520 2023-10-06T21:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a80132e0 con 0x55ac0d99a520 2023-10-06T21:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 10 client.17085 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ac0d701c00 con 0x55ac0d99a520 2023-10-06T21:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ac0d98dd50 con 0x55ac0d99a520 2023-10-06T21:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff8a8011440 con 0x55ac0d99a520 2023-10-06T21:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 10 client.17085 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 20 client.17085 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0d9b50b0 0x55ac0d9b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 4 client.17085 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.268+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac0d8e6db0 con 0x55ac0d9b50b0 2023-10-06T21:31:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.269+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0d9b50b0 0x55ac0d9b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.269+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0d9b50b0 0x55ac0d9b74d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e37f0d1dc67735a server_cookie=457864dc306b0973 in_seq=0 out_seq=0 2023-10-06T21:31:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.269+0000 7ff8a5ffb700 10 client.17085 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.270+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1447852813 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac0d8e6db0 con 0x55ac0d9b50b0 2023-10-06T21:31:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.270+0000 7ff8a5ffb700 10 client.17085 handle_command_reply: tid=0 2023-10-06T21:31:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.270+0000 7ff8d8416b80 1 client.17085 shutdown 2023-10-06T21:31:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 msgr2=0x55ac0d9a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013710 tx=0x7ff8a80047c0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 shutdown_connections 2023-10-06T21:31:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0d9b50b0 0x55ac0d9b74d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1447852813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d99a520 0x55ac0d9a2680 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 >> 172.21.15.31:0/1447852813 conn(0x55ac0d90f200 msgr2=0x55ac0d995c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8a5ffb700 0 client.17085 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 shutdown_connections 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1447852813 wait complete. 2023-10-06T21:31:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.271+0000 7ff8d8416b80 20 client.17085 trim_cache size 0 max 16384 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.286+0000 7ff8d8416b80 1 Processor -- start 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.286+0000 7ff8d8416b80 1 -- start start 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d99cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d76e8c0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d99dfb0 con 0x55ac0d999c00 2023-10-06T21:31:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0d98dbb0 con 0x55ac0d99a520 2023-10-06T21:31:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43696/0 (socket says 172.21.15.31:43696) 2023-10-06T21:31:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 -- 172.21.15.31:0/635469561 learned_addr learned my addr 172.21.15.31:0/635469561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d99cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 -- 172.21.15.31:0/635469561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 msgr2=0x55ac0d9a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 -- 172.21.15.31:0/635469561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 msgr2=0x55ac0d99cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d99cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.287+0000 7ff8bed9d700 1 -- 172.21.15.31:0/635469561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac0d73f1f0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.288+0000 7ff8be59c700 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d99cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.288+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff8b809b840 tx=0x7ff8b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad88ed033adfcb0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.288+0000 7ff8b57fa700 1 -- 172.21.15.31:0/635469561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b809bb40 con 0x55ac0d9a53d0 2023-10-06T21:31:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.288+0000 7ff8b57fa700 1 -- 172.21.15.31:0/635469561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b809fc90 con 0x55ac0d9a53d0 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.288+0000 7ff8b57fa700 1 -- 172.21.15.31:0/635469561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b801be30 con 0x55ac0d9a53d0 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- 172.21.15.31:0/635469561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 msgr2=0x55ac0d9a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff8b809b840 tx=0x7ff8b8009040 comp rx=0 tx=0).stop 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- 172.21.15.31:0/635469561 shutdown_connections 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9a2bf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d99cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/635469561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- 172.21.15.31:0/635469561 >> 172.21.15.31:0/635469561 conn(0x55ac0d90f200 msgr2=0x55ac0d99d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- 172.21.15.31:0/635469561 shutdown_connections 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- 172.21.15.31:0/635469561 wait complete. 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 -- start start 2023-10-06T21:31:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.289+0000 7ff8d8416b80 1 Processor -- start 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0da08060 con 0x55ac0d9a53d0 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0da081d0 con 0x55ac0d999c00 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0da08390 con 0x55ac0d99a520 2023-10-06T21:31:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8d8416b80 10 client.0 _ll_register_callbacks cb 0x7ff8a8010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8be59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:48940/0 (socket says 172.21.15.31:48940) 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8be59c700 1 -- 172.21.15.31:0/1302480425 learned_addr learned my addr 172.21.15.31:0/1302480425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43702/0 (socket says 172.21.15.31:43702) 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.290+0000 7ff8b7fff700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 -- 172.21.15.31:0/1302480425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 msgr2=0x55ac0d9aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 -- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 msgr2=0x55ac0d9a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 -- 172.21.15.31:0/1302480425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac0d718d80 con 0x55ac0d9a53d0 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8bed9d700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55ac0d9b4fc0 tx=0x7ff8b8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fa625650b41eae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80a3e40 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8b0015070 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff8b8067380 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b809bda0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8d8416b80 10 client.17091 fetch_fsmap learned FSMap version 39 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7ff8c01a64c0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.291+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b0015070 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff8b80a39b0 con 0x55ac0d9a53d0 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 10 client.17091 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 20 client.17091 populate_metadata read hostname 'smithi031' 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0da09770 0x55ac0da0bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 4 client.17091 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ac0d8e6ad0 con 0x55ac0da09770 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.292+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0da09770 0x55ac0da0bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.293+0000 7ff8be59c700 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0da09770 0x55ac0da0bb90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1312bba219a38d5 server_cookie=d602a3bdb690faef in_seq=0 out_seq=0 2023-10-06T21:31:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.293+0000 7ff8a5ffb700 10 client.17091 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.322+0000 7ff8a5ffb700 1 -- 172.21.15.31:0/1302480425 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ac0d8e6ad0 con 0x55ac0da09770 2023-10-06T21:31:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.322+0000 7ff8a5ffb700 10 client.17091 handle_command_reply: tid=0 2023-10-06T21:31:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.322+0000 7ff8d8416b80 1 client.17091 shutdown 2023-10-06T21:31:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 msgr2=0x55ac0d9aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55ac0d9b4fc0 tx=0x7ff8b8009960 comp rx=0 tx=0).stop 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 shutdown_connections 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ac0da09770 0x55ac0da0bb90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac0d9a53d0 0x55ac0d9aa980 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac0d999c00 0x55ac0d9a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 --2- 172.21.15.31:0/1302480425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac0d99a520 0x55ac0d9aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 >> 172.21.15.31:0/1302480425 conn(0x55ac0d90f200 msgr2=0x55ac0d99d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8a5ffb700 0 client.17091 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 shutdown_connections 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 1 -- 172.21.15.31:0/1302480425 wait complete. 2023-10-06T21:31:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.323+0000 7ff8d8416b80 20 client.17091 trim_cache size 0 max 16384 2023-10-06T21:31:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:31:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:31:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:31:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 msgr2=0x7ff8ac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 0x7ff8ac0a3ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8c019aa00 tx=0x7ff8a80103d0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 msgr2=0x7ff8c019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8007990 tx=0x7ff8b80079c0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 shutdown_connections 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff8ac0a15f0 0x7ff8ac0a3ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c019dd90 0x7ff8c019e210 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c01010d0 0x7ff8c01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 --2- 172.21.15.31:0/287765176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c01061f0 0x7ff8c0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 >> 172.21.15.31:0/287765176 conn(0x7ff8c00fc930 msgr2=0x7ff8c00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 shutdown_connections 2023-10-06T21:31:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:31:48.324+0000 7ff8c49ff700 1 -- 172.21.15.31:0/287765176 wait complete. 2023-10-06T21:31:48.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 173983 1.57 MB/sec execute 458 sec latency 3733.850 ms 2023-10-06T21:31:48.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:48 smithi142 ceph-mon[102951]: pgmap v339: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:48 smithi031 ceph-mon[98539]: pgmap v339: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:48 smithi146 ceph-mon[102809]: pgmap v339: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 354 KiB/s wr, 36 op/s 2023-10-06T21:31:49.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 174087 1.57 MB/sec execute 459 sec latency 4248.582 ms 2023-10-06T21:31:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:50 smithi146 ceph-mon[102809]: pgmap v340: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 380 KiB/s wr, 44 op/s 2023-10-06T21:31:50.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 174261 1.57 MB/sec execute 460 sec latency 16.897 ms 2023-10-06T21:31:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:50 smithi142 ceph-mon[102951]: pgmap v340: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 380 KiB/s wr, 44 op/s 2023-10-06T21:31:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:50 smithi031 ceph-mon[98539]: pgmap v340: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 380 KiB/s wr, 44 op/s 2023-10-06T21:31:51.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 174479 1.57 MB/sec execute 461 sec latency 21.359 ms 2023-10-06T21:31:52.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 174809 1.56 MB/sec execute 462 sec latency 18.909 ms 2023-10-06T21:31:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:52 smithi142 ceph-mon[102951]: pgmap v341: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-10-06T21:31:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:52 smithi031 ceph-mon[98539]: pgmap v341: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-10-06T21:31:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:52 smithi146 ceph-mon[102809]: pgmap v341: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-10-06T21:31:53.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 175213 1.57 MB/sec execute 463 sec latency 21.233 ms 2023-10-06T21:31:54.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 175512 1.57 MB/sec execute 464 sec latency 28.522 ms 2023-10-06T21:31:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:54 smithi142 ceph-mon[102951]: pgmap v342: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 483 KiB/s wr, 61 op/s 2023-10-06T21:31:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:54 smithi031 ceph-mon[98539]: pgmap v342: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 483 KiB/s wr, 61 op/s 2023-10-06T21:31:54.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:54 smithi146 ceph-mon[102809]: pgmap v342: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 483 KiB/s wr, 61 op/s 2023-10-06T21:31:55.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 175882 1.56 MB/sec execute 465 sec latency 26.329 ms 2023-10-06T21:31:56.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 176234 1.56 MB/sec execute 466 sec latency 25.636 ms 2023-10-06T21:31:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:56 smithi142 ceph-mon[102951]: pgmap v343: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:56 smithi031 ceph-mon[98539]: pgmap v343: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:56 smithi146 ceph-mon[102809]: pgmap v343: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:57.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 176698 1.57 MB/sec execute 467 sec latency 17.401 ms 2023-10-06T21:31:58.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177121 1.57 MB/sec execute 468 sec latency 26.717 ms 2023-10-06T21:31:58.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:31:58 smithi142 ceph-mon[102951]: pgmap v344: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:31:58 smithi031 ceph-mon[98539]: pgmap v344: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:31:58 smithi146 ceph-mon[102809]: pgmap v344: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 454 KiB/s wr, 52 op/s 2023-10-06T21:31:59.360 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177383 1.58 MB/sec execute 469 sec latency 24.851 ms 2023-10-06T21:32:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:00 smithi146 ceph-mon[102809]: pgmap v345: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 897 KiB/s wr, 92 op/s 2023-10-06T21:32:00.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177436 1.57 MB/sec execute 470 sec latency 902.443 ms 2023-10-06T21:32:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:00 smithi142 ceph-mon[102951]: pgmap v345: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 897 KiB/s wr, 92 op/s 2023-10-06T21:32:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:00 smithi031 ceph-mon[98539]: pgmap v345: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 897 KiB/s wr, 92 op/s 2023-10-06T21:32:01.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177436 1.57 MB/sec execute 471 sec latency 1902.500 ms 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 -- 172.21.15.31:0/1330423100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c0682d0 msgr2=0x7fd75c068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 --2- 172.21.15.31:0/1330423100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c0682d0 0x7fd75c068750 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd748009990 tx=0x7fd748009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 -- 172.21.15.31:0/1330423100 shutdown_connections 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 --2- 172.21.15.31:0/1330423100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c068c90 0x7fd75c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 --2- 172.21.15.31:0/1330423100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c106f60 0x7fd75c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 --2- 172.21.15.31:0/1330423100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c0682d0 0x7fd75c068750 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.488+0000 7fd762bff700 1 -- 172.21.15.31:0/1330423100 >> 172.21.15.31:0/1330423100 conn(0x7fd75c076b70 msgr2=0x7fd75c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.489+0000 7fd762bff700 1 -- 172.21.15.31:0/1330423100 shutdown_connections 2023-10-06T21:32:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.489+0000 7fd762bff700 1 -- 172.21.15.31:0/1330423100 wait complete. 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 Processor -- start 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 -- start start 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c0682d0 0x7fd75c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c068c90 0x7fd75c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c19c070 con 0x7fd75c106f60 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c19c1e0 con 0x7fd75c068c90 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd762bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c19c3a0 con 0x7fd75c0682d0 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.490+0000 7fd76119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41462/0 (socket says 172.21.15.31:41462) 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 -- 172.21.15.31:0/1235150326 learned_addr learned my addr 172.21.15.31:0/1235150326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd75bfff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c068c90 0x7fd75c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76099b700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c0682d0 0x7fd75c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 -- 172.21.15.31:0/1235150326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c0682d0 msgr2=0x7fd75c196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c0682d0 0x7fd75c196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 -- 172.21.15.31:0/1235150326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c068c90 msgr2=0x7fd75c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c068c90 0x7fd75c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 -- 172.21.15.31:0/1235150326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480095f0 con 0x7fd75c106f60 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd76119c700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd754007f10 tx=0x7fd75400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd759ffb700 1 -- 172.21.15.31:0/1235150326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754017070 con 0x7fd75c106f60 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c19c680 con 0x7fd75c106f60 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.491+0000 7fd759ffb700 1 -- 172.21.15.31:0/1235150326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd75400f040 con 0x7fd75c106f60 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.492+0000 7fd759ffb700 1 -- 172.21.15.31:0/1235150326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754005710 con 0x7fd75c106f60 2023-10-06T21:32:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.492+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c1a3ec0 con 0x7fd75c106f60 2023-10-06T21:32:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.495+0000 7fd759ffb700 1 -- 172.21.15.31:0/1235150326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd754005870 con 0x7fd75c106f60 2023-10-06T21:32:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.496+0000 7fd776595b80 1 Processor -- start 2023-10-06T21:32:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.496+0000 7fd776595b80 1 -- start start 2023-10-06T21:32:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd759ffb700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad930 0x55afda2b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2b87e0 0x55afda2babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda212400 con 0x55afda2ad010 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda203bc0 con 0x55afda2ad930 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd776595b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda2014e0 con 0x55afda2b87e0 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd759ffb700 1 -- 172.21.15.31:0/1235150326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd7540cc9c0 con 0x7fd75c106f60 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd76099b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd75bfff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd76099b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41468/0 (socket says 172.21.15.31:41468) 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd76099b700 1 -- 172.21.15.31:0/3386026183 learned_addr learned my addr 172.21.15.31:0/3386026183 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd76119c700 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2b87e0 0x55afda2babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.497+0000 7fd75bfff700 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad930 0x55afda2b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 -- 172.21.15.31:0/3386026183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2b87e0 msgr2=0x55afda2babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2b87e0 0x55afda2babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 -- 172.21.15.31:0/3386026183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad930 msgr2=0x55afda2b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad930 0x55afda2b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 -- 172.21.15.31:0/3386026183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afda027d80 con 0x55afda2ad010 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd76099b700 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd750009cf0 tx=0x7fd7500103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891a8bfa375625b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd75bfff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd7480055d0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd746ffd700 1 -- 172.21.15.31:0/3386026183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750010c50 con 0x55afda2ad010 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd746ffd700 1 -- 172.21.15.31:0/3386026183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd750015070 con 0x55afda2ad010 2023-10-06T21:32:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.498+0000 7fd746ffd700 1 -- 172.21.15.31:0/3386026183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750011640 con 0x55afda2ad010 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 -- 172.21.15.31:0/3386026183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 msgr2=0x55afda2ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd750009cf0 tx=0x7fd7500103d0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 -- 172.21.15.31:0/3386026183 shutdown_connections 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2b87e0 0x55afda2babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad010 0x55afda2ad3f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 --2- 172.21.15.31:0/3386026183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad930 0x55afda2b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 -- 172.21.15.31:0/3386026183 >> 172.21.15.31:0/3386026183 conn(0x55afda21e200 msgr2=0x55afda2a4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 -- 172.21.15.31:0/3386026183 shutdown_connections 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.499+0000 7fd776595b80 1 -- 172.21.15.31:0/3386026183 wait complete. 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.500+0000 7fd776595b80 1 -- start start 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.500+0000 7fd776595b80 1 Processor -- start 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad010 0x55afda2b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 0x55afda2b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda1fcdd0 con 0x55afda2ad930 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda1996b0 con 0x55afda2b87e0 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda199350 con 0x55afda2ad010 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd776595b80 10 client.0 _ll_register_callbacks cb 0x55afd9ff67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd75bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd76099b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad010 0x55afda2b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd75bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41474/0 (socket says 172.21.15.31:41474) 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd75bfff700 1 -- 172.21.15.31:0/4078650573 learned_addr learned my addr 172.21.15.31:0/4078650573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.501+0000 7fd76119c700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 0x55afda2b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 -- 172.21.15.31:0/4078650573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad010 msgr2=0x55afda2b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad010 0x55afda2b81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 -- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 msgr2=0x55afda2b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 0x55afda2b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 -- 172.21.15.31:0/4078650573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd750009920 con 0x55afda2ad930 2023-10-06T21:32:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd76119c700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 0x55afda2b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd75bfff700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd748009990 tx=0x7fd748005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8439cd6eccd7409d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd73bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.502+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800fe80 con 0x55afda2ad930 2023-10-06T21:32:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afda29cd50 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd748005020 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7480051e0 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 10 client.17103 fetch_fsmap learned FSMap version 39 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55afda010c00 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd74803e020 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd748029470 con 0x55afda2ad930 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:32:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.503+0000 7fd776595b80 20 client.17103 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.504+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda2c83d0 0x55afda2ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.504+0000 7fd776595b80 4 client.17103 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.504+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afda1f5db0 con 0x55afda2c83d0 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.504+0000 7fd76099b700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda2c83d0 0x55afda2ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.505+0000 7fd73bfff700 10 client.17103 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.505+0000 7fd76099b700 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda2c83d0 0x55afda2ca7f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f54875fefb6fc31 server_cookie=5de7e2a8ce3100 in_seq=0 out_seq=0 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd73bfff700 1 -- 172.21.15.31:0/4078650573 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55afda1f5db0 con 0x55afda2c83d0 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd73bfff700 10 client.17103 handle_command_reply: tid=0 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 client.17103 shutdown 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 msgr2=0x55afda2b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd748009990 tx=0x7fd748005960 comp rx=0 tx=0).stop 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 shutdown_connections 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2b87e0 0x55afda2b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad010 0x55afda2b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda2c83d0 0x55afda2ca7f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 --2- 172.21.15.31:0/4078650573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2ad930 0x55afda2b53c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 >> 172.21.15.31:0/4078650573 conn(0x55afda21e200 msgr2=0x55afda2a4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd73bfff700 0 client.17103 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 shutdown_connections 2023-10-06T21:32:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.506+0000 7fd776595b80 1 -- 172.21.15.31:0/4078650573 wait complete. 2023-10-06T21:32:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.507+0000 7fd776595b80 20 client.17103 trim_cache size 0 max 16384 2023-10-06T21:32:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.522+0000 7fd776595b80 1 Processor -- start 2023-10-06T21:32:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.522+0000 7fd776595b80 1 -- start start 2023-10-06T21:32:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.522+0000 7fd776595b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.522+0000 7fd776595b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.522+0000 7fd776595b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd776595b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda07d8c0 con 0x55afda2b87e0 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd776595b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda2bf890 con 0x55afda2ad010 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd776595b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda2b5fd0 con 0x55afda2ad930 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41484/0 (socket says 172.21.15.31:41484) 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 -- 172.21.15.31:0/2765485024 learned_addr learned my addr 172.21.15.31:0/2765485024 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd75bfff700 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76099b700 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 -- 172.21.15.31:0/2765485024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 msgr2=0x55afda2b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 -- 172.21.15.31:0/2765485024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 msgr2=0x55afda2bf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.523+0000 7fd76119c700 1 -- 172.21.15.31:0/2765485024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afda04e1f0 con 0x55afda2b87e0 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.524+0000 7fd76119c700 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd754096250 tx=0x7fd75400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea61c9cbca8b39e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.524+0000 7fd7477fe700 1 -- 172.21.15.31:0/2765485024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400dc60 con 0x55afda2b87e0 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.524+0000 7fd7477fe700 1 -- 172.21.15.31:0/2765485024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd754022250 con 0x55afda2b87e0 2023-10-06T21:32:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.524+0000 7fd7477fe700 1 -- 172.21.15.31:0/2765485024 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7540ca500 con 0x55afda2b87e0 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- 172.21.15.31:0/2765485024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 msgr2=0x55afda2b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd754096250 tx=0x7fd75400b040 comp rx=0 tx=0).stop 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- 172.21.15.31:0/2765485024 shutdown_connections 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda2b4c60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 --2- 172.21.15.31:0/2765485024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- 172.21.15.31:0/2765485024 >> 172.21.15.31:0/2765485024 conn(0x55afda21e200 msgr2=0x55afda2b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- 172.21.15.31:0/2765485024 shutdown_connections 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- 172.21.15.31:0/2765485024 wait complete. 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 -- start start 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.525+0000 7fd776595b80 1 Processor -- start 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda31b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda2bc170 con 0x55afda2b87e0 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda31b730 con 0x55afda2ad010 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda31b870 con 0x55afda2ad930 2023-10-06T21:32:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd776595b80 10 client.0 _ll_register_callbacks cb 0x55afd9ff67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd76099b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd76119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda31b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd76099b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33002/0 (socket says 172.21.15.31:33002) 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd76099b700 1 -- 172.21.15.31:0/62289870 learned_addr learned my addr 172.21.15.31:0/62289870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.526+0000 7fd75bfff700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 -- 172.21.15.31:0/62289870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 msgr2=0x55afda2bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 -- 172.21.15.31:0/62289870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 msgr2=0x55afda31b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda31b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 -- 172.21.15.31:0/62289870 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c1a3ec0 con 0x55afda2ad010 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd76099b700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd750009cf0 tx=0x7fd750005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1a52cc41fdb5d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd73bfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750027570 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd750015070 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.527+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd750017070 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd750010a50 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 10 client.26816 fetch_fsmap learned FSMap version 39 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55afda026250 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd750015070 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd7500279e0 con 0x55afda2ad010 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 20 client.26816 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda31cfd0 0x55afda31f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 4 client.26816 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.528+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afda1f5ad0 con 0x55afda31cfd0 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.529+0000 7fd75bfff700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda31cfd0 0x55afda31f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.529+0000 7fd75bfff700 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda31cfd0 0x55afda31f3f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e0bf6882ec7feb server_cookie=caf2080f041961ff in_seq=0 out_seq=0 2023-10-06T21:32:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.529+0000 7fd73bfff700 10 client.26816 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd73bfff700 1 -- 172.21.15.31:0/62289870 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afda1f5ad0 con 0x55afda31cfd0 2023-10-06T21:32:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd73bfff700 10 client.26816 handle_command_reply: tid=0 2023-10-06T21:32:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 client.26816 shutdown 2023-10-06T21:32:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 msgr2=0x55afda2bb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd750009cf0 tx=0x7fd750005d30 comp rx=0 tx=0).stop 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 shutdown_connections 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55afda31cfd0 0x55afda31f3f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afda2b87e0 0x55afda31b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd73bfff700 0 client.26816 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afda2ad010 0x55afda2bb330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 --2- 172.21.15.31:0/62289870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afda2ad930 0x55afda2bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 >> 172.21.15.31:0/62289870 conn(0x55afda21e200 msgr2=0x55afda2b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 shutdown_connections 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 1 -- 172.21.15.31:0/62289870 wait complete. 2023-10-06T21:32:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.562+0000 7fd776595b80 20 client.26816 trim_cache size 0 max 16384 2023-10-06T21:32:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:32:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:32:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:32:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 msgr2=0x7fd74c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd7480055d0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 msgr2=0x7fd75c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd754007f10 tx=0x7fd75400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 shutdown_connections 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd75c106f60 0x7fd75c19ba30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd75c0682d0 0x7fd75c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 --2- 172.21.15.31:0/1235150326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c068c90 0x7fd75c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 >> 172.21.15.31:0/1235150326 conn(0x7fd75c076b70 msgr2=0x7fd75c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 shutdown_connections 2023-10-06T21:32:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:01.563+0000 7fd762bff700 1 -- 172.21.15.31:0/1235150326 wait complete. 2023-10-06T21:32:02.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177436 1.57 MB/sec execute 472 sec latency 2902.575 ms 2023-10-06T21:32:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:02 smithi142 ceph-mon[102951]: pgmap v346: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 871 KiB/s wr, 84 op/s 2023-10-06T21:32:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:02 smithi031 ceph-mon[98539]: pgmap v346: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 871 KiB/s wr, 84 op/s 2023-10-06T21:32:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:02 smithi146 ceph-mon[102809]: pgmap v346: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 871 KiB/s wr, 84 op/s 2023-10-06T21:32:03.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177436 1.56 MB/sec execute 473 sec latency 3902.732 ms 2023-10-06T21:32:04.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177436 1.56 MB/sec execute 474 sec latency 4902.861 ms 2023-10-06T21:32:04.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:04 smithi142 ceph-mon[102951]: pgmap v347: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2023-10-06T21:32:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:04 smithi031 ceph-mon[98539]: pgmap v347: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2023-10-06T21:32:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:04 smithi146 ceph-mon[102809]: pgmap v347: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2023-10-06T21:32:05.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177503 1.56 MB/sec execute 475 sec latency 5001.274 ms 2023-10-06T21:32:06.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177503 1.55 MB/sec execute 476 sec latency 1544.803 ms 2023-10-06T21:32:06.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:06 smithi142 ceph-mon[102951]: pgmap v348: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:06 smithi031 ceph-mon[98539]: pgmap v348: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:06 smithi146 ceph-mon[102809]: pgmap v348: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:07.361 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177503 1.55 MB/sec execute 477 sec latency 2544.942 ms 2023-10-06T21:32:08.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177503 1.55 MB/sec execute 478 sec latency 3545.129 ms 2023-10-06T21:32:08.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:08 smithi142 ceph-mon[102951]: pgmap v349: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:08 smithi031 ceph-mon[98539]: pgmap v349: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:08 smithi146 ceph-mon[102809]: pgmap v349: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 604 KiB/s wr, 61 op/s 2023-10-06T21:32:09.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177503 1.54 MB/sec execute 479 sec latency 4545.222 ms 2023-10-06T21:32:10.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177676 1.54 MB/sec execute 480 sec latency 4641.640 ms 2023-10-06T21:32:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:10 smithi142 ceph-mon[102951]: pgmap v350: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 679 KiB/s wr, 83 op/s 2023-10-06T21:32:10.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:10 smithi142 ceph-mon[102951]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:32:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:10 smithi031 ceph-mon[98539]: pgmap v350: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 679 KiB/s wr, 83 op/s 2023-10-06T21:32:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:10 smithi031 ceph-mon[98539]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:32:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:10 smithi146 ceph-mon[102809]: pgmap v350: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 679 KiB/s wr, 83 op/s 2023-10-06T21:32:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:10 smithi146 ceph-mon[102809]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:32:11.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 177843 1.54 MB/sec execute 481 sec latency 19.628 ms 2023-10-06T21:32:12.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 178103 1.54 MB/sec execute 482 sec latency 22.786 ms 2023-10-06T21:32:12.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:12 smithi142 ceph-mon[102951]: pgmap v352: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 284 KiB/s wr, 52 op/s 2023-10-06T21:32:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:12 smithi031 ceph-mon[98539]: pgmap v352: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 284 KiB/s wr, 52 op/s 2023-10-06T21:32:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:12 smithi146 ceph-mon[102809]: pgmap v352: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 284 KiB/s wr, 52 op/s 2023-10-06T21:32:13.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 178438 1.54 MB/sec execute 483 sec latency 18.984 ms 2023-10-06T21:32:14.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 178804 1.54 MB/sec execute 484 sec latency 23.986 ms 2023-10-06T21:32:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:14 smithi142 ceph-mon[102951]: pgmap v353: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 481 KiB/s wr, 67 op/s 2023-10-06T21:32:14.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:14 smithi142 ceph-mon[102951]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:32:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:14 smithi031 ceph-mon[98539]: pgmap v353: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 481 KiB/s wr, 67 op/s 2023-10-06T21:32:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:14 smithi031 ceph-mon[98539]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:32:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:14 smithi146 ceph-mon[102809]: pgmap v353: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 481 KiB/s wr, 67 op/s 2023-10-06T21:32:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:14 smithi146 ceph-mon[102809]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:32:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 -- 172.21.15.31:0/791753505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0980105e20 msgr2=0x7f09800682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 --2- 172.21.15.31:0/791753505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0980105e20 0x7f09800682d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0970009990 tx=0x7f0970009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 -- 172.21.15.31:0/791753505 shutdown_connections 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 --2- 172.21.15.31:0/791753505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09801038b0 0x7f0980103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 --2- 172.21.15.31:0/791753505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0980105e20 0x7f09800682d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 --2- 172.21.15.31:0/791753505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 0x7f0980068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 -- 172.21.15.31:0/791753505 >> 172.21.15.31:0/791753505 conn(0x7f0980076b70 msgr2=0x7f0980076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 -- 172.21.15.31:0/791753505 shutdown_connections 2023-10-06T21:32:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.730+0000 7f0985fff700 1 -- 172.21.15.31:0/791753505 wait complete. 2023-10-06T21:32:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.731+0000 7f0985fff700 1 Processor -- start 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.731+0000 7f0985fff700 1 -- start start 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 0x7f098019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801a2600 con 0x7f0980105e20 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801a2770 con 0x7f09801038b0 2023-10-06T21:32:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f0985fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801a2930 con 0x7f0980068810 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59438/0 (socket says 172.21.15.31:59438) 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50812/0 (socket says 172.21.15.31:50812) 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097ffff700 1 -- 172.21.15.31:0/3101232573 learned_addr learned my addr 172.21.15.31:0/3101232573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.732+0000 7f097f7fe700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 0x7f098019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 -- 172.21.15.31:0/3101232573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 msgr2=0x7f098019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 0x7f098019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 -- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 msgr2=0x7f09801a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 -- 172.21.15.31:0/3101232573 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700095f0 con 0x7f09801038b0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097ffff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097effd700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f097400d950 tx=0x7f097400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097cff9700 1 -- 172.21.15.31:0/3101232573 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400f890 con 0x7f09801038b0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097cff9700 1 -- 172.21.15.31:0/3101232573 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f097400fe80 con 0x7f09801038b0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f097cff9700 1 -- 172.21.15.31:0/3101232573 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400e3b0 con 0x7f09801038b0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.733+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09801a2c10 con 0x7f09801038b0 2023-10-06T21:32:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.734+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09801aa450 con 0x7f09801038b0 2023-10-06T21:32:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.737+0000 7f097cff9700 1 -- 172.21.15.31:0/3101232573 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f097400f9f0 con 0x7f09801038b0 2023-10-06T21:32:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 Processor -- start 2023-10-06T21:32:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 -- start start 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f097cff9700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 0x7f09680a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d991c00 0x56483d991fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d99f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8fb400 con 0x56483d991c00 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8ecbc0 con 0x56483d99d3d0 2023-10-06T21:32:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.739+0000 7f09999f1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8ea4e0 con 0x56483d992520 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097cff9700 1 -- 172.21.15.31:0/3101232573 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f09740cc8b0 con 0x7f09801038b0 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097ffff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 0x7f09680a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36680/0 (socket says 172.21.15.31:36680) 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 -- 172.21.15.31:0/2673896708 learned_addr learned my addr 172.21.15.31:0/2673896708 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097f7fe700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d991c00 0x56483d991fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097ffff700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d99f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 -- 172.21.15.31:0/2673896708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 msgr2=0x56483d99f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d99f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 -- 172.21.15.31:0/2673896708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d991c00 msgr2=0x56483d991fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d991c00 0x56483d991fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097effd700 1 -- 172.21.15.31:0/2673896708 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483d710d80 con 0x56483d992520 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.740+0000 7f097ffff700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d99f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.741+0000 7f097effd700 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09740a1620 tx=0x7f0974099380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ae0281db96ac6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.741+0000 7f097ffff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 0x7f09680a3b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0978009c10 tx=0x7f0978009920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.741+0000 7f0965ffb700 1 -- 172.21.15.31:0/2673896708 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400fca0 con 0x56483d992520 2023-10-06T21:32:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.741+0000 7f0965ffb700 1 -- 172.21.15.31:0/2673896708 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0974099ec0 con 0x56483d992520 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.741+0000 7f0965ffb700 1 -- 172.21.15.31:0/2673896708 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974097b60 con 0x56483d992520 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 -- 172.21.15.31:0/2673896708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 msgr2=0x56483d99ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09740a1620 tx=0x7f0974099380 comp rx=0 tx=0).stop 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 -- 172.21.15.31:0/2673896708 shutdown_connections 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d99f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d991c00 0x56483d991fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 --2- 172.21.15.31:0/2673896708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99ce90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 -- 172.21.15.31:0/2673896708 >> 172.21.15.31:0/2673896708 conn(0x56483d907200 msgr2=0x56483d98e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 -- 172.21.15.31:0/2673896708 shutdown_connections 2023-10-06T21:32:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.742+0000 7f09999f1b80 1 -- 172.21.15.31:0/2673896708 wait complete. 2023-10-06T21:32:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.743+0000 7f09999f1b80 1 -- start start 2023-10-06T21:32:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.743+0000 7f09999f1b80 1 Processor -- start 2023-10-06T21:32:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 0x56483d99a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d99abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8e5dd0 con 0x56483d99d3d0 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8826b0 con 0x56483d991c00 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d882350 con 0x56483d992520 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f09999f1b80 10 client.0 _ll_register_callbacks cb 0x56483d6df7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 0x56483d99a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 0x56483d99a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50826/0 (socket says 172.21.15.31:50826) 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36688/0 (socket says 172.21.15.31:36688) 2023-10-06T21:32:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097f7fe700 1 -- 172.21.15.31:0/1067975437 learned_addr learned my addr 172.21.15.31:0/1067975437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097ffff700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d99abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 -- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 msgr2=0x56483d99a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 0x56483d99a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 -- 172.21.15.31:0/1067975437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 msgr2=0x56483d99abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.744+0000 7f097effd700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d99abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f097effd700 1 -- 172.21.15.31:0/1067975437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09801aa450 con 0x56483d992520 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f097effd700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09740a1620 tx=0x7f097400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2301c63406f08e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f095effd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:32:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740997c0 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f097409d910 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56483d985d50 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0974097b60 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f09999f1b80 10 client.26773 fetch_fsmap learned FSMap version 39 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.745+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56483d6f9c00 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56483d985d50 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0974099a20 con 0x56483d992520 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:32:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 10 client.26773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 20 client.26773 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483d9aca70 0x56483d9aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 4 client.26773 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56483d8dedb0 con 0x56483d9aca70 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.746+0000 7f097f7fe700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483d9aca70 0x56483d9aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.747+0000 7f095effd700 10 client.26773 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.747+0000 7f097f7fe700 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483d9aca70 0x56483d9aee90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d62f5e542a8a16 server_cookie=b9abf90f2c2bb8a0 in_seq=0 out_seq=0 2023-10-06T21:32:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f095effd700 1 -- 172.21.15.31:0/1067975437 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56483d8dedb0 con 0x56483d9aca70 2023-10-06T21:32:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f095effd700 10 client.26773 handle_command_reply: tid=0 2023-10-06T21:32:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 client.26773 shutdown 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 msgr2=0x56483d99a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09740a1620 tx=0x7f097400bf90 comp rx=0 tx=0).stop 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 shutdown_connections 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483d9aca70 0x56483d9aee90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d99abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d991c00 0x56483d99a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f095effd700 0 client.26773 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 --2- 172.21.15.31:0/1067975437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d992520 0x56483d99a690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 >> 172.21.15.31:0/1067975437 conn(0x56483d907200 msgr2=0x56483d98dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 shutdown_connections 2023-10-06T21:32:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 1 -- 172.21.15.31:0/1067975437 wait complete. 2023-10-06T21:32:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.748+0000 7f09999f1b80 20 client.26773 trim_cache size 0 max 16384 2023-10-06T21:32:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 Processor -- start 2023-10-06T21:32:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 -- start start 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d99aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d7668c0 con 0x56483d99d3d0 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d98a860 con 0x56483d992520 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.765+0000 7f09999f1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d994ff0 con 0x56483d991c00 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50840/0 (socket says 172.21.15.31:50840) 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097effd700 1 -- 172.21.15.31:0/4040553106 learned_addr learned my addr 172.21.15.31:0/4040553106 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59460/0 (socket says 172.21.15.31:59460) 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097f7fe700 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d99aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 -- 172.21.15.31:0/4040553106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 msgr2=0x56483d99aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d99aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 -- 172.21.15.31:0/4040553106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 msgr2=0x56483d999030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097ffff700 1 -- 172.21.15.31:0/4040553106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483d7371f0 con 0x56483d99d3d0 2023-10-06T21:32:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.766+0000 7f097effd700 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.767+0000 7f097ffff700 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f097800edc0 tx=0x7f0978017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfbde348385e7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.767+0000 7f09667fc700 1 -- 172.21.15.31:0/4040553106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978007e90 con 0x56483d99d3d0 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.767+0000 7f09667fc700 1 -- 172.21.15.31:0/4040553106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f097800f040 con 0x56483d99d3d0 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.767+0000 7f09667fc700 1 -- 172.21.15.31:0/4040553106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978018960 con 0x56483d99d3d0 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- 172.21.15.31:0/4040553106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 msgr2=0x56483d997400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f097800edc0 tx=0x7f0978017000 comp rx=0 tx=0).stop 2023-10-06T21:32:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- 172.21.15.31:0/4040553106 shutdown_connections 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d99d3d0 0x56483d997400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d99aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 --2- 172.21.15.31:0/4040553106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d992520 0x56483d999030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- 172.21.15.31:0/4040553106 >> 172.21.15.31:0/4040553106 conn(0x56483d907200 msgr2=0x56483d998920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- 172.21.15.31:0/4040553106 shutdown_connections 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- 172.21.15.31:0/4040553106 wait complete. 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.768+0000 7f09999f1b80 1 -- start start 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 Processor -- start 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d9a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d9ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d9a3f60 con 0x56483d992520 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d9ffde0 con 0x56483d99d3d0 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d9fff20 con 0x56483d991c00 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.769+0000 7f09999f1b80 10 client.0 _ll_register_callbacks cb 0x7f09780097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d9a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59466/0 (socket says 172.21.15.31:59466) 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 -- 172.21.15.31:0/662098778 learned_addr learned my addr 172.21.15.31:0/662098778 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097ffff700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d9ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 -- 172.21.15.31:0/662098778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 msgr2=0x56483d9a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d9a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 -- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 msgr2=0x56483d9ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d9ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 -- 172.21.15.31:0/662098778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56483d6f9c00 con 0x56483d992520 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f097effd700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f09740994c0 tx=0x7f0974097d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b8036148ec3283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.770+0000 7f095effd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400fca0 con 0x56483d992520 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0974099ec0 con 0x56483d992520 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0974059320 con 0x56483d992520 2023-10-06T21:32:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0974098cb0 con 0x56483d992520 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 10 client.17127 fetch_fsmap learned FSMap version 39 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f0978007500 con 0x56483d992520 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0974099ec0 con 0x56483d992520 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f09740a16c0 con 0x56483d992520 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 20 client.17127 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483da01300 0x56483da03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 4 client.17127 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:32:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.771+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56483d8dead0 con 0x56483da01300 2023-10-06T21:32:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.772+0000 7f097f7fe700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483da01300 0x56483da03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.772+0000 7f097f7fe700 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483da01300 0x56483da03720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66da1b35573674c server_cookie=175b24ee643f120d in_seq=0 out_seq=0 2023-10-06T21:32:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.772+0000 7f095effd700 10 client.17127 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f095effd700 1 -- 172.21.15.31:0/662098778 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56483d8dead0 con 0x56483da01300 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f095effd700 10 client.17127 handle_command_reply: tid=0 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 client.17127 shutdown 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 msgr2=0x56483d9a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f09740994c0 tx=0x7f0974097d70 comp rx=0 tx=0).stop 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 shutdown_connections 2023-10-06T21:32:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56483d99d3d0 0x56483d9ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56483d991c00 0x56483d9a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56483da01300 0x56483da03720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 --2- 172.21.15.31:0/662098778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56483d992520 0x56483d9a3620 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f095effd700 0 client.17127 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.796+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 >> 172.21.15.31:0/662098778 conn(0x56483d907200 msgr2=0x56483d9991b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 shutdown_connections 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f09999f1b80 1 -- 172.21.15.31:0/662098778 wait complete. 2023-10-06T21:32:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f09999f1b80 20 client.17127 trim_cache size 0 max 16384 2023-10-06T21:32:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:32:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:32:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:32:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:32:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 msgr2=0x7f09680a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 0x7f09680a3b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0978009c10 tx=0x7f0978009920 comp rx=0 tx=0).stop 2023-10-06T21:32:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 msgr2=0x7f098019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.797+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f097400d950 tx=0x7f097400dc60 comp rx=0 tx=0).stop 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 shutdown_connections 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f09680a1670 0x7f09680a3b30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980105e20 0x7f09801a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0980068810 0x7f098019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 --2- 172.21.15.31:0/3101232573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09801038b0 0x7f098019d950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 >> 172.21.15.31:0/3101232573 conn(0x7f0980076b70 msgr2=0x7f09800feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 shutdown_connections 2023-10-06T21:32:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:14.798+0000 7f0985fff700 1 -- 172.21.15.31:0/3101232573 wait complete. 2023-10-06T21:32:15.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 179091 1.54 MB/sec execute 485 sec latency 32.090 ms 2023-10-06T21:32:16.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 179507 1.54 MB/sec execute 486 sec latency 25.615 ms 2023-10-06T21:32:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:16 smithi031 ceph-mon[98539]: pgmap v355: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 601 KiB/s wr, 84 op/s 2023-10-06T21:32:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:16 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:32:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:16 smithi146 ceph-mon[102809]: pgmap v355: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 601 KiB/s wr, 84 op/s 2023-10-06T21:32:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:16 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:32:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:16 smithi142 ceph-mon[102951]: pgmap v355: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 601 KiB/s wr, 84 op/s 2023-10-06T21:32:16.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:16 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:32:17.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 179816 1.54 MB/sec execute 487 sec latency 27.721 ms 2023-10-06T21:32:18.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180289 1.54 MB/sec execute 488 sec latency 19.949 ms 2023-10-06T21:32:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:18 smithi031 ceph-mon[98539]: pgmap v356: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-10-06T21:32:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:18 smithi146 ceph-mon[102809]: pgmap v356: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-10-06T21:32:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:18 smithi142 ceph-mon[102951]: pgmap v356: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-10-06T21:32:19.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180678 1.54 MB/sec execute 489 sec latency 18.057 ms 2023-10-06T21:32:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:20 smithi146 ceph-mon[102809]: pgmap v357: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 904 KiB/s wr, 96 op/s 2023-10-06T21:32:20.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180982 1.55 MB/sec execute 490 sec latency 52.496 ms 2023-10-06T21:32:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:20 smithi142 ceph-mon[102951]: pgmap v357: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 904 KiB/s wr, 96 op/s 2023-10-06T21:32:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:20 smithi031 ceph-mon[98539]: pgmap v357: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 904 KiB/s wr, 96 op/s 2023-10-06T21:32:21.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180982 1.54 MB/sec execute 491 sec latency 1052.542 ms 2023-10-06T21:32:22.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180982 1.54 MB/sec execute 492 sec latency 2052.659 ms 2023-10-06T21:32:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:22 smithi142 ceph-mon[102951]: pgmap v358: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 898 KiB/s wr, 95 op/s 2023-10-06T21:32:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:22 smithi031 ceph-mon[98539]: pgmap v358: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 898 KiB/s wr, 95 op/s 2023-10-06T21:32:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:22 smithi146 ceph-mon[102809]: pgmap v358: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 898 KiB/s wr, 95 op/s 2023-10-06T21:32:23.362 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180982 1.54 MB/sec execute 493 sec latency 3052.738 ms 2023-10-06T21:32:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:23 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:23 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:23 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:24.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 180982 1.54 MB/sec execute 494 sec latency 4052.821 ms 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: pgmap v359: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 854 KiB/s wr, 80 op/s 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:24.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:24 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: pgmap v359: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 854 KiB/s wr, 80 op/s 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:24.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:24 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: pgmap v359: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 854 KiB/s wr, 80 op/s 2023-10-06T21:32:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:24 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:32:25.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181049 1.53 MB/sec execute 495 sec latency 4145.645 ms 2023-10-06T21:32:26.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181049 1.53 MB/sec execute 496 sec latency 1540.926 ms 2023-10-06T21:32:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:26 smithi031 ceph-mon[98539]: pgmap v360: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 772 KiB/s wr, 72 op/s 2023-10-06T21:32:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:26 smithi146 ceph-mon[102809]: pgmap v360: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 772 KiB/s wr, 72 op/s 2023-10-06T21:32:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:26 smithi142 ceph-mon[102951]: pgmap v360: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 772 KiB/s wr, 72 op/s 2023-10-06T21:32:27.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181049 1.53 MB/sec execute 497 sec latency 2541.039 ms 2023-10-06T21:32:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 -- 172.21.15.31:0/3279321257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 msgr2=0x7f04281014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 --2- 172.21.15.31:0/3279321257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f04281014b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0424009990 tx=0x7f0424009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 -- 172.21.15.31:0/3279321257 shutdown_connections 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 --2- 172.21.15.31:0/3279321257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 0x7f042810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 --2- 172.21.15.31:0/3279321257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f04281014b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 --2- 172.21.15.31:0/3279321257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428101a80 0x7f0428105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 -- 172.21.15.31:0/3279321257 >> 172.21.15.31:0/3279321257 conn(0x7f04280fc950 msgr2=0x7f04280fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 -- 172.21.15.31:0/3279321257 shutdown_connections 2023-10-06T21:32:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.961+0000 7f04303ff700 1 -- 172.21.15.31:0/3279321257 wait complete. 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.962+0000 7f04303ff700 1 Processor -- start 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.962+0000 7f04303ff700 1 -- start start 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 0x7f04281973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f042819ba20 0x7f042819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819c4a0 con 0x7f04281010d0 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819c610 con 0x7f04281061f0 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f04303ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042819c7d0 con 0x7f042819ba20 2023-10-06T21:32:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f042e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f042e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47816/0 (socket says 172.21.15.31:47816) 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f042e19b700 1 -- 172.21.15.31:0/3575233469 learned_addr learned my addr 172.21.15.31:0/3575233469 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f042e99c700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f042819ba20 0x7f042819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.963+0000 7f042d99a700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 0x7f04281973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 -- 172.21.15.31:0/3575233469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f042819ba20 msgr2=0x7f042819bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f042819ba20 0x7f042819bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 -- 172.21.15.31:0/3575233469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 msgr2=0x7f04281973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 0x7f04281973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 -- 172.21.15.31:0/3575233469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240095f0 con 0x7f04281010d0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f042e19b700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f04240036c0 tx=0x7f0424005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f041f7fe700 1 -- 172.21.15.31:0/3575233469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042401d070 con 0x7f04281010d0 2023-10-06T21:32:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f041f7fe700 1 -- 172.21.15.31:0/3575233469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0424003f40 con 0x7f04281010d0 2023-10-06T21:32:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04281a3cd0 con 0x7f04281010d0 2023-10-06T21:32:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.964+0000 7f041f7fe700 1 -- 172.21.15.31:0/3575233469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424022c70 con 0x7f04281010d0 2023-10-06T21:32:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.965+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04281a40f0 con 0x7f04281010d0 2023-10-06T21:32:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.969+0000 7f041f7fe700 1 -- 172.21.15.31:0/3575233469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0424003ab0 con 0x7f04281010d0 2023-10-06T21:32:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.970+0000 7f0443ca6b80 1 Processor -- start 2023-10-06T21:32:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.970+0000 7f0443ca6b80 1 -- start start 2023-10-06T21:32:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f041f7fe700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 0x7f04140a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f041f7fe700 1 -- 172.21.15.31:0/3575233469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f04240ce7f0 con 0x7f04281010d0 2023-10-06T21:32:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793930 0x55aa4f79e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f6f8400 con 0x55aa4f793010 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f6e9bc0 con 0x55aa4f793930 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f6e74e0 con 0x55aa4f79e7e0 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f042e99c700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 0x7f04140a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f042e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f042e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47828/0 (socket says 172.21.15.31:47828) 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f042e19b700 1 -- 172.21.15.31:0/2512923793 learned_addr learned my addr 172.21.15.31:0/2512923793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.971+0000 7f042d99a700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793930 0x55aa4f79e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e99c700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 -- 172.21.15.31:0/2512923793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 msgr2=0x55aa4f7a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 -- 172.21.15.31:0/2512923793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793930 msgr2=0x55aa4f79e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793930 0x55aa4f79e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 -- 172.21.15.31:0/2512923793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa4f50dd80 con 0x55aa4f793010 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e99c700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e99c700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 0x7f04140a3ab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f042000f3c0 tx=0x7f042000f780 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f042e19b700 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0424097c90 tx=0x7f042409b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dfc19fa0c34c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f041cbfe700 1 -- 172.21.15.31:0/2512923793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042409be90 con 0x55aa4f793010 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f041cbfe700 1 -- 172.21.15.31:0/2512923793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f042409b790 con 0x55aa4f793010 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.972+0000 7f041cbfe700 1 -- 172.21.15.31:0/2512923793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240860b0 con 0x55aa4f793010 2023-10-06T21:32:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2512923793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 msgr2=0x55aa4f7933f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0424097c90 tx=0x7f042409b3a0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2512923793 shutdown_connections 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793010 0x55aa4f7933f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2512923793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793930 0x55aa4f79e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2512923793 >> 172.21.15.31:0/2512923793 conn(0x55aa4f704200 msgr2=0x55aa4f78aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2512923793 shutdown_connections 2023-10-06T21:32:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.973+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2512923793 wait complete. 2023-10-06T21:32:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.974+0000 7f0443ca6b80 1 -- start start 2023-10-06T21:32:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.974+0000 7f0443ca6b80 1 Processor -- start 2023-10-06T21:32:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f79ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f6e2dd0 con 0x55aa4f793930 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f67f6b0 con 0x55aa4f793010 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f67f350 con 0x55aa4f79e7e0 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f0443ca6b80 10 client.0 _ll_register_callbacks cb 0x55aa4f4dc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f042d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f042d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47842/0 (socket says 172.21.15.31:47842) 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f042d99a700 1 -- 172.21.15.31:0/3662053968 learned_addr learned my addr 172.21.15.31:0/3662053968 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f042e19b700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f79ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.975+0000 7f042e99c700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 -- 172.21.15.31:0/3662053968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 msgr2=0x55aa4f7a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 -- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 msgr2=0x55aa4f79ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f79ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 -- 172.21.15.31:0/3662053968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04281a40f0 con 0x55aa4f793930 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042e99c700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042e19b700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f79ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f042d99a700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0418011730 tx=0x7f0418011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab397607e627d36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f0411ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800f9f0 con 0x55aa4f793930 2023-10-06T21:32:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa4f782d50 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.976+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f041800fb50 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f041800fd10 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 10 client.17139 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aa4f4f6c00 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa4f782d50 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f041800e3b0 con 0x55aa4f793930 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 10 client.17139 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 20 client.17139 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f7adfe0 0x55aa4f7b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 4 client.17139 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.977+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa4f6dbdb0 con 0x55aa4f7adfe0 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.978+0000 7f042e19b700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f7adfe0 0x55aa4f7b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.978+0000 7f042e19b700 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f7adfe0 0x55aa4f7b0400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826c6a272daaf599 server_cookie=dde018564f3ec095 in_seq=0 out_seq=0 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.978+0000 7f0411ffb700 10 client.17139 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0411ffb700 1 -- 172.21.15.31:0/3662053968 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa4f6dbdb0 con 0x55aa4f7adfe0 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0411ffb700 10 client.17139 handle_command_reply: tid=0 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 client.17139 shutdown 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 msgr2=0x55aa4f79f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0418011730 tx=0x7f0418011af0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 shutdown_connections 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f79e7e0 0x55aa4f7a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f79ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f7adfe0 0x55aa4f7b0400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3662053968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f79f100 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0411ffb700 0 client.17139 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.979+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 >> 172.21.15.31:0/3662053968 conn(0x55aa4f704200 msgr2=0x55aa4f78abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.980+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 shutdown_connections 2023-10-06T21:32:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.980+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3662053968 wait complete. 2023-10-06T21:32:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.980+0000 7f0443ca6b80 20 client.17139 trim_cache size 0 max 16384 2023-10-06T21:32:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.995+0000 7f0443ca6b80 1 Processor -- start 2023-10-06T21:32:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.995+0000 7f0443ca6b80 1 -- start start 2023-10-06T21:32:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f7a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793930 0x55aa4f79d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f5638c0 con 0x55aa4f79e7e0 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f785200 con 0x55aa4f793010 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f79f470 con 0x55aa4f793930 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47852/0 (socket says 172.21.15.31:47852) 2023-10-06T21:32:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 -- 172.21.15.31:0/3930295973 learned_addr learned my addr 172.21.15.31:0/3930295973 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e19b700 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f7a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042d99a700 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793930 0x55aa4f79d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 -- 172.21.15.31:0/3930295973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793930 msgr2=0x55aa4f79d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793930 0x55aa4f79d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 -- 172.21.15.31:0/3930295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 msgr2=0x55aa4f7a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f7a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.996+0000 7f042e99c700 1 -- 172.21.15.31:0/3930295973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa4f5341f0 con 0x55aa4f79e7e0 2023-10-06T21:32:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.997+0000 7f042e99c700 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0420016540 tx=0x7f0420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3612f118d2da36fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.997+0000 7f041d3ff700 1 -- 172.21.15.31:0/3930295973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420023420 con 0x55aa4f79e7e0 2023-10-06T21:32:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.997+0000 7f041d3ff700 1 -- 172.21.15.31:0/3930295973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0420023a10 con 0x55aa4f79e7e0 2023-10-06T21:32:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.997+0000 7f041d3ff700 1 -- 172.21.15.31:0/3930295973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420017860 con 0x55aa4f79e7e0 2023-10-06T21:32:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3930295973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 msgr2=0x55aa4f79ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0420016540 tx=0x7f0420017000 comp rx=0 tx=0).stop 2023-10-06T21:32:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3930295973 shutdown_connections 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f79e7e0 0x55aa4f79ab20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f793010 0x55aa4f7a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/3930295973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793930 0x55aa4f79d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3930295973 >> 172.21.15.31:0/3930295973 conn(0x55aa4f704200 msgr2=0x55aa4f7897b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3930295973 shutdown_connections 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- 172.21.15.31:0/3930295973 wait complete. 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.998+0000 7f0443ca6b80 1 -- start start 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 Processor -- start 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793010 0x55aa4f7f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f79e7e0 0x55aa4f800e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f7f6c00 con 0x55aa4f793930 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f801360 con 0x55aa4f79e7e0 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4f8014f0 con 0x55aa4f793010 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:27.999+0000 7f0443ca6b80 10 client.0 _ll_register_callbacks cb 0x7f0418008b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47856/0 (socket says 172.21.15.31:47856) 2023-10-06T21:32:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 -- 172.21.15.31:0/2004123553 learned_addr learned my addr 172.21.15.31:0/2004123553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042e19b700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793010 0x55aa4f7f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 -- 172.21.15.31:0/2004123553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793010 msgr2=0x55aa4f7f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793010 0x55aa4f7f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 -- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f79e7e0 msgr2=0x55aa4f800e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042e99c700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f79e7e0 0x55aa4f800e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f79e7e0 0x55aa4f800e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 -- 172.21.15.31:0/2004123553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042001f040 con 0x55aa4f793930 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f042d99a700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55aa4f79de30 tx=0x7f0418008c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f024b6da36b7a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f0411ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.000+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418005240 con 0x55aa4f793930 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f042409b790 con 0x55aa4f793930 2023-10-06T21:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0418000e40 con 0x55aa4f793930 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0418005d40 con 0x55aa4f793930 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 10 client.17145 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aa4f50c250 con 0x55aa4f793930 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f042409b790 con 0x55aa4f793930 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f0418003680 con 0x55aa4f793930 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 10 client.17145 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 20 client.17145 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f802d70 0x55aa4f805190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 4 client.17145 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.001+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa4f6dbad0 con 0x55aa4f802d70 2023-10-06T21:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.002+0000 7f042e19b700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f802d70 0x55aa4f805190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.002+0000 7f042e19b700 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f802d70 0x55aa4f805190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcdb7eb169625db server_cookie=446113d2f13600bb in_seq=0 out_seq=0 2023-10-06T21:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.002+0000 7f0411ffb700 10 client.17145 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0411ffb700 1 -- 172.21.15.31:0/2004123553 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa4f6dbad0 con 0x55aa4f802d70 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0411ffb700 10 client.17145 handle_command_reply: tid=0 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 client.17145 shutdown 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 msgr2=0x55aa4f7f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55aa4f79de30 tx=0x7f0418008c50 comp rx=0 tx=0).stop 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 shutdown_connections 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa4f79e7e0 0x55aa4f800e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa4f793010 0x55aa4f7f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aa4f802d70 0x55aa4f805190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 --2- 172.21.15.31:0/2004123553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aa4f793930 0x55aa4f7f64d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 >> 172.21.15.31:0/2004123553 conn(0x55aa4f704200 msgr2=0x55aa4f795070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0411ffb700 0 client.17145 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 shutdown_connections 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.033+0000 7f0443ca6b80 1 -- 172.21.15.31:0/2004123553 wait complete. 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f0443ca6b80 20 client.17145 trim_cache size 0 max 16384 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:32:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:32:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 msgr2=0x7f04140a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 0x7f04140a3ab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f042000f3c0 tx=0x7f042000f780 comp rx=0 tx=0).stop 2023-10-06T21:32:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 msgr2=0x7f0428196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f04240036c0 tx=0x7f0424005740 comp rx=0 tx=0).stop 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 shutdown_connections 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f042819ba20 0x7f042819bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f04140a15f0 0x7f04140a3ab0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04281010d0 0x7f0428196e70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 --2- 172.21.15.31:0/3575233469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04281061f0 0x7f04281973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 >> 172.21.15.31:0/3575233469 conn(0x7f04280fc950 msgr2=0x7f04280fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 shutdown_connections 2023-10-06T21:32:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:28.034+0000 7f04303ff700 1 -- 172.21.15.31:0/3575233469 wait complete. 2023-10-06T21:32:28.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181049 1.52 MB/sec execute 498 sec latency 3541.151 ms 2023-10-06T21:32:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:28 smithi031 ceph-mon[98539]: pgmap v361: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 711 KiB/s wr, 66 op/s 2023-10-06T21:32:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:28 smithi146 ceph-mon[102809]: pgmap v361: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 711 KiB/s wr, 66 op/s 2023-10-06T21:32:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:28 smithi142 ceph-mon[102951]: pgmap v361: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 711 KiB/s wr, 66 op/s 2023-10-06T21:32:29.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181049 1.52 MB/sec execute 499 sec latency 4541.305 ms 2023-10-06T21:32:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:30 smithi146 ceph-mon[102809]: pgmap v362: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 758 KiB/s wr, 76 op/s 2023-10-06T21:32:30.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181220 1.52 MB/sec execute 500 sec latency 4639.340 ms 2023-10-06T21:32:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:30 smithi142 ceph-mon[102951]: pgmap v362: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 758 KiB/s wr, 76 op/s 2023-10-06T21:32:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:30 smithi031 ceph-mon[98539]: pgmap v362: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 758 KiB/s wr, 76 op/s 2023-10-06T21:32:31.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181384 1.52 MB/sec execute 501 sec latency 19.167 ms 2023-10-06T21:32:32.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181634 1.51 MB/sec execute 502 sec latency 22.157 ms 2023-10-06T21:32:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:32 smithi142 ceph-mon[102951]: pgmap v363: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 334 KiB/s wr, 32 op/s 2023-10-06T21:32:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:32 smithi031 ceph-mon[98539]: pgmap v363: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 334 KiB/s wr, 32 op/s 2023-10-06T21:32:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:32 smithi146 ceph-mon[102809]: pgmap v363: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 334 KiB/s wr, 32 op/s 2023-10-06T21:32:33.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 181971 1.51 MB/sec execute 503 sec latency 22.361 ms 2023-10-06T21:32:34.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 182297 1.52 MB/sec execute 504 sec latency 209.811 ms 2023-10-06T21:32:34.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:34 smithi142 ceph-mon[102951]: pgmap v364: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 653 KiB/s wr, 71 op/s 2023-10-06T21:32:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:34 smithi031 ceph-mon[98539]: pgmap v364: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 653 KiB/s wr, 71 op/s 2023-10-06T21:32:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:34 smithi146 ceph-mon[102809]: pgmap v364: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 653 KiB/s wr, 71 op/s 2023-10-06T21:32:35.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 182571 1.51 MB/sec execute 505 sec latency 26.758 ms 2023-10-06T21:32:36.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 182950 1.51 MB/sec execute 506 sec latency 28.461 ms 2023-10-06T21:32:36.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:36 smithi142 ceph-mon[102951]: pgmap v365: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:36 smithi031 ceph-mon[98539]: pgmap v365: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:36 smithi146 ceph-mon[102809]: pgmap v365: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:37.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 183308 1.51 MB/sec execute 507 sec latency 27.932 ms 2023-10-06T21:32:38.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 183687 1.52 MB/sec execute 508 sec latency 20.135 ms 2023-10-06T21:32:38.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:38 smithi142 ceph-mon[102951]: pgmap v366: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:38 smithi031 ceph-mon[98539]: pgmap v366: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:38 smithi146 ceph-mon[102809]: pgmap v366: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 365 KiB/s wr, 49 op/s 2023-10-06T21:32:39.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184204 1.52 MB/sec execute 509 sec latency 17.584 ms 2023-10-06T21:32:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:40 smithi146 ceph-mon[102809]: pgmap v367: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 754 KiB/s wr, 88 op/s 2023-10-06T21:32:40.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184459 1.52 MB/sec execute 510 sec latency 25.558 ms 2023-10-06T21:32:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:40 smithi142 ceph-mon[102951]: pgmap v367: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 754 KiB/s wr, 88 op/s 2023-10-06T21:32:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:40 smithi031 ceph-mon[98539]: pgmap v367: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 754 KiB/s wr, 88 op/s 2023-10-06T21:32:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.187+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/1421161375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 msgr2=0x7fe2c41014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.187+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/1421161375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c41014b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009990 tx=0x7fe2b4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/1421161375 shutdown_connections 2023-10-06T21:32:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/1421161375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 0x7fe2c410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/1421161375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c41014b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/1421161375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c4101a80 0x7fe2c4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/1421161375 >> 172.21.15.31:0/1421161375 conn(0x7fe2c40fc950 msgr2=0x7fe2c40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/1421161375 shutdown_connections 2023-10-06T21:32:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.188+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/1421161375 wait complete. 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 Processor -- start 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 -- start start 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 0x7fe2c41973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 0x7fe2c419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419c4b0 con 0x7fe2c41010d0 2023-10-06T21:32:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419c620 con 0x7fe2c419ba30 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2ca1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419c7e0 con 0x7fe2c41061f0 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41482/0 (socket says 172.21.15.31:41482) 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.189+0000 7fe2c37fe700 1 -- 172.21.15.31:0/2541458216 learned_addr learned my addr 172.21.15.31:0/2541458216 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 0x7fe2c419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 0x7fe2c41973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 -- 172.21.15.31:0/2541458216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 msgr2=0x7fe2c41973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 0x7fe2c41973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 -- 172.21.15.31:0/2541458216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 msgr2=0x7fe2c419beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 0x7fe2c419beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 -- 172.21.15.31:0/2541458216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40095f0 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 0x7fe2c419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005740 tx=0x7fe2b400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c0ff9700 1 -- 172.21.15.31:0/2541458216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b401d070 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c0ff9700 1 -- 172.21.15.31:0/2541458216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b4022470 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.190+0000 7fe2c0ff9700 1 -- 172.21.15.31:0/2541458216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4017650 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.191+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c41a3ce0 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.191+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c41a40b0 con 0x7fe2c41010d0 2023-10-06T21:32:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.194+0000 7fe2c0ff9700 1 -- 172.21.15.31:0/2541458216 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe2b40225e0 con 0x7fe2c41010d0 2023-10-06T21:32:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 Processor -- start 2023-10-06T21:32:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 -- start start 2023-10-06T21:32:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2c0ff9700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 0x7fe2ac0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e251efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 0x5613e252c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2c0ff9700 1 -- 172.21.15.31:0/2541458216 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe2b40ce690 con 0x7fe2c41010d0 2023-10-06T21:32:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.196+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e2488400 con 0x5613e251f520 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e2479bc0 con 0x5613e251ec00 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e24774e0 con 0x5613e252a3d0 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 0x7fe2ac0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41494/0 (socket says 172.21.15.31:41494) 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c2ffd700 1 -- 172.21.15.31:0/800212632 learned_addr learned my addr 172.21.15.31:0/800212632 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e251efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 0x5613e252c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 -- 172.21.15.31:0/800212632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 msgr2=0x5613e252c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 0x5613e252c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 -- 172.21.15.31:0/800212632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 msgr2=0x5613e2529e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c37fe700 1 -- 172.21.15.31:0/800212632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613e229dd80 con 0x5613e251ec00 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.197+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 0x5613e252c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.198+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 0x7fe2ac0a3b30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc010370 tx=0x7fe2bc007b50 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.198+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e251efe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cbd10 tx=0x7fe2b409b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc395905015f34b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.198+0000 7fe2a9ffb700 1 -- 172.21.15.31:0/800212632 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40860f0 con 0x5613e251ec00 2023-10-06T21:32:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.198+0000 7fe2a9ffb700 1 -- 172.21.15.31:0/800212632 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b409b630 con 0x5613e251ec00 2023-10-06T21:32:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.198+0000 7fe2a9ffb700 1 -- 172.21.15.31:0/800212632 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b409baf0 con 0x5613e251ec00 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/800212632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 msgr2=0x5613e251efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e251efe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cbd10 tx=0x7fe2b409b1f0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/800212632 shutdown_connections 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e252a3d0 0x5613e252c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e251efe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/800212632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e2529e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/800212632 >> 172.21.15.31:0/800212632 conn(0x5613e2494200 msgr2=0x5613e251b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/800212632 shutdown_connections 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/800212632 wait complete. 2023-10-06T21:32:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.199+0000 7fe2ddb6db80 1 -- start start 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 Processor -- start 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2535830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e2472dd0 con 0x5613e251ec00 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e240f6b0 con 0x5613e252a3d0 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e240f350 con 0x5613e251f520 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2ddb6db80 10 client.0 _ll_register_callbacks cb 0x5613e226c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41508/0 (socket says 172.21.15.31:41508) 2023-10-06T21:32:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2c37fe700 1 -- 172.21.15.31:0/1822932180 learned_addr learned my addr 172.21.15.31:0/1822932180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2535830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.200+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 -- 172.21.15.31:0/1822932180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 msgr2=0x5613e252acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 -- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 msgr2=0x5613e2535830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2535830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 -- 172.21.15.31:0/1822932180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b80098b0 con 0x5613e251ec00 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2535830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cc290 tx=0x7fe2b409add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc21d777b315b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b409f780 con 0x5613e251ec00 2023-10-06T21:32:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe2b40a3930 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.201+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613e2512d50 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b409baa0 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 10 client.17157 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fe2c41a40b0 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5613e2512d50 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe2b4097420 con 0x5613e251ec00 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 10 client.17157 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 20 client.17157 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2539670 0x5613e253ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 4 client.17157 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613e246bdb0 con 0x5613e2539670 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.202+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2539670 0x5613e253ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.203+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2539670 0x5613e253ba90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6352bcd20c8bb8d4 server_cookie=308ad98c95767d34 in_seq=0 out_seq=0 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.203+0000 7fe2a2ffd700 10 client.17157 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/1822932180 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613e246bdb0 con 0x5613e2539670 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2a2ffd700 10 client.17157 handle_command_reply: tid=0 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 client.17157 shutdown 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 msgr2=0x5613e252a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cc290 tx=0x7fe2b409add0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 shutdown_connections 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2535830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2539670 0x5613e253ba90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251ec00 0x5613e252a7b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/1822932180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2a2ffd700 0 client.17157 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 >> 172.21.15.31:0/1822932180 conn(0x5613e2494200 msgr2=0x5613e251abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 shutdown_connections 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/1822932180 wait complete. 2023-10-06T21:32:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.204+0000 7fe2ddb6db80 20 client.17157 trim_cache size 0 max 16384 2023-10-06T21:32:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 Processor -- start 2023-10-06T21:32:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 -- start start 2023-10-06T21:32:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251ec00 0x5613e252f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2524e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e22f38c0 con 0x5613e251f520 2023-10-06T21:32:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e25152c0 con 0x5613e252a3d0 2023-10-06T21:32:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.220+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e252b060 con 0x5613e251ec00 2023-10-06T21:32:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41512/0 (socket says 172.21.15.31:41512) 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 -- 172.21.15.31:0/3645103660 learned_addr learned my addr 172.21.15.31:0/3645103660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251ec00 0x5613e252f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2524e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 -- 172.21.15.31:0/3645103660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251ec00 msgr2=0x5613e252f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251ec00 0x5613e252f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 -- 172.21.15.31:0/3645103660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 msgr2=0x5613e2524e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2524e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 -- 172.21.15.31:0/3645103660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613e2286c00 con 0x5613e251f520 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2524e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.221+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5613e253c360 tx=0x7fe2b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2ae5f30c142e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2aa7fc700 1 -- 172.21.15.31:0/3645103660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8017040 con 0x5613e251f520 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2aa7fc700 1 -- 172.21.15.31:0/3645103660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b8018040 con 0x5613e251f520 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2aa7fc700 1 -- 172.21.15.31:0/3645103660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8003d20 con 0x5613e251f520 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/3645103660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 msgr2=0x5613e25248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5613e253c360 tx=0x7fe2b8009260 comp rx=0 tx=0).stop 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/3645103660 shutdown_connections 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e252a3d0 0x5613e2524e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251ec00 0x5613e252f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/3645103660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e251f520 0x5613e25248f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/3645103660 >> 172.21.15.31:0/3645103660 conn(0x5613e2494200 msgr2=0x5613e251a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/3645103660 shutdown_connections 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.222+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/3645103660 wait complete. 2023-10-06T21:32:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 -- start start 2023-10-06T21:32:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 Processor -- start 2023-10-06T21:32:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e252d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e252e660 con 0x5613e252a3d0 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e2590de0 con 0x5613e251ec00 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.223+0000 7fe2ddb6db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e2590fa0 con 0x5613e251f520 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2ddb6db80 10 client.0 _ll_register_callbacks cb 0x5613e226c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41516/0 (socket says 172.21.15.31:41516) 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 -- 172.21.15.31:0/915200756 learned_addr learned my addr 172.21.15.31:0/915200756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c2ffd700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e252d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 -- 172.21.15.31:0/915200756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 msgr2=0x5613e252ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 -- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 msgr2=0x5613e252d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e252d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 -- 172.21.15.31:0/915200756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8009c80 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e252d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.224+0000 7fe2c3fff700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0186d0 tx=0x7fe2bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8d91e71e991417 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc017e80 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b409b630 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe2bc01d040 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2bc004bb0 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2ddb6db80 10 client.17163 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5613e22c41f0 con 0x5613e252a3d0 2023-10-06T21:32:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.225+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b409b630 con 0x5613e252a3d0 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe2bc023470 con 0x5613e252a3d0 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 10 client.17163 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 20 client.17163 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2592840 0x5613e2594c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 4 client.17163 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613e246bad0 con 0x5613e2592840 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2592840 0x5613e2594c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.226+0000 7fe2c37fe700 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2592840 0x5613e2594c60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3321b28e061cfecb server_cookie=a8b1c626bbcbb4db in_seq=0 out_seq=0 2023-10-06T21:32:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.227+0000 7fe2a2ffd700 10 client.17163 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2a2ffd700 1 -- 172.21.15.31:0/915200756 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613e246bad0 con 0x5613e2592840 2023-10-06T21:32:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2a2ffd700 10 client.17163 handle_command_reply: tid=0 2023-10-06T21:32:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 client.17163 shutdown 2023-10-06T21:32:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 msgr2=0x5613e2590870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0186d0 tx=0x7fe2bc004b40 comp rx=0 tx=0).stop 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 shutdown_connections 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5613e2592840 0x5613e2594c60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5613e252a3d0 0x5613e2590870 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613e251ec00 0x5613e252d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 --2- 172.21.15.31:0/915200756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5613e251f520 0x5613e252ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2a2ffd700 0 client.17163 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.255+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 >> 172.21.15.31:0/915200756 conn(0x5613e2494200 msgr2=0x5613e25248f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 shutdown_connections 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ddb6db80 1 -- 172.21.15.31:0/915200756 wait complete. 2023-10-06T21:32:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ddb6db80 20 client.17163 trim_cache size 0 max 16384 2023-10-06T21:32:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:32:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:32:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:32:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:32:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 msgr2=0x7fe2ac0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 0x7fe2ac0a3b30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc010370 tx=0x7fe2bc007b50 comp rx=0 tx=0).stop 2023-10-06T21:32:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 msgr2=0x7fe2c4196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.256+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005740 tx=0x7fe2b400bfa0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 shutdown_connections 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2c419ba30 0x7fe2c419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe2ac0a1670 0x7fe2ac0a3b30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2c41010d0 0x7fe2c4196e80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 --2- 172.21.15.31:0/2541458216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2c41061f0 0x7fe2c41973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 >> 172.21.15.31:0/2541458216 conn(0x7fe2c40fc950 msgr2=0x7fe2c40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 shutdown_connections 2023-10-06T21:32:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:41.257+0000 7fe2ca1ff700 1 -- 172.21.15.31:0/2541458216 wait complete. 2023-10-06T21:32:41.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184528 1.52 MB/sec execute 511 sec latency 818.967 ms 2023-10-06T21:32:42.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184528 1.52 MB/sec execute 512 sec latency 1819.048 ms 2023-10-06T21:32:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:42 smithi142 ceph-mon[102951]: pgmap v368: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 708 KiB/s wr, 78 op/s 2023-10-06T21:32:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:42 smithi031 ceph-mon[98539]: pgmap v368: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 708 KiB/s wr, 78 op/s 2023-10-06T21:32:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:42 smithi146 ceph-mon[102809]: pgmap v368: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 708 KiB/s wr, 78 op/s 2023-10-06T21:32:43.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184528 1.52 MB/sec execute 513 sec latency 2819.124 ms 2023-10-06T21:32:44.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184528 1.51 MB/sec execute 514 sec latency 3819.247 ms 2023-10-06T21:32:44.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:44 smithi142 ceph-mon[102951]: pgmap v369: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-06T21:32:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:44 smithi031 ceph-mon[98539]: pgmap v369: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-06T21:32:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:44 smithi146 ceph-mon[102809]: pgmap v369: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-06T21:32:45.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184595 1.51 MB/sec execute 515 sec latency 3912.925 ms 2023-10-06T21:32:46.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184595 1.51 MB/sec execute 516 sec latency 1552.221 ms 2023-10-06T21:32:46.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:46 smithi142 ceph-mon[102951]: pgmap v370: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:46 smithi031 ceph-mon[98539]: pgmap v370: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:46 smithi146 ceph-mon[102809]: pgmap v370: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:47.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184595 1.50 MB/sec execute 517 sec latency 2552.386 ms 2023-10-06T21:32:48.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184595 1.50 MB/sec execute 518 sec latency 3552.455 ms 2023-10-06T21:32:48.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:48 smithi142 ceph-mon[102951]: pgmap v371: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:48 smithi031 ceph-mon[98539]: pgmap v371: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:48 smithi146 ceph-mon[102809]: pgmap v371: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:32:49.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184595 1.50 MB/sec execute 519 sec latency 4552.558 ms 2023-10-06T21:32:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:50 smithi146 ceph-mon[102809]: pgmap v372: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 745 KiB/s wr, 78 op/s 2023-10-06T21:32:50.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184769 1.50 MB/sec execute 520 sec latency 4646.639 ms 2023-10-06T21:32:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:50 smithi142 ceph-mon[102951]: pgmap v372: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 745 KiB/s wr, 78 op/s 2023-10-06T21:32:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:50 smithi031 ceph-mon[98539]: pgmap v372: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 745 KiB/s wr, 78 op/s 2023-10-06T21:32:51.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 184933 1.50 MB/sec execute 521 sec latency 19.201 ms 2023-10-06T21:32:52.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 185190 1.49 MB/sec execute 522 sec latency 20.428 ms 2023-10-06T21:32:52.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:52 smithi142 ceph-mon[102951]: pgmap v373: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 355 KiB/s wr, 38 op/s 2023-10-06T21:32:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:52 smithi031 ceph-mon[98539]: pgmap v373: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 355 KiB/s wr, 38 op/s 2023-10-06T21:32:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:52 smithi146 ceph-mon[102809]: pgmap v373: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 355 KiB/s wr, 38 op/s 2023-10-06T21:32:53.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 185527 1.49 MB/sec execute 523 sec latency 17.662 ms 2023-10-06T21:32:54.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 185909 1.50 MB/sec execute 524 sec latency 25.554 ms 2023-10-06T21:32:54.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:54 smithi142 ceph-mon[102951]: pgmap v374: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 669 KiB/s wr, 73 op/s 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.418+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2766066327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac0ffff0 msgr2=0x7fbeac1003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.418+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2766066327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac0ffff0 0x7fbeac1003d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009990 tx=0x7fbea8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2766066327 shutdown_connections 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2766066327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac104f30 0x7fbeac10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2766066327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac0ffff0 0x7fbeac1003d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2766066327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac1009a0 0x7fbeac104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2766066327 >> 172.21.15.31:0/2766066327 conn(0x7fbeac0756e0 msgr2=0x7fbeac075af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2766066327 shutdown_connections 2023-10-06T21:32:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.419+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2766066327 wait complete. 2023-10-06T21:32:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.420+0000 7fbeb2f9d700 1 Processor -- start 2023-10-06T21:32:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.420+0000 7fbeb2f9d700 1 -- start start 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 0x7fbeac197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac19b780 0x7fbeac19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac19c200 con 0x7fbeac104f30 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac19c370 con 0x7fbeac19b780 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb2f9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac19c530 con 0x7fbeac0ffff0 2023-10-06T21:32:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 0x7fbeac197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:50934/0 (socket says 172.21.15.31:50934) 2023-10-06T21:32:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/2010967859 learned_addr learned my addr 172.21.15.31:0/2010967859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb279c700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac19b780 0x7fbeac19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/2010967859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac19b780 msgr2=0x7fbeac19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac19b780 0x7fbeac19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 msgr2=0x7fbeac197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 0x7fbeac197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/2010967859 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80095f0 con 0x7fbeac0ffff0 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.421+0000 7fbeb179a700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 0x7fbeac197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbea2ffd700 1 -- 172.21.15.31:0/2010967859 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea801d070 con 0x7fbeac0ffff0 2023-10-06T21:32:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac1a3a30 con 0x7fbeac0ffff0 2023-10-06T21:32:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbea2ffd700 1 -- 172.21.15.31:0/2010967859 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea8022470 con 0x7fbeac0ffff0 2023-10-06T21:32:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbea2ffd700 1 -- 172.21.15.31:0/2010967859 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8017650 con 0x7fbeac0ffff0 2023-10-06T21:32:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.422+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac1a3eb0 con 0x7fbeac0ffff0 2023-10-06T21:32:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.426+0000 7fbea2ffd700 1 -- 172.21.15.31:0/2010967859 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbea80225e0 con 0x7fbeac0ffff0 2023-10-06T21:32:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.427+0000 7fbec7b7ab80 1 Processor -- start 2023-10-06T21:32:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.427+0000 7fbec7b7ab80 1 -- start start 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978f9f4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f950400 con 0x55978f9e7680 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f941bc0 con 0x55978f9e6d60 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f93f4e0 con 0x55978f9f2530 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbea2ffd700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 0x7fbe980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbea2ffd700 1 -- 172.21.15.31:0/2010967859 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbea80ce610 con 0x7fbeac0ffff0 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb279c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb279c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43156/0 (socket says 172.21.15.31:43156) 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb279c700 1 -- 172.21.15.31:0/1813500349 learned_addr learned my addr 172.21.15.31:0/1813500349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb279c700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 0x7fbe980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.428+0000 7fbeb179a700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978f9f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 -- 172.21.15.31:0/1813500349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 msgr2=0x55978f9f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978f9f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 -- 172.21.15.31:0/1813500349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 msgr2=0x55978f9f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 -- 172.21.15.31:0/1813500349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978f765d80 con 0x55978f9e6d60 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbea4016a40 tx=0x7fbea4016d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89fefa575dd17ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbeb279c700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 0x7fbe980a3b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbea40077e0 tx=0x7fbea400d000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.429+0000 7fbe97fff700 1 -- 172.21.15.31:0/1813500349 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4016ed0 con 0x55978f9e6d60 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbe97fff700 1 -- 172.21.15.31:0/1813500349 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea400f040 con 0x55978f9e6d60 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbe97fff700 1 -- 172.21.15.31:0/1813500349 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4004bb0 con 0x55978f9e6d60 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/1813500349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 msgr2=0x55978f9e7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbea4016a40 tx=0x7fbea4016d50 comp rx=0 tx=0).stop 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/1813500349 shutdown_connections 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978f9f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9e7140 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/1813500349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.430+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/1813500349 >> 172.21.15.31:0/1813500349 conn(0x55978f95c200 msgr2=0x55978f9e31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.431+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/1813500349 shutdown_connections 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.431+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/1813500349 wait complete. 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.431+0000 7fbec7b7ab80 1 -- start start 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.431+0000 7fbec7b7ab80 1 Processor -- start 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f93add0 con 0x55978f9e6d60 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f8d76b0 con 0x55978f9f2530 2023-10-06T21:32:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f8d7350 con 0x55978f9e7680 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbec7b7ab80 10 client.0 _ll_register_callbacks cb 0x55978f7347a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34778/0 (socket says 172.21.15.31:34778) 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/3760984497 learned_addr learned my addr 172.21.15.31:0/3760984497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb179a700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb279c700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.432+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/3760984497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 msgr2=0x55978f9ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 msgr2=0x55978f9ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbeb1f9b700 1 -- 172.21.15.31:0/3760984497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac1a3eb0 con 0x55978f9e6d60 2023-10-06T21:32:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbea800f740 tx=0x7fbea80148d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c278226db07fffe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbe957fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800fcc0 con 0x55978f9e6d60 2023-10-06T21:32:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55978f9daeb0 con 0x55978f9e6d60 2023-10-06T21:32:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbea80228c0 con 0x55978f9e6d60 2023-10-06T21:32:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea809b0f0 con 0x55978f9e6d60 2023-10-06T21:32:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbec7b7ab80 10 client.17175 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.433+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fbea4011040 con 0x55978f9e6d60 2023-10-06T21:32:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55978f9daeb0 con 0x55978f9e6d60 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbea809b810 con 0x55978f9e6d60 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 10 client.17175 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 20 client.17175 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa01db0 0x55978fa041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 4 client.17175 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55978f933db0 con 0x55978fa01db0 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.434+0000 7fbeb179a700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa01db0 0x55978fa041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.435+0000 7fbeb179a700 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa01db0 0x55978fa041d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc504819845666 server_cookie=50f17a40f75df36a in_seq=0 out_seq=0 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.435+0000 7fbe957fa700 10 client.17175 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.435+0000 7fbe957fa700 1 -- 172.21.15.31:0/3760984497 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55978f933db0 con 0x55978fa01db0 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.435+0000 7fbe957fa700 10 client.17175 handle_command_reply: tid=0 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.435+0000 7fbec7b7ab80 1 client.17175 shutdown 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 msgr2=0x55978f9f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbea800f740 tx=0x7fbea80148d0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 shutdown_connections 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa01db0 0x55978fa041d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9f1f80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/3760984497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 >> 172.21.15.31:0/3760984497 conn(0x55978f95c200 msgr2=0x55978f9e2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbe957fa700 0 client.17175 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 shutdown_connections 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/3760984497 wait complete. 2023-10-06T21:32:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.436+0000 7fbec7b7ab80 20 client.17175 trim_cache size 0 max 16384 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 Processor -- start 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 -- start start 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f7bb8c0 con 0x55978f9e6d60 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f9e8820 con 0x55978f9f2530 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.456+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f9e9c00 con 0x55978f9e7680 2023-10-06T21:32:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb179a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb179a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:50968/0 (socket says 172.21.15.31:50968) 2023-10-06T21:32:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb179a700 1 -- 172.21.15.31:0/613359754 learned_addr learned my addr 172.21.15.31:0/613359754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 -- 172.21.15.31:0/613359754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 msgr2=0x55978f9ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 -- 172.21.15.31:0/613359754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 msgr2=0x55978f9ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9ef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.457+0000 7fbeb279c700 1 -- 172.21.15.31:0/613359754 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978f74ec00 con 0x55978f9f2530 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.458+0000 7fbeb279c700 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9fd770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007fd0 tx=0x7fbea4007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c5ce5521c52958 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.458+0000 7fbea0ff9700 1 -- 172.21.15.31:0/613359754 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4036070 con 0x55978f9f2530 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.458+0000 7fbea0ff9700 1 -- 172.21.15.31:0/613359754 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea4016ed0 con 0x55978f9f2530 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.458+0000 7fbea0ff9700 1 -- 172.21.15.31:0/613359754 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003cd0 con 0x55978f9f2530 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/613359754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 msgr2=0x55978f9fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9fd770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007fd0 tx=0x7fbea4007f60 comp rx=0 tx=0).stop 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/613359754 shutdown_connections 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9f2530 0x55978f9fd770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e6d60 0x55978f9ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/613359754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9e7680 0x55978f9ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/613359754 >> 172.21.15.31:0/613359754 conn(0x55978f95c200 msgr2=0x55978f9dee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/613359754 shutdown_connections 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/613359754 wait complete. 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.459+0000 7fbec7b7ab80 1 -- start start 2023-10-06T21:32:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 Processor -- start 2023-10-06T21:32:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978fa54ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f9f7ed0 con 0x55978f9e7680 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978fa55110 con 0x55978f9e6d60 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978fa552d0 con 0x55978f9f2530 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.460+0000 7fbec7b7ab80 10 client.0 _ll_register_callbacks cb 0x7fbea80860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34798/0 (socket says 172.21.15.31:34798) 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb1f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43180/0 (socket says 172.21.15.31:43180) 2023-10-06T21:32:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 -- 172.21.15.31:0/2118671085 learned_addr learned my addr 172.21.15.31:0/2118671085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb279c700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978fa54ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 -- 172.21.15.31:0/2118671085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 msgr2=0x55978fa54ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978fa54ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 -- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 msgr2=0x55978f9f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 -- 172.21.15.31:0/2118671085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55978f765d80 con 0x55978f9e7680 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbeb179a700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbeac073bb0 tx=0x7fbe9c01fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d2b56d43fd5ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.461+0000 7fbe957fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c016070 con 0x55978f9e7680 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbea400f040 con 0x55978f9e7680 2023-10-06T21:32:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbe9c003e80 con 0x55978f9e7680 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe9c01f040 con 0x55978f9e7680 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 10 client.17181 fetch_fsmap learned FSMap version 40 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55978f78c1f0 con 0x55978f9e7680 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea400f040 con 0x55978f9e7680 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbe9c010b90 con 0x55978f9e7680 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 10 client.17181 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.462+0000 7fbec7b7ab80 20 client.17181 populate_metadata read hostname 'smithi031' 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa56b60 0x55978fa58f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbec7b7ab80 4 client.17181 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55978f933ad0 con 0x55978fa56b60 2023-10-06T21:32:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa56b60 0x55978fa58f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbe957fa700 10 client.17181 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.463+0000 7fbeb1f9b700 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa56b60 0x55978fa58f80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac83601f183622c server_cookie=5de3e905442092ff in_seq=0 out_seq=0 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbe957fa700 1 -- 172.21.15.31:0/2118671085 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55978f933ad0 con 0x55978fa56b60 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbe957fa700 10 client.17181 handle_command_reply: tid=0 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 client.17181 shutdown 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 msgr2=0x55978f9f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbeac073bb0 tx=0x7fbe9c01fd40 comp rx=0 tx=0).stop 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 shutdown_connections 2023-10-06T21:32:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55978f9f2530 0x55978fa54ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55978f9e6d60 0x55978f9f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55978fa56b60 0x55978fa58f80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 --2- 172.21.15.31:0/2118671085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55978f9e7680 0x55978f9f7730 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.491+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 >> 172.21.15.31:0/2118671085 conn(0x55978f95c200 msgr2=0x55978f9fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.492+0000 7fbe957fa700 0 client.17181 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.492+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 shutdown_connections 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.492+0000 7fbec7b7ab80 1 -- 172.21.15.31:0/2118671085 wait complete. 2023-10-06T21:32:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.492+0000 7fbec7b7ab80 20 client.17181 trim_cache size 0 max 16384 2023-10-06T21:32:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:32:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:32:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:32:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:32:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 msgr2=0x7fbe980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 0x7fbe980a3b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbea40077e0 tx=0x7fbea400d000 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 msgr2=0x7fbeac196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea8004730 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 shutdown_connections 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbeac19b780 0x7fbeac19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeac0ffff0 0x7fbeac196de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbe980a1670 0x7fbe980a3b30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 --2- 172.21.15.31:0/2010967859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeac104f30 0x7fbeac197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 >> 172.21.15.31:0/2010967859 conn(0x7fbeac0756e0 msgr2=0x7fbeac068470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 shutdown_connections 2023-10-06T21:32:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:32:54.493+0000 7fbeb2f9d700 1 -- 172.21.15.31:0/2010967859 wait complete. 2023-10-06T21:32:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:54 smithi031 ceph-mon[98539]: pgmap v374: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 669 KiB/s wr, 73 op/s 2023-10-06T21:32:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:54 smithi146 ceph-mon[102809]: pgmap v374: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 669 KiB/s wr, 73 op/s 2023-10-06T21:32:55.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 186221 1.49 MB/sec execute 525 sec latency 25.916 ms 2023-10-06T21:32:56.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 186597 1.49 MB/sec execute 526 sec latency 26.903 ms 2023-10-06T21:32:56.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:56 smithi142 ceph-mon[102951]: pgmap v375: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:56 smithi031 ceph-mon[98539]: pgmap v375: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:56 smithi146 ceph-mon[102809]: pgmap v375: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:57.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 186911 1.49 MB/sec execute 527 sec latency 30.461 ms 2023-10-06T21:32:58.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 187420 1.50 MB/sec execute 528 sec latency 16.474 ms 2023-10-06T21:32:58.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:32:58 smithi142 ceph-mon[102951]: pgmap v376: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:32:58 smithi031 ceph-mon[98539]: pgmap v376: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:32:58 smithi146 ceph-mon[102809]: pgmap v376: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:32:59.365 INFO:tasks.workunit.client.0.smithi031.stdout: 1 187776 1.50 MB/sec execute 529 sec latency 18.980 ms 2023-10-06T21:33:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:00 smithi146 ceph-mon[102809]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-10-06T21:33:00.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188073 1.50 MB/sec execute 530 sec latency 143.565 ms 2023-10-06T21:33:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:00 smithi142 ceph-mon[102951]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-10-06T21:33:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:00 smithi031 ceph-mon[98539]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-10-06T21:33:01.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188073 1.50 MB/sec execute 531 sec latency 1143.702 ms 2023-10-06T21:33:02.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188073 1.50 MB/sec execute 532 sec latency 2143.751 ms 2023-10-06T21:33:02.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:02 smithi142 ceph-mon[102951]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 735 KiB/s wr, 79 op/s 2023-10-06T21:33:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:02 smithi031 ceph-mon[98539]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 735 KiB/s wr, 79 op/s 2023-10-06T21:33:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:02 smithi146 ceph-mon[102809]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 735 KiB/s wr, 79 op/s 2023-10-06T21:33:03.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188073 1.49 MB/sec execute 533 sec latency 3143.818 ms 2023-10-06T21:33:04.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188073 1.49 MB/sec execute 534 sec latency 4143.911 ms 2023-10-06T21:33:04.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:04 smithi142 ceph-mon[102951]: pgmap v379: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1024 KiB/s wr, 102 op/s 2023-10-06T21:33:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:04 smithi031 ceph-mon[98539]: pgmap v379: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1024 KiB/s wr, 102 op/s 2023-10-06T21:33:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:04 smithi146 ceph-mon[102809]: pgmap v379: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1024 KiB/s wr, 102 op/s 2023-10-06T21:33:05.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188140 1.49 MB/sec execute 535 sec latency 4238.224 ms 2023-10-06T21:33:06.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188140 1.49 MB/sec execute 536 sec latency 1559.857 ms 2023-10-06T21:33:06.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:06 smithi142 ceph-mon[102951]: pgmap v380: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:06 smithi031 ceph-mon[98539]: pgmap v380: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:06 smithi146 ceph-mon[102809]: pgmap v380: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:07.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188140 1.48 MB/sec execute 537 sec latency 2559.969 ms 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 -- 172.21.15.31:0/1869103462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8181010d0 msgr2=0x7fb8181014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 --2- 172.21.15.31:0/1869103462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8181010d0 0x7fb8181014b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009990 tx=0x7fb80c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 -- 172.21.15.31:0/1869103462 shutdown_connections 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 --2- 172.21.15.31:0/1869103462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 0x7fb81810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 --2- 172.21.15.31:0/1869103462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8181010d0 0x7fb8181014b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 --2- 172.21.15.31:0/1869103462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb818101a80 0x7fb818105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 -- 172.21.15.31:0/1869103462 >> 172.21.15.31:0/1869103462 conn(0x7fb8180fc950 msgr2=0x7fb8180fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 -- 172.21.15.31:0/1869103462 shutdown_connections 2023-10-06T21:33:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.653+0000 7fb81edff700 1 -- 172.21.15.31:0/1869103462 wait complete. 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.654+0000 7fb81edff700 1 Processor -- start 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 -- start start 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 0x7fb818199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 0x7fb818199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819e760 con 0x7fb81819dce0 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819e8d0 con 0x7fb8181061f0 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81edff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819ea90 con 0x7fb8181010d0 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39008/0 (socket says 172.21.15.31:39008) 2023-10-06T21:33:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81cb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 0x7fb818199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.655+0000 7fb81d39c700 1 -- 172.21.15.31:0/614786373 learned_addr learned my addr 172.21.15.31:0/614786373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb817fff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 0x7fb818199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 -- 172.21.15.31:0/614786373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 msgr2=0x7fb818199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 0x7fb818199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 -- 172.21.15.31:0/614786373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 msgr2=0x7fb818199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 0x7fb818199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 -- 172.21.15.31:0/614786373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0095f0 con 0x7fb81819dce0 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 0x7fb818199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb81d39c700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ef30 tx=0x7fb81000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb815ffb700 1 -- 172.21.15.31:0/614786373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810018070 con 0x7fb81819dce0 2023-10-06T21:33:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.656+0000 7fb815ffb700 1 -- 172.21.15.31:0/614786373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81000f040 con 0x7fb81819dce0 2023-10-06T21:33:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.657+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8181a5f90 con 0x7fb81819dce0 2023-10-06T21:33:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.657+0000 7fb815ffb700 1 -- 172.21.15.31:0/614786373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000cbc0 con 0x7fb81819dce0 2023-10-06T21:33:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.657+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181a6420 con 0x7fb81819dce0 2023-10-06T21:33:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.660+0000 7fb815ffb700 1 -- 172.21.15.31:0/614786373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb810007500 con 0x7fb81819dce0 2023-10-06T21:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 Processor -- start 2023-10-06T21:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 -- start start 2023-10-06T21:33:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb815ffb700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 0x7fb8080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb815ffb700 1 -- 172.21.15.31:0/614786373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb8100ccc10 con 0x7fb81819dce0 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeabfefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeab68400 con 0x563aeac0a3d0 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeab59bc0 con 0x563aeabff520 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb832702b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeab574e0 con 0x563aeabfec00 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.662+0000 7fb817fff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 0x7fb8080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39020/0 (socket says 172.21.15.31:39020) 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 -- 172.21.15.31:0/1035047097 learned_addr learned my addr 172.21.15.31:0/1035047097 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeabfefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb817fff700 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 -- 172.21.15.31:0/1035047097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 msgr2=0x563aeabfefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeabfefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 -- 172.21.15.31:0/1035047097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 msgr2=0x563aeac09e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac09e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb817fff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 0x7fb8080a3ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb81819a950 tx=0x7fb80400f3c0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 -- 172.21.15.31:0/1035047097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aea97dd80 con 0x563aeac0a3d0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.663+0000 7fb81d39c700 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb8100090a0 tx=0x7fb8100997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1320605c18ab43dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.664+0000 7fb802ffd700 1 -- 172.21.15.31:0/1035047097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81009a2b0 con 0x563aeac0a3d0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.664+0000 7fb802ffd700 1 -- 172.21.15.31:0/1035047097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb810099bb0 con 0x563aeac0a3d0 2023-10-06T21:33:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.664+0000 7fb802ffd700 1 -- 172.21.15.31:0/1035047097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810099e10 con 0x563aeac0a3d0 2023-10-06T21:33:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.664+0000 7fb832702b80 1 -- 172.21.15.31:0/1035047097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 msgr2=0x563aeac0c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.664+0000 7fb832702b80 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb8100090a0 tx=0x7fb8100997c0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 -- 172.21.15.31:0/1035047097 shutdown_connections 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac0c7c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeabfefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 --2- 172.21.15.31:0/1035047097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac09e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 -- 172.21.15.31:0/1035047097 >> 172.21.15.31:0/1035047097 conn(0x563aeab74200 msgr2=0x563aeabfb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 -- 172.21.15.31:0/1035047097 shutdown_connections 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 -- 172.21.15.31:0/1035047097 wait complete. 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.665+0000 7fb832702b80 1 -- start start 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 Processor -- start 2023-10-06T21:33:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeac07200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac07740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeab52dd0 con 0x563aeac0a3d0 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeaaef6b0 con 0x563aeabff520 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.666+0000 7fb832702b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeaaef350 con 0x563aeabfec00 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb832702b80 10 client.0 _ll_register_callbacks cb 0x563aea94c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81cb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeac07200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39036/0 (socket says 172.21.15.31:39036) 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 -- 172.21.15.31:0/2991121796 learned_addr learned my addr 172.21.15.31:0/2991121796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb817fff700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac07740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 -- 172.21.15.31:0/2991121796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 msgr2=0x563aeac07200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeac07200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 -- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 msgr2=0x563aeac07740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac07740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb81d39c700 1 -- 172.21.15.31:0/2991121796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181a6420 con 0x563aeac0a3d0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.667+0000 7fb817fff700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac07740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb81d39c700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8100ca810 tx=0x7fb810009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b52886ae1e4a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81009dd00 con 0x563aeac0a3d0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb810084040 con 0x563aeac0a3d0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aeabf2d50 con 0x563aeac0a3d0 2023-10-06T21:33:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb810099e10 con 0x563aeac0a3d0 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 10 client.17193 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563aea966c00 con 0x563aeac0a3d0 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81007c080 con 0x563aeac0a3d0 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fb8100b20d0 con 0x563aeac0a3d0 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.668+0000 7fb832702b80 20 client.17193 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.669+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac19b50 0x563aeac1bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.669+0000 7fb832702b80 4 client.17193 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.669+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aeab4bdb0 con 0x563aeac19b50 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.669+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac19b50 0x563aeac1bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.670+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac19b50 0x563aeac1bf70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555c3fbae1487b36 server_cookie=620fbabf651d650a in_seq=0 out_seq=0 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.670+0000 7fb7f7fff700 10 client.17193 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb7f7fff700 1 -- 172.21.15.31:0/2991121796 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563aeab4bdb0 con 0x563aeac19b50 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb7f7fff700 10 client.17193 handle_command_reply: tid=0 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 client.17193 shutdown 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 msgr2=0x563aeac07c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8100ca810 tx=0x7fb810009ed0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 shutdown_connections 2023-10-06T21:33:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac19b50 0x563aeac1bf70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeac0a3d0 0x563aeac07c80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeabfec00 0x563aeac07200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb7f7fff700 0 client.17193 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 --2- 172.21.15.31:0/2991121796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabff520 0x563aeac07740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 >> 172.21.15.31:0/2991121796 conn(0x563aeab74200 msgr2=0x563aeabfac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 shutdown_connections 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.671+0000 7fb832702b80 1 -- 172.21.15.31:0/2991121796 wait complete. 2023-10-06T21:33:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.672+0000 7fb832702b80 20 client.17193 trim_cache size 0 max 16384 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 Processor -- start 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 -- start start 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea9d38c0 con 0x563aeabff520 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeac05760 con 0x563aeabfec00 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.687+0000 7fb832702b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeac03070 con 0x563aeac0a3d0 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39042/0 (socket says 172.21.15.31:39042) 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 -- 172.21.15.31:0/1342261338 learned_addr learned my addr 172.21.15.31:0/1342261338 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb81d39c700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 -- 172.21.15.31:0/1342261338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 msgr2=0x563aeac060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac01b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 -- 172.21.15.31:0/1342261338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 msgr2=0x563aeac01b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac01b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 -- 172.21.15.31:0/1342261338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aea9a41f0 con 0x563aeabff520 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb817fff700 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ff20 tx=0x7fb804014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d7548e62851abc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.688+0000 7fb8037fe700 1 -- 172.21.15.31:0/1342261338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804022070 con 0x563aeabff520 2023-10-06T21:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb8037fe700 1 -- 172.21.15.31:0/1342261338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb804016070 con 0x563aeabff520 2023-10-06T21:33:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb8037fe700 1 -- 172.21.15.31:0/1342261338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8040278a0 con 0x563aeabff520 2023-10-06T21:33:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 -- 172.21.15.31:0/1342261338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 msgr2=0x563aeac07ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ff20 tx=0x7fb804014000 comp rx=0 tx=0).stop 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 -- 172.21.15.31:0/1342261338 shutdown_connections 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 --2- 172.21.15.31:0/1342261338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac07ab0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 -- 172.21.15.31:0/1342261338 >> 172.21.15.31:0/1342261338 conn(0x563aeab74200 msgr2=0x563aeac059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 -- 172.21.15.31:0/1342261338 shutdown_connections 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.689+0000 7fb832702b80 1 -- 172.21.15.31:0/1342261338 wait complete. 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 -- start start 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 Processor -- start 2023-10-06T21:33:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeac0ff40 con 0x563aeabff520 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeac100b0 con 0x563aeabfec00 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aeac6d950 con 0x563aeac0a3d0 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.690+0000 7fb832702b80 10 client.0 _ll_register_callbacks cb 0x7fb8100097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39050/0 (socket says 172.21.15.31:39050) 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 -- 172.21.15.31:0/29680540 learned_addr learned my addr 172.21.15.31:0/29680540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb81d39c700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 -- 172.21.15.31:0/29680540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 msgr2=0x563aeac0f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac0f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 -- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 msgr2=0x563aeac10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 -- 172.21.15.31:0/29680540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80400fbd0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb817fff700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb804005de0 tx=0x7fb804005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4a2bdd0be68a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.691+0000 7fb7f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804022040 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb810099bb0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb8040106c0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8040108a0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb832702b80 10 client.17199 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563aea97c250 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb810099bb0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.692+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fb804027ab0 con 0x563aeabff520 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 10 client.17199 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 20 client.17199 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac6ef50 0x563aeac71370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 4 client.17199 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563aeab4bad0 con 0x563aeac6ef50 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac6ef50 0x563aeac71370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb81cb9b700 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac6ef50 0x563aeac71370 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb777dcd55ed5363 server_cookie=8b3a52fb1754c7e9 in_seq=0 out_seq=0 2023-10-06T21:33:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.693+0000 7fb7f7fff700 10 client.17199 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb7f7fff700 1 -- 172.21.15.31:0/29680540 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563aeab4bad0 con 0x563aeac6ef50 2023-10-06T21:33:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb7f7fff700 10 client.17199 handle_command_reply: tid=0 2023-10-06T21:33:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 client.17199 shutdown 2023-10-06T21:33:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 msgr2=0x563aeac0f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb804005de0 tx=0x7fb804005640 comp rx=0 tx=0).stop 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 shutdown_connections 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563aeac0a3d0 0x563aeac0f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aeabfec00 0x563aeac10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563aeac6ef50 0x563aeac71370 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 --2- 172.21.15.31:0/29680540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeabff520 0x563aeac0f330 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb7f7fff700 0 client.17199 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 >> 172.21.15.31:0/29680540 conn(0x563aeab74200 msgr2=0x563aeac06240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 shutdown_connections 2023-10-06T21:33:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 1 -- 172.21.15.31:0/29680540 wait complete. 2023-10-06T21:33:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.721+0000 7fb832702b80 20 client.17199 trim_cache size 0 max 16384 2023-10-06T21:33:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:33:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:33:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:33:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:33:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 msgr2=0x7fb8080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 0x7fb8080a3ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb81819a950 tx=0x7fb80400f3c0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 msgr2=0x7fb81819e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ef30 tx=0x7fb81000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 shutdown_connections 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb8080a15f0 0x7fb8080a3ab0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb81819dce0 0x7fb81819e160 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8181010d0 0x7fb818199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 --2- 172.21.15.31:0/614786373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8181061f0 0x7fb818199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 >> 172.21.15.31:0/614786373 conn(0x7fb8180fc950 msgr2=0x7fb81810d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 shutdown_connections 2023-10-06T21:33:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:07.722+0000 7fb81edff700 1 -- 172.21.15.31:0/614786373 wait complete. 2023-10-06T21:33:08.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188140 1.48 MB/sec execute 538 sec latency 3560.084 ms 2023-10-06T21:33:08.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:08 smithi142 ceph-mon[102951]: pgmap v381: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:08 smithi031 ceph-mon[98539]: pgmap v381: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:08 smithi146 ceph-mon[102809]: pgmap v381: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 710 KiB/s wr, 67 op/s 2023-10-06T21:33:09.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188140 1.48 MB/sec execute 539 sec latency 4560.219 ms 2023-10-06T21:33:10.366 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188315 1.48 MB/sec execute 540 sec latency 4655.545 ms 2023-10-06T21:33:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:10 smithi031 ceph-mon[98539]: pgmap v382: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 808 KiB/s wr, 94 op/s 2023-10-06T21:33:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:10 smithi031 ceph-mon[98539]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:33:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:10 smithi146 ceph-mon[102809]: pgmap v382: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 808 KiB/s wr, 94 op/s 2023-10-06T21:33:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:10 smithi146 ceph-mon[102809]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:33:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:10 smithi142 ceph-mon[102951]: pgmap v382: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 808 KiB/s wr, 94 op/s 2023-10-06T21:33:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:10 smithi142 ceph-mon[102951]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:33:11.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188480 1.48 MB/sec execute 541 sec latency 21.022 ms 2023-10-06T21:33:12.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 188741 1.47 MB/sec execute 542 sec latency 24.011 ms 2023-10-06T21:33:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:12 smithi031 ceph-mon[98539]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 464 KiB/s wr, 60 op/s 2023-10-06T21:33:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:12 smithi146 ceph-mon[102809]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 464 KiB/s wr, 60 op/s 2023-10-06T21:33:12.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:12 smithi142 ceph-mon[102951]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 464 KiB/s wr, 60 op/s 2023-10-06T21:33:13.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 189099 1.47 MB/sec execute 543 sec latency 16.423 ms 2023-10-06T21:33:14.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 189460 1.48 MB/sec execute 544 sec latency 24.974 ms 2023-10-06T21:33:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:14 smithi031 ceph-mon[98539]: pgmap v385: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 505 KiB/s wr, 74 op/s 2023-10-06T21:33:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:14 smithi031 ceph-mon[98539]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:33:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:14 smithi146 ceph-mon[102809]: pgmap v385: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 505 KiB/s wr, 74 op/s 2023-10-06T21:33:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:14 smithi146 ceph-mon[102809]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:33:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:14 smithi142 ceph-mon[102951]: pgmap v385: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 505 KiB/s wr, 74 op/s 2023-10-06T21:33:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:14 smithi142 ceph-mon[102951]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:33:15.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 189782 1.47 MB/sec execute 545 sec latency 25.972 ms 2023-10-06T21:33:16.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 190155 1.47 MB/sec execute 546 sec latency 29.311 ms 2023-10-06T21:33:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:16 smithi031 ceph-mon[98539]: pgmap v387: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 631 KiB/s wr, 92 op/s 2023-10-06T21:33:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:16 smithi146 ceph-mon[102809]: pgmap v387: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 631 KiB/s wr, 92 op/s 2023-10-06T21:33:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:16 smithi142 ceph-mon[102951]: pgmap v387: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 631 KiB/s wr, 92 op/s 2023-10-06T21:33:17.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 190462 1.47 MB/sec execute 547 sec latency 19.718 ms 2023-10-06T21:33:18.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 190941 1.48 MB/sec execute 548 sec latency 18.539 ms 2023-10-06T21:33:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:18 smithi031 ceph-mon[98539]: pgmap v388: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 484 KiB/s wr, 51 op/s 2023-10-06T21:33:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:18 smithi146 ceph-mon[102809]: pgmap v388: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 484 KiB/s wr, 51 op/s 2023-10-06T21:33:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:18 smithi142 ceph-mon[102951]: pgmap v388: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 484 KiB/s wr, 51 op/s 2023-10-06T21:33:19.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191322 1.48 MB/sec execute 549 sec latency 19.475 ms 2023-10-06T21:33:20.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:20 smithi146 ceph-mon[102809]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 879 KiB/s wr, 89 op/s 2023-10-06T21:33:20.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191620 1.48 MB/sec execute 550 sec latency 141.268 ms 2023-10-06T21:33:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:20 smithi142 ceph-mon[102951]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 879 KiB/s wr, 89 op/s 2023-10-06T21:33:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:20 smithi031 ceph-mon[98539]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 879 KiB/s wr, 89 op/s 2023-10-06T21:33:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 -- 172.21.15.31:0/2931921091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c201010d0 msgr2=0x7f8c201014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2931921091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c201010d0 0x7f8c201014b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009990 tx=0x7f8c10009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 -- 172.21.15.31:0/2931921091 shutdown_connections 2023-10-06T21:33:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2931921091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 0x7f8c2010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2931921091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c201010d0 0x7f8c201014b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2931921091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c20101a80 0x7f8c20105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 -- 172.21.15.31:0/2931921091 >> 172.21.15.31:0/2931921091 conn(0x7f8c200fc950 msgr2=0x7f8c200fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 -- 172.21.15.31:0/2931921091 shutdown_connections 2023-10-06T21:33:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.881+0000 7f8c259ff700 1 -- 172.21.15.31:0/2931921091 wait complete. 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.882+0000 7f8c259ff700 1 Processor -- start 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.882+0000 7f8c259ff700 1 -- start start 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 0x7f8c201990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 0x7f8c20199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2019e6f0 con 0x7f8c2019dc70 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2019e860 con 0x7f8c201010d0 2023-10-06T21:33:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c259ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2019ea20 con 0x7f8c201061f0 2023-10-06T21:33:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 0x7f8c20199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47270/0 (socket says 172.21.15.31:47270) 2023-10-06T21:33:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2371375333 learned_addr learned my addr 172.21.15.31:0/2371375333 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 0x7f8c201990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2371375333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 msgr2=0x7f8c20199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 0x7f8c20199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2371375333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 msgr2=0x7f8c201990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 0x7f8c201990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2371375333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100095f0 con 0x7f8c2019dc70 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.883+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 0x7f8c201990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8c18007ee0 tx=0x7f8c1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c249fd700 1 -- 172.21.15.31:0/2371375333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18017070 con 0x7f8c2019dc70 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c249fd700 1 -- 172.21.15.31:0/2371375333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c1800f040 con 0x7f8c2019dc70 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c201a5f80 con 0x7f8c2019dc70 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c249fd700 1 -- 172.21.15.31:0/2371375333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18005710 con 0x7f8c2019dc70 2023-10-06T21:33:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.884+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c201a6410 con 0x7f8c2019dc70 2023-10-06T21:33:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.887+0000 7f8c249fd700 1 -- 172.21.15.31:0/2371375333 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8c18005870 con 0x7f8c2019dc70 2023-10-06T21:33:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 Processor -- start 2023-10-06T21:33:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c249fd700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 0x7f8c080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c249fd700 1 -- 172.21.15.31:0/2371375333 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8c180cc940 con 0x7f8c2019dc70 2023-10-06T21:33:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 -- start start 2023-10-06T21:33:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 0x7f8c080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 0x557270f337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e8f400 con 0x557270f25c00 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e80bc0 con 0x557270f313d0 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.889+0000 7f8c39230b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e7e4e0 con 0x557270f26520 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 0x7f8c080a3ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c2019a8e0 tx=0x7f8c1400a380 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47280/0 (socket says 172.21.15.31:47280) 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 -- 172.21.15.31:0/2920125397 learned_addr learned my addr 172.21.15.31:0/2920125397 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 0x557270f337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 -- 172.21.15.31:0/2920125397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 msgr2=0x557270f30e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f30e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 -- 172.21.15.31:0/2920125397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 msgr2=0x557270f337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 0x557270f337c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 -- 172.21.15.31:0/2920125397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270ca4d80 con 0x557270f25c00 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 0x557270f337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.890+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b5c0 tx=0x7f8c10009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ff0b488681c5f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.891+0000 7f8c05ffb700 1 -- 172.21.15.31:0/2920125397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10005230 con 0x557270f25c00 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.891+0000 7f8c05ffb700 1 -- 172.21.15.31:0/2920125397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c1001e040 con 0x557270f25c00 2023-10-06T21:33:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.891+0000 7f8c05ffb700 1 -- 172.21.15.31:0/2920125397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000b9d0 con 0x557270f25c00 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.891+0000 7f8c39230b80 1 -- 172.21.15.31:0/2920125397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 msgr2=0x557270f25fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.891+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b5c0 tx=0x7f8c10009f60 comp rx=0 tx=0).stop 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 -- 172.21.15.31:0/2920125397 shutdown_connections 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f313d0 0x557270f337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f25fe0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2920125397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f30e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 -- 172.21.15.31:0/2920125397 >> 172.21.15.31:0/2920125397 conn(0x557270e9b200 msgr2=0x557270f22080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 -- 172.21.15.31:0/2920125397 shutdown_connections 2023-10-06T21:33:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 -- 172.21.15.31:0/2920125397 wait complete. 2023-10-06T21:33:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.892+0000 7f8c39230b80 1 -- start start 2023-10-06T21:33:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 Processor -- start 2023-10-06T21:33:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 0x557270f30dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 0x557270f2e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e79dd0 con 0x557270f313d0 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e166b0 con 0x557270f25c00 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.893+0000 7f8c39230b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e16350 con 0x557270f26520 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c39230b80 10 client.0 _ll_register_callbacks cb 0x557270c737a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 0x557270f2e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 0x557270f30dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46936/0 (socket says 172.21.15.31:46936) 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1e7fc700 1 -- 172.21.15.31:0/3857662941 learned_addr learned my addr 172.21.15.31:0/3857662941 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 -- 172.21.15.31:0/3857662941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 msgr2=0x557270f2e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 -- 172.21.15.31:0/3857662941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 msgr2=0x557270f2e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 0x557270f2e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1effd700 1 -- 172.21.15.31:0/3857662941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2010e5e0 con 0x557270f25c00 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 0x557270f2e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.894+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8c1effd700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 0x557270f30dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c10025040 tx=0x7f8c1000ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45f94fd0ce254ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8bfeffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10003820 con 0x557270f25c00 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8c10003980 con 0x557270f25c00 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557270f19d50 con 0x557270f25c00 2023-10-06T21:33:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c10003b40 con 0x557270f25c00 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8c39230b80 10 client.26900 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.895+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557270c8dc00 con 0x557270f25c00 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557270f19d50 con 0x557270f25c00 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8c10021070 con 0x557270f25c00 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 10 client.26900 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 20 client.26900 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f408c0 0x557270f42ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 4 client.26900 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.896+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557270e72db0 con 0x557270f408c0 2023-10-06T21:33:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.897+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f408c0 0x557270f42ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.897+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f408c0 0x557270f42ce0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105d4379f47d41b4 server_cookie=a1545f5a4f3150b0 in_seq=0 out_seq=0 2023-10-06T21:33:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.897+0000 7f8bfeffd700 10 client.26900 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8bfeffd700 1 -- 172.21.15.31:0/3857662941 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557270e72db0 con 0x557270f408c0 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8bfeffd700 10 client.26900 handle_command_reply: tid=0 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 client.26900 shutdown 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 msgr2=0x557270f30dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 0x557270f30dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c10025040 tx=0x7f8c1000ba00 comp rx=0 tx=0).stop 2023-10-06T21:33:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 shutdown_connections 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f408c0 0x557270f42ce0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f313d0 0x557270f2e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f25c00 0x557270f30dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 --2- 172.21.15.31:0/3857662941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f26520 0x557270f2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8bfeffd700 0 client.26900 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 >> 172.21.15.31:0/3857662941 conn(0x557270e9b200 msgr2=0x557270f21c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 shutdown_connections 2023-10-06T21:33:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 1 -- 172.21.15.31:0/3857662941 wait complete. 2023-10-06T21:33:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.898+0000 7f8c39230b80 20 client.26900 trim_cache size 0 max 16384 2023-10-06T21:33:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 Processor -- start 2023-10-06T21:33:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 -- start start 2023-10-06T21:33:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f28270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270cfa8c0 con 0x557270f25c00 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f2ec50 con 0x557270f26520 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c39230b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f22dc0 con 0x557270f313d0 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47290/0 (socket says 172.21.15.31:47290) 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.914+0000 7f8c1effd700 1 -- 172.21.15.31:0/1182548590 learned_addr learned my addr 172.21.15.31:0/1182548590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f28270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 -- 172.21.15.31:0/1182548590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 msgr2=0x557270f287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 -- 172.21.15.31:0/1182548590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 msgr2=0x557270f28270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f28270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 -- 172.21.15.31:0/1182548590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270ccb1f0 con 0x557270f25c00 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f28270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c1effd700 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8c100036c0 tx=0x7f8c10005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892189b44d2b423c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c067fc700 1 -- 172.21.15.31:0/1182548590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000ba70 con 0x557270f25c00 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c067fc700 1 -- 172.21.15.31:0/1182548590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c10005360 con 0x557270f25c00 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.915+0000 7f8c067fc700 1 -- 172.21.15.31:0/1182548590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10003db0 con 0x557270f25c00 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- 172.21.15.31:0/1182548590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 msgr2=0x557270f37ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8c100036c0 tx=0x7f8c10005a90 comp rx=0 tx=0).stop 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- 172.21.15.31:0/1182548590 shutdown_connections 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f25c00 0x557270f37ec0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 --2- 172.21.15.31:0/1182548590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f28270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- 172.21.15.31:0/1182548590 >> 172.21.15.31:0/1182548590 conn(0x557270e9b200 msgr2=0x557270f27b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- 172.21.15.31:0/1182548590 shutdown_connections 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- 172.21.15.31:0/1182548590 wait complete. 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.916+0000 7f8c39230b80 1 -- start start 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 Processor -- start 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f93fd0 con 0x557270f916a0 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f94110 con 0x557270f26520 2023-10-06T21:33:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f942d0 con 0x557270f313d0 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c39230b80 10 client.0 _ll_register_callbacks cb 0x7f8c10009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c1effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56356/0 (socket says 172.21.15.31:56356) 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47300/0 (socket says 172.21.15.31:47300) 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.917+0000 7f8c1effd700 1 -- 172.21.15.31:0/2295649280 learned_addr learned my addr 172.21.15.31:0/2295649280 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1e7fc700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2295649280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 msgr2=0x557270f36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 msgr2=0x557270f35c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 -- 172.21.15.31:0/2295649280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557270c8dc00 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c1f7fe700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8c180c9fc0 tx=0x7f8c18099390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5695833f8962ce44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8bfeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18099cb0 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8c1800dc60 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.918+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c1001e040 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c18099f50 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 10 client.17217 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557270ca3250 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c1001e040 con 0x557270f916a0 2023-10-06T21:33:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8c18098e60 con 0x557270f916a0 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 10 client.17217 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 20 client.17217 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f95ab0 0x557270f97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 4 client.17217 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557270e72ad0 con 0x557270f95ab0 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.919+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f95ab0 0x557270f97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.920+0000 7f8c1effd700 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f95ab0 0x557270f97ed0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705b80ca4d5c9588 server_cookie=d6b0c6aa04c5d7d9 in_seq=0 out_seq=0 2023-10-06T21:33:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.920+0000 7f8bfeffd700 10 client.17217 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.944+0000 7f8bfeffd700 1 -- 172.21.15.31:0/2295649280 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557270e72ad0 con 0x557270f95ab0 2023-10-06T21:33:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.944+0000 7f8bfeffd700 10 client.17217 handle_command_reply: tid=0 2023-10-06T21:33:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.944+0000 7f8c39230b80 1 client.17217 shutdown 2023-10-06T21:33:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 msgr2=0x557270f93a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8c180c9fc0 tx=0x7f8c18099390 comp rx=0 tx=0).stop 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 shutdown_connections 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557270f95ab0 0x557270f97ed0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557270f916a0 0x557270f93a90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557270f26520 0x557270f35c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 --2- 172.21.15.31:0/2295649280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557270f313d0 0x557270f36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8bfeffd700 0 client.17217 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 >> 172.21.15.31:0/2295649280 conn(0x557270e9b200 msgr2=0x557270f283f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 shutdown_connections 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 1 -- 172.21.15.31:0/2295649280 wait complete. 2023-10-06T21:33:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.945+0000 7f8c39230b80 20 client.17217 trim_cache size 0 max 16384 2023-10-06T21:33:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:33:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:33:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:33:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 msgr2=0x7f8c080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 0x7f8c080a3ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c2019a8e0 tx=0x7f8c1400a380 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 msgr2=0x7f8c2019e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8c18007ee0 tx=0x7f8c1800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 shutdown_connections 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8c080a15f0 0x7f8c080a3ab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c2019dc70 0x7f8c2019e0f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c201010d0 0x7f8c201990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 --2- 172.21.15.31:0/2371375333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c201061f0 0x7f8c20199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 >> 172.21.15.31:0/2371375333 conn(0x7f8c200fc950 msgr2=0x7f8c200fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 shutdown_connections 2023-10-06T21:33:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:20.946+0000 7f8c259ff700 1 -- 172.21.15.31:0/2371375333 wait complete. 2023-10-06T21:33:21.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191620 1.48 MB/sec execute 551 sec latency 1141.348 ms 2023-10-06T21:33:22.367 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191620 1.48 MB/sec execute 552 sec latency 2141.473 ms 2023-10-06T21:33:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:22 smithi142 ceph-mon[102951]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 873 KiB/s wr, 88 op/s 2023-10-06T21:33:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:22 smithi031 ceph-mon[98539]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 873 KiB/s wr, 88 op/s 2023-10-06T21:33:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:22 smithi146 ceph-mon[102809]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 873 KiB/s wr, 88 op/s 2023-10-06T21:33:23.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191620 1.48 MB/sec execute 553 sec latency 3141.600 ms 2023-10-06T21:33:24.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191620 1.47 MB/sec execute 554 sec latency 4141.714 ms 2023-10-06T21:33:24.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:24 smithi142 ceph-mon[102951]: pgmap v391: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 866 KiB/s wr, 81 op/s 2023-10-06T21:33:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:24 smithi031 ceph-mon[98539]: pgmap v391: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 866 KiB/s wr, 81 op/s 2023-10-06T21:33:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:24 smithi146 ceph-mon[102809]: pgmap v391: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 866 KiB/s wr, 81 op/s 2023-10-06T21:33:25.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191687 1.47 MB/sec execute 555 sec latency 4233.335 ms 2023-10-06T21:33:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:25 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:25 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:25 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:26.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191687 1.47 MB/sec execute 556 sec latency 1596.033 ms 2023-10-06T21:33:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:26 smithi031 ceph-mon[98539]: pgmap v392: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 783 KiB/s wr, 73 op/s 2023-10-06T21:33:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:26 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:26 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:26 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:33:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:26 smithi146 ceph-mon[102809]: pgmap v392: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 783 KiB/s wr, 73 op/s 2023-10-06T21:33:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:26 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:26 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:26 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:33:26.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:26 smithi142 ceph-mon[102951]: pgmap v392: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 783 KiB/s wr, 73 op/s 2023-10-06T21:33:26.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:26 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:26.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:26 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:26.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:26 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:33:27.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191687 1.46 MB/sec execute 557 sec latency 2596.154 ms 2023-10-06T21:33:28.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191687 1.46 MB/sec execute 558 sec latency 3596.270 ms 2023-10-06T21:33:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:28 smithi031 ceph-mon[98539]: pgmap v393: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 721 KiB/s wr, 67 op/s 2023-10-06T21:33:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:28 smithi146 ceph-mon[102809]: pgmap v393: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 721 KiB/s wr, 67 op/s 2023-10-06T21:33:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:28 smithi142 ceph-mon[102951]: pgmap v393: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 721 KiB/s wr, 67 op/s 2023-10-06T21:33:29.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191687 1.46 MB/sec execute 559 sec latency 4596.408 ms 2023-10-06T21:33:30.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:30 smithi146 ceph-mon[102809]: pgmap v394: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 748 KiB/s wr, 76 op/s 2023-10-06T21:33:30.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 191861 1.46 MB/sec execute 560 sec latency 4688.904 ms 2023-10-06T21:33:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:30 smithi142 ceph-mon[102951]: pgmap v394: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 748 KiB/s wr, 76 op/s 2023-10-06T21:33:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:30 smithi031 ceph-mon[98539]: pgmap v394: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 748 KiB/s wr, 76 op/s 2023-10-06T21:33:31.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 192026 1.46 MB/sec execute 561 sec latency 17.963 ms 2023-10-06T21:33:32.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 192284 1.45 MB/sec execute 562 sec latency 23.817 ms 2023-10-06T21:33:32.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:32 smithi142 ceph-mon[102951]: pgmap v395: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 343 KiB/s wr, 36 op/s 2023-10-06T21:33:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:32 smithi031 ceph-mon[98539]: pgmap v395: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 343 KiB/s wr, 36 op/s 2023-10-06T21:33:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:32 smithi146 ceph-mon[102809]: pgmap v395: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 343 KiB/s wr, 36 op/s 2023-10-06T21:33:33.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 192640 1.45 MB/sec execute 563 sec latency 18.343 ms 2023-10-06T21:33:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 -- 172.21.15.31:0/3251608861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 msgr2=0x7fee88103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 --2- 172.21.15.31:0/3251608861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee88103700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fee84009990 tx=0x7fee84009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 -- 172.21.15.31:0/3251608861 shutdown_connections 2023-10-06T21:33:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 --2- 172.21.15.31:0/3251608861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee88108440 0x7fee8810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 --2- 172.21.15.31:0/3251608861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee88103700 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 --2- 172.21.15.31:0/3251608861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88103cd0 0x7fee88107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 -- 172.21.15.31:0/3251608861 >> 172.21.15.31:0/3251608861 conn(0x7fee880feb60 msgr2=0x7fee88100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 -- 172.21.15.31:0/3251608861 shutdown_connections 2023-10-06T21:33:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.106+0000 7fee901ff700 1 -- 172.21.15.31:0/3251608861 wait complete. 2023-10-06T21:33:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 Processor -- start 2023-10-06T21:33:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 -- start start 2023-10-06T21:33:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 0x7fee88199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 0x7fee8819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.107+0000 7fee901ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819e6f0 con 0x7fee88103320 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee901ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819e860 con 0x7fee8819dc70 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee901ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819ea20 con 0x7fee88108440 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43746/0 (socket says 172.21.15.31:43746) 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 -- 172.21.15.31:0/1672659468 learned_addr learned my addr 172.21.15.31:0/1672659468 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 0x7fee88199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8e79c700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 0x7fee8819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 -- 172.21.15.31:0/1672659468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 msgr2=0x7fee88199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 0x7fee88199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 -- 172.21.15.31:0/1672659468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 msgr2=0x7fee8819e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 0x7fee8819e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8df9b700 1 -- 172.21.15.31:0/1672659468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840095f0 con 0x7fee88103320 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8e79c700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 0x7fee8819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.108+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 0x7fee88199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee7effd700 1 -- 172.21.15.31:0/1672659468 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8401d070 con 0x7fee88103320 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee7effd700 1 -- 172.21.15.31:0/1672659468 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee84022470 con 0x7fee88103320 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee881a5f20 con 0x7fee88103320 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee7effd700 1 -- 172.21.15.31:0/1672659468 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84017650 con 0x7fee88103320 2023-10-06T21:33:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.109+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee881a62e0 con 0x7fee88103320 2023-10-06T21:33:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.113+0000 7fee7effd700 1 -- 172.21.15.31:0/1672659468 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fee840225e0 con 0x7fee88103320 2023-10-06T21:33:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.114+0000 7feea3b35b80 1 Processor -- start 2023-10-06T21:33:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.114+0000 7feea3b35b80 1 -- start start 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a32be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a32e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a28a400 con 0x55b58a320c00 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a27bbc0 con 0x55b58a32c3d0 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7feea3b35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a2794e0 con 0x55b58a321520 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee8e79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee8e79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43748/0 (socket says 172.21.15.31:43748) 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee8e79c700 1 -- 172.21.15.31:0/2079020023 learned_addr learned my addr 172.21.15.31:0/2079020023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee8d79a700 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a32e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee7effd700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 0x7fee740a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.115+0000 7fee8df9b700 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a32be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 -- 172.21.15.31:0/2079020023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 msgr2=0x55b58a32be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a32be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 -- 172.21.15.31:0/2079020023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 msgr2=0x55b58a32e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a32e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 -- 172.21.15.31:0/2079020023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58a09fd80 con 0x55b58a320c00 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee7effd700 1 -- 172.21.15.31:0/1672659468 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee840ce7f0 con 0x7fee88103320 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 0x7fee740a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee8e79c700 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee80007530 tx=0x7fee80007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc1c4a601a59339 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee73fff700 1 -- 172.21.15.31:0/2079020023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8000e040 con 0x55b58a320c00 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee73fff700 1 -- 172.21.15.31:0/2079020023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee8000f040 con 0x55b58a320c00 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.116+0000 7fee73fff700 1 -- 172.21.15.31:0/2079020023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee800176f0 con 0x55b58a320c00 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 -- 172.21.15.31:0/2079020023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 msgr2=0x55b58a320fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7fee8e79c700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 0x7fee740a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee80020000 tx=0x7fee80021000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee80007530 tx=0x7fee80007840 comp rx=0 tx=0).stop 2023-10-06T21:33:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 -- 172.21.15.31:0/2079020023 shutdown_connections 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a32e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a320fe0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2079020023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a32be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 -- 172.21.15.31:0/2079020023 >> 172.21.15.31:0/2079020023 conn(0x55b58a296200 msgr2=0x55b58a31d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 -- 172.21.15.31:0/2079020023 shutdown_connections 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.117+0000 7feea3b35b80 1 -- 172.21.15.31:0/2079020023 wait complete. 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.118+0000 7feea3b35b80 1 -- start start 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.118+0000 7feea3b35b80 1 Processor -- start 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a329610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a329b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a274dd0 con 0x55b58a320c00 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a2116b0 con 0x55b58a32c3d0 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a211350 con 0x55b58a321520 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7feea3b35b80 10 client.0 _ll_register_callbacks cb 0x55b58a06e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43752/0 (socket says 172.21.15.31:43752) 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 -- 172.21.15.31:0/1707697168 learned_addr learned my addr 172.21.15.31:0/1707697168 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a329610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8e79c700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a329b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 -- 172.21.15.31:0/1707697168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 msgr2=0x55b58a329610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a329610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 -- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 msgr2=0x55b58a329b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a329b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8df9b700 1 -- 172.21.15.31:0/1707697168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee80009140 con 0x55b58a320c00 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.119+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a329610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7fee8df9b700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee840cbed0 tx=0x7fee8406d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353e20d32ec2718d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7fee717fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840bb0b0 con 0x55b58a320c00 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b58a314d50 con 0x55b58a320c00 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee84094e40 con 0x55b58a320c00 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8409be90 con 0x55b58a320c00 2023-10-06T21:33:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7feea3b35b80 10 client.17229 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.120+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b58a088c00 con 0x55b58a320c00 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b58a314d50 con 0x55b58a320c00 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fee8409baf0 con 0x55b58a320c00 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 20 client.17229 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a33bb20 0x55b58a33df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 4 client.17229 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b58a26ddb0 con 0x55b58a33bb20 2023-10-06T21:33:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.121+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a33bb20 0x55b58a33df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.122+0000 7fee8d79a700 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a33bb20 0x55b58a33df40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5bfe151d9c73163 server_cookie=aa7a6005e688a3a in_seq=0 out_seq=0 2023-10-06T21:33:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.122+0000 7fee717fa700 10 client.17229 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.122+0000 7fee717fa700 1 -- 172.21.15.31:0/1707697168 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b58a26ddb0 con 0x55b58a33bb20 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.122+0000 7fee717fa700 10 client.17229 handle_command_reply: tid=0 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 client.17229 shutdown 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 msgr2=0x55b58a3290d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee840cbed0 tx=0x7fee8406d650 comp rx=0 tx=0).stop 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 shutdown_connections 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a329b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a33bb20 0x55b58a33df40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a3290d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 --2- 172.21.15.31:0/1707697168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a329610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7fee717fa700 0 client.17229 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 >> 172.21.15.31:0/1707697168 conn(0x55b58a296200 msgr2=0x55b58a31cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 shutdown_connections 2023-10-06T21:33:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 1 -- 172.21.15.31:0/1707697168 wait complete. 2023-10-06T21:33:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.123+0000 7feea3b35b80 20 client.17229 trim_cache size 0 max 16384 2023-10-06T21:33:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 Processor -- start 2023-10-06T21:33:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 -- start start 2023-10-06T21:33:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a332f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a3268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a0f58c0 con 0x55b58a320c00 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a323f80 con 0x55b58a32c3d0 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.139+0000 7feea3b35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a31de40 con 0x55b58a321520 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a332f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:58462/0 (socket says 172.21.15.31:58462) 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 -- 172.21.15.31:0/605875081 learned_addr learned my addr 172.21.15.31:0/605875081 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a332f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43762/0 (socket says 172.21.15.31:43762) 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8e79c700 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a3268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 -- 172.21.15.31:0/605875081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 msgr2=0x55b58a3268a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a3268a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 -- 172.21.15.31:0/605875081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 msgr2=0x55b58a332f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a332f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.140+0000 7fee8d79a700 1 -- 172.21.15.31:0/605875081 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58a0c61f0 con 0x55b58a321520 2023-10-06T21:33:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.141+0000 7fee8d79a700 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee78005fd0 tx=0x7fee7801bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1f44d94b474e06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.141+0000 7fee7cff9700 1 -- 172.21.15.31:0/605875081 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78016040 con 0x55b58a321520 2023-10-06T21:33:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.141+0000 7fee7cff9700 1 -- 172.21.15.31:0/605875081 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee7801b5b0 con 0x55b58a321520 2023-10-06T21:33:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.141+0000 7fee7cff9700 1 -- 172.21.15.31:0/605875081 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78005020 con 0x55b58a321520 2023-10-06T21:33:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- 172.21.15.31:0/605875081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 msgr2=0x55b58a326360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee78005fd0 tx=0x7fee7801bbe0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- 172.21.15.31:0/605875081 shutdown_connections 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a32c3d0 0x55b58a3268a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a320c00 0x55b58a332f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 --2- 172.21.15.31:0/605875081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a321520 0x55b58a326360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- 172.21.15.31:0/605875081 >> 172.21.15.31:0/605875081 conn(0x55b58a296200 msgr2=0x55b58a3243d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- 172.21.15.31:0/605875081 shutdown_connections 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- 172.21.15.31:0/605875081 wait complete. 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 -- start start 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.142+0000 7feea3b35b80 1 Processor -- start 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 0x55b58a387f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a32c3d0 0x55b58a383900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a384010 con 0x55b58a321520 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a384180 con 0x55b58a320c00 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58a384340 con 0x55b58a32c3d0 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7feea3b35b80 10 client.0 _ll_register_callbacks cb 0x55b58a06e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7fee8d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7fee8d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43766/0 (socket says 172.21.15.31:43766) 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7fee8d79a700 1 -- 172.21.15.31:0/2149504578 learned_addr learned my addr 172.21.15.31:0/2149504578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7fee8df9b700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 0x55b58a387f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.143+0000 7fee8e79c700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a32c3d0 0x55b58a383900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 -- 172.21.15.31:0/2149504578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a32c3d0 msgr2=0x55b58a383900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a32c3d0 0x55b58a383900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 -- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 msgr2=0x55b58a387f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 0x55b58a387f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 -- 172.21.15.31:0/2149504578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee78011040 con 0x55b58a321520 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8df9b700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 0x55b58a387f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee8d79a700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee7801f000 tx=0x7fee78004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22f6fdfc8125f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.144+0000 7fee717fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78016040 con 0x55b58a321520 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee78004250 con 0x55b58a321520 2023-10-06T21:33:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b58a26dad0 con 0x55b58a321520 2023-10-06T21:33:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee78004410 con 0x55b58a321520 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 10 client.17235 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b58a09e250 con 0x55b58a321520 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b58a26dad0 con 0x55b58a321520 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fee78011980 con 0x55b58a321520 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 20 client.17235 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a390f80 0x55b58a3933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 4 client.17235 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.145+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b58a0c0f10 con 0x55b58a390f80 2023-10-06T21:33:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.146+0000 7fee8df9b700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a390f80 0x55b58a3933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.146+0000 7fee8df9b700 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a390f80 0x55b58a3933a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b98f405df1bf01 server_cookie=be686966f820fd46 in_seq=0 out_seq=0 2023-10-06T21:33:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.146+0000 7fee717fa700 10 client.17235 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7fee717fa700 1 -- 172.21.15.31:0/2149504578 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b58a0c0f10 con 0x55b58a390f80 2023-10-06T21:33:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7fee717fa700 10 client.17235 handle_command_reply: tid=0 2023-10-06T21:33:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 client.17235 shutdown 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 msgr2=0x55b58a3884b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee7801f000 tx=0x7fee78004840 comp rx=0 tx=0).stop 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 shutdown_connections 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b58a32c3d0 0x55b58a383900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b58a320c00 0x55b58a387f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b58a390f80 0x55b58a3933a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 --2- 172.21.15.31:0/2149504578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b58a321520 0x55b58a3884b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 >> 172.21.15.31:0/2149504578 conn(0x55b58a296200 msgr2=0x55b58a322a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7fee717fa700 0 client.17235 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 shutdown_connections 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.173+0000 7feea3b35b80 1 -- 172.21.15.31:0/2149504578 wait complete. 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.174+0000 7feea3b35b80 20 client.17235 trim_cache size 0 max 16384 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:33:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.174+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 msgr2=0x7fee740a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.174+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 0x7fee740a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee80020000 tx=0x7fee80021000 comp rx=0 tx=0).stop 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.174+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 msgr2=0x7fee881990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.174+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84009570 comp rx=0 tx=0).stop 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 shutdown_connections 2023-10-06T21:33:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8819dc70 0x7fee8819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee740a15f0 0x7fee740a3ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee88103320 0x7fee881990c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 --2- 172.21.15.31:0/1672659468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee88108440 0x7fee88199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 >> 172.21.15.31:0/1672659468 conn(0x7fee880feb60 msgr2=0x7fee88106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 shutdown_connections 2023-10-06T21:33:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:34.175+0000 7fee901ff700 1 -- 172.21.15.31:0/1672659468 wait complete. 2023-10-06T21:33:34.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 193000 1.46 MB/sec execute 564 sec latency 24.258 ms 2023-10-06T21:33:34.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:34 smithi142 ceph-mon[102951]: pgmap v396: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 71 op/s 2023-10-06T21:33:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:34 smithi031 ceph-mon[98539]: pgmap v396: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 71 op/s 2023-10-06T21:33:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:34 smithi146 ceph-mon[102809]: pgmap v396: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 71 op/s 2023-10-06T21:33:35.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 193322 1.46 MB/sec execute 565 sec latency 29.235 ms 2023-10-06T21:33:36.368 INFO:tasks.workunit.client.0.smithi031.stdout: 1 193690 1.45 MB/sec execute 566 sec latency 29.581 ms 2023-10-06T21:33:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:36 smithi031 ceph-mon[98539]: pgmap v397: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:36 smithi146 ceph-mon[102809]: pgmap v397: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:36 smithi142 ceph-mon[102951]: pgmap v397: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:37.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 194015 1.45 MB/sec execute 567 sec latency 19.176 ms 2023-10-06T21:33:38.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 194511 1.46 MB/sec execute 568 sec latency 19.869 ms 2023-10-06T21:33:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:38 smithi031 ceph-mon[98539]: pgmap v398: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:38 smithi146 ceph-mon[102809]: pgmap v398: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:38.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:38 smithi142 ceph-mon[102951]: pgmap v398: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 341 KiB/s wr, 43 op/s 2023-10-06T21:33:39.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 194866 1.46 MB/sec execute 569 sec latency 18.275 ms 2023-10-06T21:33:40.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:40 smithi146 ceph-mon[102809]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 759 KiB/s wr, 88 op/s 2023-10-06T21:33:40.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195165 1.46 MB/sec execute 570 sec latency 134.837 ms 2023-10-06T21:33:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:40 smithi142 ceph-mon[102951]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 759 KiB/s wr, 88 op/s 2023-10-06T21:33:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:40 smithi031 ceph-mon[98539]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 759 KiB/s wr, 88 op/s 2023-10-06T21:33:41.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195165 1.46 MB/sec execute 571 sec latency 1134.920 ms 2023-10-06T21:33:42.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195165 1.46 MB/sec execute 572 sec latency 2134.973 ms 2023-10-06T21:33:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:42 smithi142 ceph-mon[102951]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 732 KiB/s wr, 80 op/s 2023-10-06T21:33:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:42 smithi031 ceph-mon[98539]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 732 KiB/s wr, 80 op/s 2023-10-06T21:33:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:42 smithi146 ceph-mon[102809]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 732 KiB/s wr, 80 op/s 2023-10-06T21:33:43.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195165 1.46 MB/sec execute 573 sec latency 3135.108 ms 2023-10-06T21:33:44.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195165 1.45 MB/sec execute 574 sec latency 4135.244 ms 2023-10-06T21:33:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:44 smithi031 ceph-mon[98539]: pgmap v401: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1019 KiB/s wr, 102 op/s 2023-10-06T21:33:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:44 smithi146 ceph-mon[102809]: pgmap v401: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1019 KiB/s wr, 102 op/s 2023-10-06T21:33:44.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:44 smithi142 ceph-mon[102951]: pgmap v401: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1019 KiB/s wr, 102 op/s 2023-10-06T21:33:45.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195232 1.45 MB/sec execute 575 sec latency 4226.308 ms 2023-10-06T21:33:46.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195232 1.45 MB/sec execute 576 sec latency 1530.759 ms 2023-10-06T21:33:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:46 smithi031 ceph-mon[98539]: pgmap v402: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:46 smithi146 ceph-mon[102809]: pgmap v402: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:46.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:46 smithi142 ceph-mon[102951]: pgmap v402: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 -- 172.21.15.31:0/726768236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e40732e0 msgr2=0x7ff5e40736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/726768236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e40732e0 0x7ff5e40736c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009990 tx=0x7ff5d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 -- 172.21.15.31:0/726768236 shutdown_connections 2023-10-06T21:33:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/726768236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e410d6a0 0x7ff5e410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/726768236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e40732e0 0x7ff5e40736c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/726768236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 0x7ff5e410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 -- 172.21.15.31:0/726768236 >> 172.21.15.31:0/726768236 conn(0x7ff5e40fc950 msgr2=0x7ff5e40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 -- 172.21.15.31:0/726768236 shutdown_connections 2023-10-06T21:33:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.340+0000 7ff5e95ff700 1 -- 172.21.15.31:0/726768236 wait complete. 2023-10-06T21:33:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 Processor -- start 2023-10-06T21:33:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 -- start start 2023-10-06T21:33:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 0x7ff5e4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 0x7ff5e419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419e300 con 0x7ff5e410d6a0 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419e470 con 0x7ff5e40732e0 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.341+0000 7ff5e95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419e630 con 0x7ff5e4073c00 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 0x7ff5e419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41748/0 (socket says 172.21.15.31:41748) 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3463215329 learned_addr learned my addr 172.21.15.31:0/3463215329 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 0x7ff5e4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3463215329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 msgr2=0x7ff5e4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 0x7ff5e4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 msgr2=0x7ff5e419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 0x7ff5e419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3463215329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40095f0 con 0x7ff5e40732e0 2023-10-06T21:33:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 0x7ff5e419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.342+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005740 tx=0x7ff5d4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.343+0000 7ff5cbfff700 1 -- 172.21.15.31:0/3463215329 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d401d070 con 0x7ff5e40732e0 2023-10-06T21:33:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.343+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e419e8b0 con 0x7ff5e40732e0 2023-10-06T21:33:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.343+0000 7ff5cbfff700 1 -- 172.21.15.31:0/3463215329 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5d4022470 con 0x7ff5e40732e0 2023-10-06T21:33:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.343+0000 7ff5cbfff700 1 -- 172.21.15.31:0/3463215329 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4017650 con 0x7ff5e40732e0 2023-10-06T21:33:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.343+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41a6150 con 0x7ff5e40732e0 2023-10-06T21:33:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.346+0000 7ff5cbfff700 1 -- 172.21.15.31:0/3463215329 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff5d400fb00 con 0x7ff5e40732e0 2023-10-06T21:33:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.347+0000 7ff5fcf19b80 1 Processor -- start 2023-10-06T21:33:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 -- start start 2023-10-06T21:33:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5cbfff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 0x7ff5cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84cac400 con 0x557e84d43520 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84c9dbc0 con 0x557e84d42c00 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84c9b4e0 con 0x557e84d4e3d0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5cbfff700 1 -- 172.21.15.31:0/3463215329 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff5d40ce700 con 0x7ff5e40732e0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 0x7ff5cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.348+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41758/0 (socket says 172.21.15.31:41758) 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e37fe700 1 -- 172.21.15.31:0/3648318116 learned_addr learned my addr 172.21.15.31:0/3648318116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3648318116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 msgr2=0x557e84d507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3648318116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 msgr2=0x557e84d42fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/3648318116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e84ac1d80 con 0x557e84d43520 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4de90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cbd80 tx=0x7ff5d4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8569d8ffbadfc15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 0x7ff5cc0a3ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5e419a930 tx=0x7ff5d8009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5c97fa700 1 -- 172.21.15.31:0/3648318116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d409b1e0 con 0x557e84d43520 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5c97fa700 1 -- 172.21.15.31:0/3648318116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5d409b000 con 0x557e84d43520 2023-10-06T21:33:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.349+0000 7ff5c97fa700 1 -- 172.21.15.31:0/3648318116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d409b6a0 con 0x557e84d43520 2023-10-06T21:33:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3648318116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 msgr2=0x557e84d4de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4de90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cbd80 tx=0x7ff5d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3648318116 shutdown_connections 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d42fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3648318116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4de90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3648318116 >> 172.21.15.31:0/3648318116 conn(0x557e84cb8200 msgr2=0x557e84d3f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3648318116 shutdown_connections 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.350+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3648318116 wait complete. 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.351+0000 7ff5fcf19b80 1 -- start start 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.351+0000 7ff5fcf19b80 1 Processor -- start 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84c96dd0 con 0x557e84d43520 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84c336b0 con 0x557e84d42c00 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84c33350 con 0x557e84d4e3d0 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5fcf19b80 10 client.0 _ll_register_callbacks cb 0x557e84a907a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50556/0 (socket says 172.21.15.31:50556) 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 -- 172.21.15.31:0/3146737933 learned_addr learned my addr 172.21.15.31:0/3146737933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 -- 172.21.15.31:0/3146737933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 msgr2=0x557e84d59830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d59830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 -- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 msgr2=0x557e84d4e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e27fc700 1 -- 172.21.15.31:0/3146737933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc011040 con 0x557e84d43520 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.352+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800e7b0 tx=0x7ff5d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e38dfebd71f3b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8004220 con 0x557e84d43520 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e84d36d50 con 0x557e84d43520 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff5d8023070 con 0x557e84d43520 2023-10-06T21:33:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d8004380 con 0x557e84d43520 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5fcf19b80 10 client.17247 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7ff5e41a6150 con 0x557e84d43520 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.353+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5d8033080 con 0x557e84d43520 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff5d801c7f0 con 0x557e84d43520 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 10 client.17247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 20 client.17247 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84d5dca0 0x557e84d600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 4 client.17247 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e84c8fdb0 con 0x557e84d5dca0 2023-10-06T21:33:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.354+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84d5dca0 0x557e84d600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.355+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84d5dca0 0x557e84d600c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8e17e20d4f305 server_cookie=58b1c9271d54bd7f in_seq=0 out_seq=0 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.355+0000 7ff5c6ff5700 10 client.17247 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/3146737933 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e84c8fdb0 con 0x557e84d5dca0 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5c6ff5700 10 client.17247 handle_command_reply: tid=0 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 client.17247 shutdown 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 msgr2=0x557e84d4ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800e7b0 tx=0x7ff5d80047c0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 shutdown_connections 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d4e3d0 0x557e84d59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84d5dca0 0x557e84d600c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/3146737933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d43520 0x557e84d4ecf0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5c6ff5700 0 client.17247 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 >> 172.21.15.31:0/3146737933 conn(0x557e84cb8200 msgr2=0x557e84d3ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 shutdown_connections 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/3146737933 wait complete. 2023-10-06T21:33:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.356+0000 7ff5fcf19b80 20 client.17247 trim_cache size 0 max 16384 2023-10-06T21:33:47.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195232 1.45 MB/sec execute 577 sec latency 2530.804 ms 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 Processor -- start 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 -- start start 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d58f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84b178c0 con 0x557e84d4e3d0 2023-10-06T21:33:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84d4f040 con 0x557e84d42c00 2023-10-06T21:33:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84d5b230 con 0x557e84d43520 2023-10-06T21:33:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50562/0 (socket says 172.21.15.31:50562) 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.372+0000 7ff5e37fe700 1 -- 172.21.15.31:0/2992986721 learned_addr learned my addr 172.21.15.31:0/2992986721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d58f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 -- 172.21.15.31:0/2992986721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 msgr2=0x557e84d58f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d58f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 -- 172.21.15.31:0/2992986721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 msgr2=0x557e84d4cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 -- 172.21.15.31:0/2992986721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e84d4e120 con 0x557e84d4e3d0 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc007de0 tx=0x7ff5dc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb169e3727b7c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5c9ffb700 1 -- 172.21.15.31:0/2992986721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc003d90 con 0x557e84d4e3d0 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5c9ffb700 1 -- 172.21.15.31:0/2992986721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5dc00f040 con 0x557e84d4e3d0 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.373+0000 7ff5c9ffb700 1 -- 172.21.15.31:0/2992986721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01baf0 con 0x557e84d4e3d0 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2992986721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 msgr2=0x557e84d59490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc007de0 tx=0x7ff5dc004b40 comp rx=0 tx=0).stop 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2992986721 shutdown_connections 2023-10-06T21:33:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d4e3d0 0x557e84d59490 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d42c00 0x557e84d4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2992986721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2992986721 >> 172.21.15.31:0/2992986721 conn(0x557e84cb8200 msgr2=0x557e84d488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2992986721 shutdown_connections 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.374+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2992986721 wait complete. 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 -- start start 2023-10-06T21:33:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 Processor -- start 2023-10-06T21:33:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d4ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 0x557e84da9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84daa050 con 0x557e84d42c00 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84daa1c0 con 0x557e84d4e3d0 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e84daa380 con 0x557e84d43520 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5fcf19b80 10 client.0 _ll_register_callbacks cb 0x7ff5d800e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.375+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50566/0 (socket says 172.21.15.31:50566) 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/2390211768 learned_addr learned my addr 172.21.15.31:0/2390211768 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:33:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d4ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/2390211768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 msgr2=0x557e84d4ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d4ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 0x557e84da9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 msgr2=0x557e84da9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 0x557e84da9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 -- 172.21.15.31:0/2390211768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc007ba0 con 0x557e84d42c00 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e37fe700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 0x557e84da9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5e2ffd700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cbde0 tx=0x7ff5d400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a155ccb58a90bfa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d409bda0 con 0x557e84d42c00 2023-10-06T21:33:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.376+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff5d409f7f0 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5d409b000 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d40a39a0 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 10 client.17253 fetch_fsmap learned FSMap version 40 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557e84aaac00 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5d409b000 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff5d40b9410 con 0x557e84d42c00 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 20 client.17253 populate_metadata read hostname 'smithi031' 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84db2930 0x557e84db4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 4 client.17253 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:33:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.377+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e84c8fad0 con 0x557e84db2930 2023-10-06T21:33:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.378+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84db2930 0x557e84db4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.378+0000 7ff5e27fc700 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84db2930 0x557e84db4d50 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628cfd32a2553f6 server_cookie=418a82fdd312df9a in_seq=0 out_seq=0 2023-10-06T21:33:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.378+0000 7ff5c6ff5700 10 client.17253 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.407+0000 7ff5c6ff5700 1 -- 172.21.15.31:0/2390211768 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e84c8fad0 con 0x557e84db2930 2023-10-06T21:33:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.407+0000 7ff5c6ff5700 10 client.17253 handle_command_reply: tid=0 2023-10-06T21:33:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.407+0000 7ff5fcf19b80 1 client.17253 shutdown 2023-10-06T21:33:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 msgr2=0x557e84d4fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cbde0 tx=0x7ff5d400b390 comp rx=0 tx=0).stop 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 shutdown_connections 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e84d4e3d0 0x557e84da9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557e84db2930 0x557e84db4d50 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e84d42c00 0x557e84d4fa50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 --2- 172.21.15.31:0/2390211768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e84d43520 0x557e84d4ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5c6ff5700 0 client.17253 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 >> 172.21.15.31:0/2390211768 conn(0x557e84cb8200 msgr2=0x557e84d44bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 shutdown_connections 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 1 -- 172.21.15.31:0/2390211768 wait complete. 2023-10-06T21:33:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.408+0000 7ff5fcf19b80 20 client.17253 trim_cache size 0 max 16384 2023-10-06T21:33:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:33:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:33:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:33:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 msgr2=0x7ff5cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 0x7ff5cc0a3ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5e419a930 tx=0x7ff5d8009260 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 msgr2=0x7ff5e4199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005740 tx=0x7ff5d4005770 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 shutdown_connections 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5cc0a15f0 0x7ff5cc0a3ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e410d6a0 0x7ff5e419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5e40732e0 0x7ff5e4199110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 --2- 172.21.15.31:0/3463215329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e4073c00 0x7ff5e4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 >> 172.21.15.31:0/3463215329 conn(0x7ff5e40fc950 msgr2=0x7ff5e410f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 shutdown_connections 2023-10-06T21:33:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:33:47.409+0000 7ff5e95ff700 1 -- 172.21.15.31:0/3463215329 wait complete. 2023-10-06T21:33:48.369 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195232 1.44 MB/sec execute 578 sec latency 3530.894 ms 2023-10-06T21:33:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:48 smithi031 ceph-mon[98539]: pgmap v403: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:48 smithi146 ceph-mon[102809]: pgmap v403: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:48.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:48 smithi142 ceph-mon[102951]: pgmap v403: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 705 KiB/s wr, 66 op/s 2023-10-06T21:33:49.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195232 1.44 MB/sec execute 579 sec latency 4530.983 ms 2023-10-06T21:33:50.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:50 smithi146 ceph-mon[102809]: pgmap v404: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 761 KiB/s wr, 80 op/s 2023-10-06T21:33:50.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195407 1.44 MB/sec execute 580 sec latency 4623.757 ms 2023-10-06T21:33:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:50 smithi142 ceph-mon[102951]: pgmap v404: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 761 KiB/s wr, 80 op/s 2023-10-06T21:33:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:50 smithi031 ceph-mon[98539]: pgmap v404: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 761 KiB/s wr, 80 op/s 2023-10-06T21:33:51.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195576 1.44 MB/sec execute 581 sec latency 18.658 ms 2023-10-06T21:33:52.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 195848 1.44 MB/sec execute 582 sec latency 23.128 ms 2023-10-06T21:33:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:52 smithi031 ceph-mon[98539]: pgmap v405: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 343 KiB/s wr, 35 op/s 2023-10-06T21:33:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:52 smithi146 ceph-mon[102809]: pgmap v405: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 343 KiB/s wr, 35 op/s 2023-10-06T21:33:52.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:52 smithi142 ceph-mon[102951]: pgmap v405: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 343 KiB/s wr, 35 op/s 2023-10-06T21:33:53.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 196189 1.44 MB/sec execute 583 sec latency 27.493 ms 2023-10-06T21:33:54.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 196549 1.44 MB/sec execute 584 sec latency 20.944 ms 2023-10-06T21:33:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:54 smithi031 ceph-mon[98539]: pgmap v406: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 70 op/s 2023-10-06T21:33:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:54 smithi146 ceph-mon[102809]: pgmap v406: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 70 op/s 2023-10-06T21:33:54.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:54 smithi142 ceph-mon[102951]: pgmap v406: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 657 KiB/s wr, 70 op/s 2023-10-06T21:33:55.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 196872 1.44 MB/sec execute 585 sec latency 26.762 ms 2023-10-06T21:33:56.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 197253 1.44 MB/sec execute 586 sec latency 28.139 ms 2023-10-06T21:33:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:56 smithi031 ceph-mon[98539]: pgmap v407: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:56 smithi146 ceph-mon[102809]: pgmap v407: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:56 smithi142 ceph-mon[102951]: pgmap v407: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:57.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 197570 1.44 MB/sec execute 587 sec latency 17.964 ms 2023-10-06T21:33:58.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198063 1.44 MB/sec execute 588 sec latency 18.683 ms 2023-10-06T21:33:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:33:58 smithi031 ceph-mon[98539]: pgmap v408: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:33:58 smithi146 ceph-mon[102809]: pgmap v408: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:33:58 smithi142 ceph-mon[102951]: pgmap v408: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 370 KiB/s wr, 48 op/s 2023-10-06T21:33:59.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198411 1.44 MB/sec execute 589 sec latency 21.786 ms 2023-10-06T21:34:00.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:00 smithi146 ceph-mon[102809]: pgmap v409: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 774 KiB/s wr, 88 op/s 2023-10-06T21:34:00.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198710 1.45 MB/sec execute 590 sec latency 107.975 ms 2023-10-06T21:34:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:00 smithi142 ceph-mon[102951]: pgmap v409: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 774 KiB/s wr, 88 op/s 2023-10-06T21:34:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:00 smithi031 ceph-mon[98539]: pgmap v409: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 774 KiB/s wr, 88 op/s 2023-10-06T21:34:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 -- 172.21.15.31:0/2645283710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 msgr2=0x7f759c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 --2- 172.21.15.31:0/2645283710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c10dd70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f759400b0a0 tx=0x7f759400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 -- 172.21.15.31:0/2645283710 shutdown_connections 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 --2- 172.21.15.31:0/2645283710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c10dd70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 --2- 172.21.15.31:0/2645283710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c1010d0 0x7f759c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 --2- 172.21.15.31:0/2645283710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c101a80 0x7f759c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 -- 172.21.15.31:0/2645283710 >> 172.21.15.31:0/2645283710 conn(0x7f759c0fc930 msgr2=0x7f759c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 -- 172.21.15.31:0/2645283710 shutdown_connections 2023-10-06T21:34:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.560+0000 7f75a09ff700 1 -- 172.21.15.31:0/2645283710 wait complete. 2023-10-06T21:34:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.561+0000 7f75a09ff700 1 Processor -- start 2023-10-06T21:34:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.561+0000 7f75a09ff700 1 -- start start 2023-10-06T21:34:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c1010d0 0x7f759c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 0x7f759c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19e830 con 0x7f759c1061f0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19e9a0 con 0x7f759c1010d0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f75a09ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19eb60 con 0x7f759c19ddb0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57184/0 (socket says 172.21.15.31:57184) 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f7599d9b700 1 -- 172.21.15.31:0/4057396497 learned_addr learned my addr 172.21.15.31:0/4057396497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f759a59c700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c1010d0 0x7f759c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f759ad9d700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 0x7f759c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f7599d9b700 1 -- 172.21.15.31:0/4057396497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 msgr2=0x7f759c19e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.562+0000 7f7599d9b700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 0x7f759c19e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f7599d9b700 1 -- 172.21.15.31:0/4057396497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c1010d0 msgr2=0x7f759c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f7599d9b700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c1010d0 0x7f759c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f7599d9b700 1 -- 172.21.15.31:0/4057396497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590009520 con 0x7f759c1061f0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f759ad9d700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 0x7f759c19e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f7599d9b700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea30 tx=0x7f759000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f758b7fe700 1 -- 172.21.15.31:0/4057396497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590009820 con 0x7f759c1061f0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7594009d00 con 0x7f759c1061f0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f758b7fe700 1 -- 172.21.15.31:0/4057396497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75900044b0 con 0x7f759c1061f0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f758b7fe700 1 -- 172.21.15.31:0/4057396497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590003ce0 con 0x7f759c1061f0 2023-10-06T21:34:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.563+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c1a6460 con 0x7f759c1061f0 2023-10-06T21:34:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.568+0000 7f758b7fe700 1 -- 172.21.15.31:0/4057396497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7590003ea0 con 0x7f759c1061f0 2023-10-06T21:34:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.569+0000 7f75b4266b80 1 Processor -- start 2023-10-06T21:34:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.569+0000 7f75b4266b80 1 -- start start 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.569+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 0x561f67167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.569+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.569+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f671733d0 0x561f671757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f75b4266b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f670d1400 con 0x561f67168520 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f75b4266b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f670c2bc0 con 0x561f67167c00 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f75b4266b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f670c04e0 con 0x561f671733d0 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f758b7fe700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 0x7f75840a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57188/0 (socket says 172.21.15.31:57188) 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 -- 172.21.15.31:0/2400603947 learned_addr learned my addr 172.21.15.31:0/2400603947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759ad9d700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 0x561f67167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f758b7fe700 1 -- 172.21.15.31:0/4057396497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f7590014070 con 0x7f759c1061f0 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f7599d9b700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f671733d0 0x561f671757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759ad9d700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 0x7f75840a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 -- 172.21.15.31:0/2400603947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f671733d0 msgr2=0x561f671757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f671733d0 0x561f671757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 -- 172.21.15.31:0/2400603947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 msgr2=0x561f67167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 0x561f67167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759a59c700 1 -- 172.21.15.31:0/2400603947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f66ee6d80 con 0x561f67168520 2023-10-06T21:34:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.570+0000 7f759ad9d700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 0x561f67167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:34:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.571+0000 7f759a59c700 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f758c009db0 tx=0x7f758c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e24112acb37b0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.571+0000 7f759ad9d700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 0x7f75840a3b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f759400b780 tx=0x7f75940060c0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.571+0000 7f7588ff9700 1 -- 172.21.15.31:0/2400603947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c019630 con 0x561f67168520 2023-10-06T21:34:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.571+0000 7f7588ff9700 1 -- 172.21.15.31:0/2400603947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758c019c20 con 0x561f67168520 2023-10-06T21:34:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.571+0000 7f7588ff9700 1 -- 172.21.15.31:0/2400603947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00eb00 con 0x561f67168520 2023-10-06T21:34:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- 172.21.15.31:0/2400603947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 msgr2=0x561f67172e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f758c009db0 tx=0x7f758c011720 comp rx=0 tx=0).stop 2023-10-06T21:34:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- 172.21.15.31:0/2400603947 shutdown_connections 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f671733d0 0x561f671757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67167c00 0x561f67167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 --2- 172.21.15.31:0/2400603947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67172e90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- 172.21.15.31:0/2400603947 >> 172.21.15.31:0/2400603947 conn(0x561f670dd200 msgr2=0x561f67164080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- 172.21.15.31:0/2400603947 shutdown_connections 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- 172.21.15.31:0/2400603947 wait complete. 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.572+0000 7f75b4266b80 1 -- start start 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 Processor -- start 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67172e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f671707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f670bbdd0 con 0x561f67168520 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f670586b0 con 0x561f671733d0 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f67058350 con 0x561f67167c00 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f75b4266b80 10 client.0 _ll_register_callbacks cb 0x561f66eb57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.573+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57198/0 (socket says 172.21.15.31:57198) 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 -- 172.21.15.31:0/3320796164 learned_addr learned my addr 172.21.15.31:0/3320796164 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f759ad9d700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f671707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f759a59c700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67172e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 -- 172.21.15.31:0/3320796164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 msgr2=0x561f67172e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67172e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 -- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 msgr2=0x561f671707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f671707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 -- 172.21.15.31:0/3320796164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0099e0 con 0x561f67168520 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f759a59c700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67172e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f759009a260 tx=0x7f7590000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d67efad4fb8af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7581ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759001fde0 con 0x561f67168520 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f759009e160 con 0x561f67168520 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.574+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f6715bd50 con 0x561f67168520 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75900cad10 con 0x561f67168520 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 10 client.17265 fetch_fsmap learned FSMap version 40 2023-10-06T21:34:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561f66ecfc00 con 0x561f67168520 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75900b8020 con 0x561f67168520 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f759009a610 con 0x561f67168520 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 20 client.17265 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f67182fb0 0x561f671853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 4 client.17265 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.575+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f670b4db0 con 0x561f67182fb0 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.576+0000 7f759a59c700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f67182fb0 0x561f671853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.576+0000 7f759a59c700 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f67182fb0 0x561f671853d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82dcbabbc75e3e79 server_cookie=44972359aac85153 in_seq=0 out_seq=0 2023-10-06T21:34:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.576+0000 7f7581ffb700 10 client.17265 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.577+0000 7f7581ffb700 1 -- 172.21.15.31:0/3320796164 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f670b4db0 con 0x561f67182fb0 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.577+0000 7f7581ffb700 10 client.17265 handle_command_reply: tid=0 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.577+0000 7f75b4266b80 1 client.17265 shutdown 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.577+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 msgr2=0x561f67170080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.577+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f759009a260 tx=0x7f7590000bc0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 shutdown_connections 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f671707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67172e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f67182fb0 0x561f671853d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3320796164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f67170080 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 >> 172.21.15.31:0/3320796164 conn(0x561f670dd200 msgr2=0x561f67163c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f7581ffb700 0 client.17265 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 shutdown_connections 2023-10-06T21:34:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 1 -- 172.21.15.31:0/3320796164 wait complete. 2023-10-06T21:34:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.578+0000 7f75b4266b80 20 client.17265 trim_cache size 0 max 16384 2023-10-06T21:34:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 Processor -- start 2023-10-06T21:34:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 -- start start 2023-10-06T21:34:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67179f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f6716b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f66f3c8c0 con 0x561f67168520 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f67164e30 con 0x561f671733d0 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.593+0000 7f75b4266b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6717a4b0 con 0x561f67167c00 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57202/0 (socket says 172.21.15.31:57202) 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 -- 172.21.15.31:0/3003057942 learned_addr learned my addr 172.21.15.31:0/3003057942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f759ad9d700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f6716b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f759a59c700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67179f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 -- 172.21.15.31:0/3003057942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 msgr2=0x561f67179f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67179f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 -- 172.21.15.31:0/3003057942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 msgr2=0x561f6716b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f6716b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 -- 172.21.15.31:0/3003057942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f66f0d1f0 con 0x561f67168520 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f759ad9d700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f6716b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f7599d9b700 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7590096820 tx=0x7f7590099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8c8d11e0d4827 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f75897fa700 1 -- 172.21.15.31:0/3003057942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590096660 con 0x561f67168520 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f75897fa700 1 -- 172.21.15.31:0/3003057942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75900b8040 con 0x561f67168520 2023-10-06T21:34:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.594+0000 7f75897fa700 1 -- 172.21.15.31:0/3003057942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900cad10 con 0x561f67168520 2023-10-06T21:34:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 -- 172.21.15.31:0/3003057942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 msgr2=0x561f6716b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7590096820 tx=0x7f7590099f90 comp rx=0 tx=0).stop 2023-10-06T21:34:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 -- 172.21.15.31:0/3003057942 shutdown_connections 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f671733d0 0x561f6716b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67179f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 --2- 172.21.15.31:0/3003057942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f67168520 0x561f6716b390 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 -- 172.21.15.31:0/3003057942 >> 172.21.15.31:0/3003057942 conn(0x561f670dd200 msgr2=0x561f6717f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 -- 172.21.15.31:0/3003057942 shutdown_connections 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.595+0000 7f75b4266b80 1 -- 172.21.15.31:0/3003057942 wait complete. 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.596+0000 7f75b4266b80 1 -- start start 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.596+0000 7f75b4266b80 1 Processor -- start 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67177a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f671733d0 0x561f671d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f67178720 con 0x561f671733d0 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f671d6320 con 0x561f67168520 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f671d64e0 con 0x561f67167c00 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f75b4266b80 10 client.0 _ll_register_callbacks cb 0x561f66eb57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f759ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f671733d0 0x561f671d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54948/0 (socket says 172.21.15.31:54948) 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 -- 172.21.15.31:0/599599317 learned_addr learned my addr 172.21.15.31:0/599599317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f759a59c700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67177a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 -- 172.21.15.31:0/599599317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 msgr2=0x561f67177a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67177a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 -- 172.21.15.31:0/599599317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f671733d0 msgr2=0x561f671d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f671733d0 0x561f671d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.597+0000 7f7599d9b700 1 -- 172.21.15.31:0/599599317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f66ecfc00 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7599d9b700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f759009a260 tx=0x7f759009e370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131ee14a5436944d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7581ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759009e160 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f7590099920 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f758c019c20 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7590099ae0 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f75b4266b80 10 client.26951 fetch_fsmap learned FSMap version 40 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561f66ee5250 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.598+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758c019c20 con 0x561f67168520 2023-10-06T21:34:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f759009a580 con 0x561f67168520 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 10 client.26951 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 10 client.26951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 20 client.26951 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f671d7740 0x561f671d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 4 client.26951 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f670b4ad0 con 0x561f671d7740 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f759a59c700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f671d7740 0x561f671d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f759a59c700 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f671d7740 0x561f671d9b30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfcf43cc81b980f server_cookie=ba1ab9c2f8cb4774 in_seq=0 out_seq=0 2023-10-06T21:34:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.599+0000 7f7581ffb700 10 client.26951 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f7581ffb700 1 -- 172.21.15.31:0/599599317 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f670b4ad0 con 0x561f671d7740 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f7581ffb700 10 client.26951 handle_command_reply: tid=0 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 client.26951 shutdown 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 msgr2=0x561f67177fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f759009a260 tx=0x7f759009e370 comp rx=0 tx=0).stop 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 shutdown_connections 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561f671d7740 0x561f671d9b30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561f671733d0 0x561f671d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f67167c00 0x561f67177a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 --2- 172.21.15.31:0/599599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f67168520 0x561f67177fb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f7581ffb700 0 client.26951 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 >> 172.21.15.31:0/599599317 conn(0x561f670dd200 msgr2=0x561f6716b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 shutdown_connections 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 1 -- 172.21.15.31:0/599599317 wait complete. 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.631+0000 7f75b4266b80 20 client.26951 trim_cache size 0 max 16384 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:34:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:34:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 msgr2=0x7f75840a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 0x7f75840a3b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f759400b780 tx=0x7f75940060c0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 msgr2=0x7f759c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea30 tx=0x7f759000ed40 comp rx=0 tx=0).stop 2023-10-06T21:34:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 shutdown_connections 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f759c19ddb0 0x7f759c19e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f759c1010d0 0x7f759c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f75840a1670 0x7f75840a3b30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 --2- 172.21.15.31:0/4057396497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f759c1061f0 0x7f759c199740 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 >> 172.21.15.31:0/4057396497 conn(0x7f759c0fc930 msgr2=0x7f759c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 shutdown_connections 2023-10-06T21:34:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:00.632+0000 7f75a09ff700 1 -- 172.21.15.31:0/4057396497 wait complete. 2023-10-06T21:34:01.370 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198710 1.44 MB/sec execute 591 sec latency 1108.061 ms 2023-10-06T21:34:02.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198710 1.44 MB/sec execute 592 sec latency 2108.213 ms 2023-10-06T21:34:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:02 smithi031 ceph-mon[98539]: pgmap v410: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 718 KiB/s wr, 75 op/s 2023-10-06T21:34:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:02 smithi146 ceph-mon[102809]: pgmap v410: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 718 KiB/s wr, 75 op/s 2023-10-06T21:34:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:02 smithi142 ceph-mon[102951]: pgmap v410: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 718 KiB/s wr, 75 op/s 2023-10-06T21:34:03.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198710 1.44 MB/sec execute 593 sec latency 3108.356 ms 2023-10-06T21:34:04.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198710 1.44 MB/sec execute 594 sec latency 4108.493 ms 2023-10-06T21:34:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:04 smithi031 ceph-mon[98539]: pgmap v411: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:34:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:04 smithi146 ceph-mon[102809]: pgmap v411: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:34:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:04 smithi142 ceph-mon[102951]: pgmap v411: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2023-10-06T21:34:05.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198777 1.44 MB/sec execute 595 sec latency 4199.460 ms 2023-10-06T21:34:06.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198777 1.43 MB/sec execute 596 sec latency 1544.669 ms 2023-10-06T21:34:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:06 smithi031 ceph-mon[98539]: pgmap v412: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:06 smithi146 ceph-mon[102809]: pgmap v412: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:06 smithi142 ceph-mon[102951]: pgmap v412: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:07.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198777 1.43 MB/sec execute 597 sec latency 2544.813 ms 2023-10-06T21:34:08.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198777 1.43 MB/sec execute 598 sec latency 3544.961 ms 2023-10-06T21:34:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:08 smithi031 ceph-mon[98539]: pgmap v413: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:08 smithi146 ceph-mon[102809]: pgmap v413: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:08 smithi142 ceph-mon[102951]: pgmap v413: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 718 KiB/s wr, 69 op/s 2023-10-06T21:34:09.372 INFO:tasks.workunit.client.0.smithi031.stdout: 1 198777 1.43 MB/sec execute 599 sec latency 4545.098 ms 2023-10-06T21:34:10.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 cleanup 600 sec 2023-10-06T21:34:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:10 smithi031 ceph-mon[98539]: pgmap v414: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 792 KiB/s wr, 91 op/s 2023-10-06T21:34:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:10 smithi031 ceph-mon[98539]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:34:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:10 smithi146 ceph-mon[102809]: pgmap v414: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 792 KiB/s wr, 91 op/s 2023-10-06T21:34:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:10 smithi146 ceph-mon[102809]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:34:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:10 smithi142 ceph-mon[102951]: pgmap v414: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 792 KiB/s wr, 91 op/s 2023-10-06T21:34:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:10 smithi142 ceph-mon[102951]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:34:11.371 INFO:tasks.workunit.client.0.smithi031.stdout: 1 cleanup 601 sec 2023-10-06T21:34:11.643 INFO:tasks.workunit.client.0.smithi031.stdout: 0 cleanup 601 sec 2023-10-06T21:34:11.643 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:34:11.643 INFO:tasks.workunit.client.0.smithi031.stdout: Operation Count AvgLat MaxLat 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: ---------------------------------------- 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: NTCreateX 27375 6.169 4256.917 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Close 20109 0.072 0.835 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Rename 1160 99.681 5001.260 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Unlink 5512 6.692 4248.540 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Qpathinfo 24848 4.635 403.541 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Qfileinfo 4344 0.158 0.920 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Qfsinfo 4549 0.464 3.495 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Sfileinfo 2216 2.142 14.416 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: Find 9604 14.919 405.823 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: WriteX 13547 0.294 4.609 2023-10-06T21:34:11.644 INFO:tasks.workunit.client.0.smithi031.stdout: ReadX 43065 0.005 0.550 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout: LockX 90 0.874 1.408 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout: UnlockX 90 0.743 1.234 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout: Flush 1911 3.210 301.564 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout:Throughput 1.42582 MB/sec 1 clients 1 procs max_latency=5001.274 ms 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stderr:+ dbench 10 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-10-06T21:34:11.645 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:34:11.646 INFO:tasks.workunit.client.0.smithi031.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-10-06T21:34:11.648 INFO:tasks.workunit.client.0.smithi031.stdout:9 of 10 processes prepared for launch 0 sec 2023-10-06T21:34:11.748 INFO:tasks.workunit.client.0.smithi031.stdout:10 of 10 processes prepared for launch 0 sec 2023-10-06T21:34:11.748 INFO:tasks.workunit.client.0.smithi031.stdout:releasing clients 2023-10-06T21:34:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:12 smithi031 ceph-mon[98539]: pgmap v416: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 466 KiB/s wr, 61 op/s 2023-10-06T21:34:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:12 smithi146 ceph-mon[102809]: pgmap v416: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 466 KiB/s wr, 61 op/s 2023-10-06T21:34:12.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:12 smithi142 ceph-mon[102951]: pgmap v416: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 466 KiB/s wr, 61 op/s 2023-10-06T21:34:12.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 203 80.10 MB/sec warmup 1 sec latency 53.313 ms 2023-10-06T21:34:13.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 371 66.33 MB/sec warmup 2 sec latency 111.814 ms 2023-10-06T21:34:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 -- 172.21.15.31:0/3255773107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c103cc0 msgr2=0x7fa01c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 --2- 172.21.15.31:0/3255773107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c103cc0 0x7fa01c107d00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa008009990 tx=0x7fa008009ca0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 -- 172.21.15.31:0/3255773107 shutdown_connections 2023-10-06T21:34:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 --2- 172.21.15.31:0/3255773107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c108430 0x7fa01c10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 --2- 172.21.15.31:0/3255773107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c103310 0x7fa01c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 --2- 172.21.15.31:0/3255773107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c103cc0 0x7fa01c107d00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 -- 172.21.15.31:0/3255773107 >> 172.21.15.31:0/3255773107 conn(0x7fa01c0feb70 msgr2=0x7fa01c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.852+0000 7fa0245ff700 1 -- 172.21.15.31:0/3255773107 shutdown_connections 2023-10-06T21:34:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.853+0000 7fa0245ff700 1 -- 172.21.15.31:0/3255773107 wait complete. 2023-10-06T21:34:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.854+0000 7fa0245ff700 1 Processor -- start 2023-10-06T21:34:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.854+0000 7fa0245ff700 1 -- start start 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c103310 0x7fa01c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c19dca0 0x7fa01c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19e720 con 0x7fa01c19dca0 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19e890 con 0x7fa01c108430 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.855+0000 7fa0245ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19ea50 con 0x7fa01c103310 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c19dca0 0x7fa01c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43150/0 (socket says 172.21.15.31:43150) 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 -- 172.21.15.31:0/2666838455 learned_addr learned my addr 172.21.15.31:0/2666838455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa02239b700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c103310 0x7fa01c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 -- 172.21.15.31:0/2666838455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c103310 msgr2=0x7fa01c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c103310 0x7fa01c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 -- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c19dca0 msgr2=0x7fa01c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c19dca0 0x7fa01c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 -- 172.21.15.31:0/2666838455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080095f0 con 0x7fa01c108430 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.856+0000 7fa021b9a700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa008004730 tx=0x7fa008004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.857+0000 7fa01b7fe700 1 -- 172.21.15.31:0/2666838455 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00801d070 con 0x7fa01c108430 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.857+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c1a5f50 con 0x7fa01c108430 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.857+0000 7fa01b7fe700 1 -- 172.21.15.31:0/2666838455 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa008022470 con 0x7fa01c108430 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.857+0000 7fa01b7fe700 1 -- 172.21.15.31:0/2666838455 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008017650 con 0x7fa01c108430 2023-10-06T21:34:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.857+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1a6370 con 0x7fa01c108430 2023-10-06T21:34:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.864+0000 7fa01b7fe700 1 -- 172.21.15.31:0/2666838455 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa0080225e0 con 0x7fa01c108430 2023-10-06T21:34:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.864+0000 7fa037e1fb80 1 Processor -- start 2023-10-06T21:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 -- start start 2023-10-06T21:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e5266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e528fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e47c400 con 0x55a60e51b410 2023-10-06T21:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e46dbc0 con 0x55a60e51bd30 2023-10-06T21:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.865+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e46b4e0 con 0x55a60e526be0 2023-10-06T21:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.866+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.866+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32866/0 (socket says 172.21.15.31:32866) 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.866+0000 7fa022b9c700 1 -- 172.21.15.31:0/2294096256 learned_addr learned my addr 172.21.15.31:0/2294096256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.866+0000 7fa02239b700 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e5266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 -- 172.21.15.31:0/2294096256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 msgr2=0x55a60e528fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e528fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 -- 172.21.15.31:0/2294096256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 msgr2=0x55a60e5266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e5266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 -- 172.21.15.31:0/2294096256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a60e291d80 con 0x55a60e51b410 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa022b9c700 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa014009cb0 tx=0x7fa014011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f209b661ef26a8bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.867+0000 7fa01b7fe700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 0x7fa00c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.868+0000 7fa01b7fe700 1 -- 172.21.15.31:0/2666838455 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fa0080ce440 con 0x7fa01c108430 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.868+0000 7fa018bfe700 1 -- 172.21.15.31:0/2294096256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014011e30 con 0x55a60e51b410 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.868+0000 7fa018bfe700 1 -- 172.21.15.31:0/2294096256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0140044c0 con 0x55a60e51b410 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.868+0000 7fa018bfe700 1 -- 172.21.15.31:0/2294096256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014019420 con 0x55a60e51b410 2023-10-06T21:34:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.868+0000 7fa02239b700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 0x7fa00c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 -- 172.21.15.31:0/2294096256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 msgr2=0x55a60e51b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa02239b700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 0x7fa00c0a3b30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa010007500 tx=0x7fa01000b040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa014009cb0 tx=0x7fa014011440 comp rx=0 tx=0).stop 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 -- 172.21.15.31:0/2294096256 shutdown_connections 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e528fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e51b7f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/2294096256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e5266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.869+0000 7fa037e1fb80 1 -- 172.21.15.31:0/2294096256 >> 172.21.15.31:0/2294096256 conn(0x55a60e488200 msgr2=0x55a60e50eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.870+0000 7fa037e1fb80 1 -- 172.21.15.31:0/2294096256 shutdown_connections 2023-10-06T21:34:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.870+0000 7fa037e1fb80 1 -- 172.21.15.31:0/2294096256 wait complete. 2023-10-06T21:34:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.871+0000 7fa037e1fb80 1 -- start start 2023-10-06T21:34:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.871+0000 7fa037e1fb80 1 Processor -- start 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e51b410 0x55a60e523a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 0x55a60e5244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e466dd0 con 0x55a60e526be0 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e4036b0 con 0x55a60e51bd30 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e403350 con 0x55a60e51b410 2023-10-06T21:34:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.872+0000 7fa037e1fb80 10 client.0 _ll_register_callbacks cb 0x55a60e2607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.873+0000 7fa021b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.873+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 0x55a60e5244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.873+0000 7fa021b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43160/0 (socket says 172.21.15.31:43160) 2023-10-06T21:34:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.873+0000 7fa021b9a700 1 -- 172.21.15.31:0/798997442 learned_addr learned my addr 172.21.15.31:0/798997442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 -- 172.21.15.31:0/798997442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e51b410 msgr2=0x55a60e523a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa02239b700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e51b410 0x55a60e523a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e51b410 0x55a60e523a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 -- 172.21.15.31:0/798997442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 msgr2=0x55a60e5244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 0x55a60e5244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 -- 172.21.15.31:0/798997442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014009960 con 0x55a60e51bd30 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa022b9c700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 0x55a60e5244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa021b9a700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0080a3180 tx=0x7fa00809afe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f758a432dba95f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.874+0000 7fa005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00809f320 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fa00809b3b0 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a60e506d50 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa00809f500 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa037e1fb80 10 client.26963 fetch_fsmap learned FSMap version 40 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.875+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a60e27ac00 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a60e506d50 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa00809a8a0 con 0x55a60e51bd30 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 10 client.26963 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 10 client.26963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 20 client.26963 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e536510 0x55a60e538930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 4 client.26963 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.876+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a60e45fdb0 con 0x55a60e536510 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.877+0000 7fa02239b700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e536510 0x55a60e538930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.877+0000 7fa02239b700 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e536510 0x55a60e538930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfafb151f0899d3a server_cookie=6f594686dee7327c in_seq=0 out_seq=0 2023-10-06T21:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.877+0000 7fa005ffb700 10 client.26963 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.880+0000 7fa005ffb700 1 -- 172.21.15.31:0/798997442 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a60e45fdb0 con 0x55a60e536510 2023-10-06T21:34:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.880+0000 7fa005ffb700 10 client.26963 handle_command_reply: tid=0 2023-10-06T21:34:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.880+0000 7fa037e1fb80 1 client.26963 shutdown 2023-10-06T21:34:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 msgr2=0x55a60e523f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0080a3180 tx=0x7fa00809afe0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 shutdown_connections 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e536510 0x55a60e538930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e526be0 0x55a60e5244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e51b410 0x55a60e523a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/798997442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e523f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa005ffb700 0 client.26963 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.881+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 >> 172.21.15.31:0/798997442 conn(0x55a60e488200 msgr2=0x55a60e50eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.882+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 shutdown_connections 2023-10-06T21:34:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.882+0000 7fa037e1fb80 1 -- 172.21.15.31:0/798997442 wait complete. 2023-10-06T21:34:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.882+0000 7fa037e1fb80 20 client.26963 trim_cache size 0 max 16384 2023-10-06T21:34:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.906+0000 7fa037e1fb80 1 Processor -- start 2023-10-06T21:34:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.906+0000 7fa037e1fb80 1 -- start start 2023-10-06T21:34:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51b410 0x55a60e52d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 0x55a60e51e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e2e78c0 con 0x55a60e51bd30 2023-10-06T21:34:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e52dcc0 con 0x55a60e51b410 2023-10-06T21:34:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.907+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e521fa0 con 0x55a60e526be0 2023-10-06T21:34:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:45434/0 (socket says 172.21.15.31:45434) 2023-10-06T21:34:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 -- 172.21.15.31:0/1172649635 learned_addr learned my addr 172.21.15.31:0/1172649635 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa02239b700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51b410 0x55a60e52d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa021b9a700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 0x55a60e51e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 -- 172.21.15.31:0/1172649635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51b410 msgr2=0x55a60e52d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51b410 0x55a60e52d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 -- 172.21.15.31:0/1172649635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 msgr2=0x55a60e51e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 0x55a60e51e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 -- 172.21.15.31:0/1172649635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a60e2b81f0 con 0x55a60e526be0 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa021b9a700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 0x55a60e51e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:34:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.908+0000 7fa022b9c700 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa01401a000 tx=0x7fa014011f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb09bf05edadece server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.909+0000 7fa0193ff700 1 -- 172.21.15.31:0/1172649635 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014003880 con 0x55a60e526be0 2023-10-06T21:34:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.909+0000 7fa0193ff700 1 -- 172.21.15.31:0/1172649635 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0140039e0 con 0x55a60e526be0 2023-10-06T21:34:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.909+0000 7fa0193ff700 1 -- 172.21.15.31:0/1172649635 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014004200 con 0x55a60e526be0 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 -- 172.21.15.31:0/1172649635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 msgr2=0x55a60e520cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa01401a000 tx=0x7fa014011f80 comp rx=0 tx=0).stop 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 -- 172.21.15.31:0/1172649635 shutdown_connections 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e520cd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51b410 0x55a60e52d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/1172649635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51bd30 0x55a60e51e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 -- 172.21.15.31:0/1172649635 >> 172.21.15.31:0/1172649635 conn(0x55a60e488200 msgr2=0x55a60e51ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.910+0000 7fa037e1fb80 1 -- 172.21.15.31:0/1172649635 shutdown_connections 2023-10-06T21:34:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.911+0000 7fa037e1fb80 1 -- 172.21.15.31:0/1172649635 wait complete. 2023-10-06T21:34:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.911+0000 7fa037e1fb80 1 -- start start 2023-10-06T21:34:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.911+0000 7fa037e1fb80 1 Processor -- start 2023-10-06T21:34:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e52b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e52bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e52c570 con 0x55a60e51b410 2023-10-06T21:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e589840 con 0x55a60e51bd30 2023-10-06T21:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60e589980 con 0x55a60e526be0 2023-10-06T21:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.912+0000 7fa037e1fb80 10 client.0 _ll_register_callbacks cb 0x7fa008097780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa021b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e52bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:45444/0 (socket says 172.21.15.31:45444) 2023-10-06T21:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 -- 172.21.15.31:0/3120710194 learned_addr learned my addr 172.21.15.31:0/3120710194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa02239b700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e52b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 -- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 msgr2=0x55a60e52bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e52bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 -- 172.21.15.31:0/3120710194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 msgr2=0x55a60e52b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e52b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.913+0000 7fa022b9c700 1 -- 172.21.15.31:0/3120710194 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a60e291d80 con 0x55a60e526be0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa022b9c700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa014011360 tx=0x7fa014011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72b5536af21b9bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014030040 con 0x55a60e526be0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0140044c0 con 0x55a60e526be0 2023-10-06T21:34:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fa0140056e0 con 0x55a60e526be0 2023-10-06T21:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa014003b50 con 0x55a60e526be0 2023-10-06T21:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa037e1fb80 10 client.26911 fetch_fsmap learned FSMap version 40 2023-10-06T21:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.914+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a60e290250 con 0x55a60e526be0 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.915+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0140044c0 con 0x55a60e526be0 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa014033030 con 0x55a60e526be0 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 10 client.26911 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 20 client.26911 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e58aba0 0x55a60e58cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 4 client.26911 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a60e45fad0 con 0x55a60e58aba0 2023-10-06T21:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.916+0000 7fa02239b700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e58aba0 0x55a60e58cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.917+0000 7fa02239b700 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e58aba0 0x55a60e58cf90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ca2ffa16ff88ca server_cookie=ee7e696da08148ec in_seq=0 out_seq=0 2023-10-06T21:34:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:13.917+0000 7fa005ffb700 10 client.26911 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.143+0000 7fa005ffb700 1 -- 172.21.15.31:0/3120710194 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a60e45fad0 con 0x55a60e58aba0 2023-10-06T21:34:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.143+0000 7fa005ffb700 10 client.26911 handle_command_reply: tid=0 2023-10-06T21:34:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.143+0000 7fa037e1fb80 1 client.26911 shutdown 2023-10-06T21:34:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 msgr2=0x55a60e589300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa014011360 tx=0x7fa014011440 comp rx=0 tx=0).stop 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 shutdown_connections 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a60e526be0 0x55a60e589300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a60e58aba0 0x55a60e58cf90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a60e51b410 0x55a60e52b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 --2- 172.21.15.31:0/3120710194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a60e51bd30 0x55a60e52bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa005ffb700 0 client.26911 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 >> 172.21.15.31:0/3120710194 conn(0x55a60e488200 msgr2=0x55a60e523960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 shutdown_connections 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 1 -- 172.21.15.31:0/3120710194 wait complete. 2023-10-06T21:34:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.144+0000 7fa037e1fb80 20 client.26911 trim_cache size 0 max 16384 2023-10-06T21:34:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:34:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:34:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:34:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 msgr2=0x7fa00c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 0x7fa00c0a3b30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa010007500 tx=0x7fa01000b040 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 msgr2=0x7fa01c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa008004730 tx=0x7fa008004760 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 shutdown_connections 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa00c0a1670 0x7fa00c0a3b30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa01c19dca0 0x7fa01c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa01c103310 0x7fa01c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 --2- 172.21.15.31:0/2666838455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c108430 0x7fa01c199630 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 >> 172.21.15.31:0/2666838455 conn(0x7fa01c0feb70 msgr2=0x7fa01c0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 shutdown_connections 2023-10-06T21:34:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:14.145+0000 7fa0245ff700 1 -- 172.21.15.31:0/2666838455 wait complete. 2023-10-06T21:34:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:14 smithi031 ceph-mon[98539]: pgmap v417: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 9.6 MiB/s wr, 101 op/s 2023-10-06T21:34:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:14 smithi031 ceph-mon[98539]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:34:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:14 smithi146 ceph-mon[102809]: pgmap v417: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 9.6 MiB/s wr, 101 op/s 2023-10-06T21:34:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:14 smithi146 ceph-mon[102809]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:34:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:14 smithi142 ceph-mon[102951]: pgmap v417: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 9.6 MiB/s wr, 101 op/s 2023-10-06T21:34:14.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:14 smithi142 ceph-mon[102951]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:34:14.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 506 58.74 MB/sec warmup 3 sec latency 177.247 ms 2023-10-06T21:34:15.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 636 51.47 MB/sec warmup 4 sec latency 324.172 ms 2023-10-06T21:34:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:16 smithi031 ceph-mon[98539]: pgmap v419: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 12 MiB/s wr, 126 op/s 2023-10-06T21:34:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:16 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:16 smithi146 ceph-mon[102809]: pgmap v419: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 12 MiB/s wr, 126 op/s 2023-10-06T21:34:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:16 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:16 smithi142 ceph-mon[102951]: pgmap v419: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 12 MiB/s wr, 126 op/s 2023-10-06T21:34:16.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:16 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:16.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 742 42.61 MB/sec warmup 5 sec latency 485.992 ms 2023-10-06T21:34:17.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 854 35.96 MB/sec warmup 6 sec latency 125.316 ms 2023-10-06T21:34:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:18 smithi031 ceph-mon[98539]: pgmap v420: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 12 MiB/s wr, 94 op/s 2023-10-06T21:34:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:18 smithi146 ceph-mon[102809]: pgmap v420: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 12 MiB/s wr, 94 op/s 2023-10-06T21:34:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:18 smithi142 ceph-mon[102951]: pgmap v420: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 12 MiB/s wr, 94 op/s 2023-10-06T21:34:18.750 INFO:tasks.workunit.client.0.smithi031.stdout: 10 935 31.09 MB/sec warmup 7 sec latency 208.239 ms 2023-10-06T21:34:19.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1044 28.48 MB/sec warmup 8 sec latency 191.521 ms 2023-10-06T21:34:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:20 smithi142 ceph-mon[102951]: pgmap v421: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 21 MiB/s wr, 320 op/s 2023-10-06T21:34:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:20 smithi031 ceph-mon[98539]: pgmap v421: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 21 MiB/s wr, 320 op/s 2023-10-06T21:34:20.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:20 smithi146 ceph-mon[102809]: pgmap v421: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 21 MiB/s wr, 320 op/s 2023-10-06T21:34:20.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1151 25.55 MB/sec warmup 9 sec latency 145.808 ms 2023-10-06T21:34:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:21 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:21 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:21 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:34:21.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1259 23.38 MB/sec warmup 10 sec latency 146.863 ms 2023-10-06T21:34:22.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:22 smithi142 ceph-mon[102951]: pgmap v422: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-10-06T21:34:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:22 smithi031 ceph-mon[98539]: pgmap v422: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-10-06T21:34:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:22 smithi146 ceph-mon[102809]: pgmap v422: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-10-06T21:34:22.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1401 23.38 MB/sec warmup 11 sec latency 112.035 ms 2023-10-06T21:34:23.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1486 21.47 MB/sec warmup 12 sec latency 233.090 ms 2023-10-06T21:34:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:24 smithi031 ceph-mon[98539]: pgmap v423: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 13 MiB/s wr, 312 op/s 2023-10-06T21:34:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:24 smithi146 ceph-mon[102809]: pgmap v423: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 13 MiB/s wr, 312 op/s 2023-10-06T21:34:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:24 smithi142 ceph-mon[102951]: pgmap v423: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 13 MiB/s wr, 312 op/s 2023-10-06T21:34:24.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1604 20.12 MB/sec warmup 13 sec latency 73.467 ms 2023-10-06T21:34:25.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1739 18.75 MB/sec warmup 14 sec latency 83.550 ms 2023-10-06T21:34:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:26 smithi031 ceph-mon[98539]: pgmap v424: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 12 MiB/s wr, 283 op/s 2023-10-06T21:34:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:26 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:26 smithi146 ceph-mon[102809]: pgmap v424: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 12 MiB/s wr, 283 op/s 2023-10-06T21:34:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:26 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:26 smithi142 ceph-mon[102951]: pgmap v424: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 12 MiB/s wr, 283 op/s 2023-10-06T21:34:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:26 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:26.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1848 17.57 MB/sec warmup 15 sec latency 88.570 ms 2023-10-06T21:34:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.366+0000 7f5a41fff700 1 -- 172.21.15.31:0/4014434399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c101a80 msgr2=0x7f5a3c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.366+0000 7f5a41fff700 1 --2- 172.21.15.31:0/4014434399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c101a80 0x7f5a3c105ac0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5a240098f0 tx=0x7f5a24009c00 comp rx=0 tx=0).stop 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 -- 172.21.15.31:0/4014434399 shutdown_connections 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 --2- 172.21.15.31:0/4014434399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1061f0 0x7f5a3c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 --2- 172.21.15.31:0/4014434399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 --2- 172.21.15.31:0/4014434399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c101a80 0x7f5a3c105ac0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 -- 172.21.15.31:0/4014434399 >> 172.21.15.31:0/4014434399 conn(0x7f5a3c0fc950 msgr2=0x7f5a3c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 -- 172.21.15.31:0/4014434399 shutdown_connections 2023-10-06T21:34:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.367+0000 7f5a41fff700 1 -- 172.21.15.31:0/4014434399 wait complete. 2023-10-06T21:34:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.368+0000 7f5a41fff700 1 Processor -- start 2023-10-06T21:34:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 -- start start 2023-10-06T21:34:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1061f0 0x7f5a3c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 0x7f5a3c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19e7a0 con 0x7f5a3c19dd20 2023-10-06T21:34:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19e910 con 0x7f5a3c1061f0 2023-10-06T21:34:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.369+0000 7f5a41fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19ead0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60462/0 (socket says 172.21.15.31:60462) 2023-10-06T21:34:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2572647443 learned_addr learned my addr 172.21.15.31:0/2572647443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3affd700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1061f0 0x7f5a3c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 0x7f5a3c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2572647443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1061f0 msgr2=0x7f5a3c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1061f0 0x7f5a3c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 msgr2=0x7f5a3c19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 0x7f5a3c19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2572647443 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24009520 con 0x7f5a3c1010d0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.370+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 0x7f5a3c19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00e9c0 tx=0x7f5a2c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a38ff9700 1 -- 172.21.15.31:0/2572647443 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00cbd0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a38ff9700 1 -- 172.21.15.31:0/2572647443 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c00eeb0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a38ff9700 1 -- 172.21.15.31:0/2572647443 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c005410 con 0x7f5a3c1010d0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c1a5fd0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.371+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c1a63a0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.376+0000 7f5a38ff9700 1 -- 172.21.15.31:0/2572647443 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5a2c0055b0 con 0x7f5a3c1010d0 2023-10-06T21:34:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.378+0000 7f5a55857b80 1 Processor -- start 2023-10-06T21:34:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.378+0000 7f5a38ff9700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 0x7f5a280a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.379+0000 7f5a38ff9700 1 -- 172.21.15.31:0/2572647443 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5a2c014070 con 0x7f5a3c1010d0 2023-10-06T21:34:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.379+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 0x7f5a280a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.379+0000 7f5a55857b80 1 -- start start 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f3e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e9a400 con 0x555a51f30c00 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e8bbc0 con 0x555a51f31520 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a55857b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e894e0 con 0x555a51f3c3d0 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 0x7f5a280a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a3400b040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42966/0 (socket says 172.21.15.31:42966) 2023-10-06T21:34:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2418981465 learned_addr learned my addr 172.21.15.31:0/2418981465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3affd700 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.380+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f3e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2418981465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 msgr2=0x555a51f3e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f3e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2418981465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 msgr2=0x555a51f3be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f3be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2418981465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a51cafd80 con 0x555a51f30c00 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c096ca0 tx=0x7f5a2c099ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc9cf872d89665c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.381+0000 7f5a31ffb700 1 -- 172.21.15.31:0/2418981465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c09a230 con 0x555a51f30c00 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.382+0000 7f5a31ffb700 1 -- 172.21.15.31:0/2418981465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c09e380 con 0x555a51f30c00 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.382+0000 7f5a31ffb700 1 -- 172.21.15.31:0/2418981465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c09a490 con 0x555a51f30c00 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 -- 172.21.15.31:0/2418981465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 msgr2=0x555a51f30fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c096ca0 tx=0x7f5a2c099ea0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 -- 172.21.15.31:0/2418981465 shutdown_connections 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f3e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f30fe0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2418981465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f3be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 -- 172.21.15.31:0/2418981465 >> 172.21.15.31:0/2418981465 conn(0x555a51ea6200 msgr2=0x555a51f2d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 -- 172.21.15.31:0/2418981465 shutdown_connections 2023-10-06T21:34:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.383+0000 7f5a55857b80 1 -- 172.21.15.31:0/2418981465 wait complete. 2023-10-06T21:34:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.384+0000 7f5a55857b80 1 -- start start 2023-10-06T21:34:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.384+0000 7f5a55857b80 1 Processor -- start 2023-10-06T21:34:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f39110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f39850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e84dd0 con 0x555a51f30c00 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e216b0 con 0x555a51f31520 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51e21350 con 0x555a51f3c3d0 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a55857b80 10 client.0 _ll_register_callbacks cb 0x555a51c7e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f39850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42970/0 (socket says 172.21.15.31:42970) 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f39850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60466/0 (socket says 172.21.15.31:60466) 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1527948705 learned_addr learned my addr 172.21.15.31:0/1527948705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f39110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1527948705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 msgr2=0x555a51f39850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f39850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 msgr2=0x555a51f39110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f39110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.388+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1527948705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c1a63a0 con 0x555a51f30c00 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c099f30 tx=0x7f5a2c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a33016c6c79addc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a1affd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:34:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c016540 con 0x555a51f30c00 2023-10-06T21:34:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5a2c07b020 con 0x555a51f30c00 2023-10-06T21:34:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a51f24d50 con 0x555a51f30c00 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a2c09a230 con 0x555a51f30c00 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a55857b80 10 client.17298 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.389+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x555a51c98c00 con 0x555a51f30c00 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c082030 con 0x555a51f30c00 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5a2c099720 con 0x555a51f30c00 2023-10-06T21:34:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 20 client.17298 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51f652e0 0x555a51f67700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 4 client.17298 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a51e7ddb0 con 0x555a51f652e0 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.390+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51f652e0 0x555a51f67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.391+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51f652e0 0x555a51f67700 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd5b8ee7d69e0b2 server_cookie=8d6ab056f915dac0 in_seq=0 out_seq=0 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.391+0000 7f5a1affd700 10 client.17298 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.392+0000 7f5a1affd700 1 -- 172.21.15.31:0/1527948705 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a51e7ddb0 con 0x555a51f652e0 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.392+0000 7f5a1affd700 10 client.17298 handle_command_reply: tid=0 2023-10-06T21:34:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.392+0000 7f5a55857b80 1 client.17298 shutdown 2023-10-06T21:34:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.392+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 msgr2=0x555a51f3be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.392+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c099f30 tx=0x7f5a2c009550 comp rx=0 tx=0).stop 2023-10-06T21:34:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 shutdown_connections 2023-10-06T21:34:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f3c3d0 0x555a51f39850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51f652e0 0x555a51f67700 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f30c00 0x555a51f3be90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1527948705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f31520 0x555a51f39110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 >> 172.21.15.31:0/1527948705 conn(0x555a51ea6200 msgr2=0x555a51f2cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a1affd700 0 client.17298 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 shutdown_connections 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.393+0000 7f5a55857b80 1 -- 172.21.15.31:0/1527948705 wait complete. 2023-10-06T21:34:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.394+0000 7f5a55857b80 20 client.17298 trim_cache size 0 max 16384 2023-10-06T21:34:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.419+0000 7f5a55857b80 1 Processor -- start 2023-10-06T21:34:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 -- start start 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51f34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51d058c0 con 0x555a51f31520 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51f3a6f0 con 0x555a51f3c3d0 2023-10-06T21:34:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a55857b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51f29860 con 0x555a51f30c00 2023-10-06T21:34:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a3b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60476/0 (socket says 172.21.15.31:60476) 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.420+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2159030337 learned_addr learned my addr 172.21.15.31:0/2159030337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3affd700 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2159030337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 msgr2=0x555a51f34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51f34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2159030337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 msgr2=0x555a51f344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/2159030337 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a51cd61f0 con 0x555a51f30c00 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c096730 tx=0x7f5a2c096590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e2ce643b2ef48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a327fc700 1 -- 172.21.15.31:0/2159030337 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c016540 con 0x555a51f30c00 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a327fc700 1 -- 172.21.15.31:0/2159030337 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c082070 con 0x555a51f30c00 2023-10-06T21:34:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.421+0000 7f5a327fc700 1 -- 172.21.15.31:0/2159030337 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c09a530 con 0x555a51f30c00 2023-10-06T21:34:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 -- 172.21.15.31:0/2159030337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 msgr2=0x555a51f3ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c096730 tx=0x7f5a2c096590 comp rx=0 tx=0).stop 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 -- 172.21.15.31:0/2159030337 shutdown_connections 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51f34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f3ba30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 --2- 172.21.15.31:0/2159030337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.422+0000 7f5a55857b80 1 -- 172.21.15.31:0/2159030337 >> 172.21.15.31:0/2159030337 conn(0x555a51ea6200 msgr2=0x555a51f61a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.423+0000 7f5a55857b80 1 -- 172.21.15.31:0/2159030337 shutdown_connections 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.423+0000 7f5a55857b80 1 -- 172.21.15.31:0/2159030337 wait complete. 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.423+0000 7f5a55857b80 1 -- start start 2023-10-06T21:34:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 Processor -- start 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51fb8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51f5b5b0 con 0x555a51f31520 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51fb8650 con 0x555a51f3c3d0 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.424+0000 7f5a55857b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51fb8790 con 0x555a51f30c00 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a55857b80 10 client.0 _ll_register_callbacks cb 0x7f5a2c0091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42994/0 (socket says 172.21.15.31:42994) 2023-10-06T21:34:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3affd700 1 -- 172.21.15.31:0/1898599397 learned_addr learned my addr 172.21.15.31:0/1898599397 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3bfff700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51fb8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 msgr2=0x555a51fb8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51fb8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1898599397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 msgr2=0x555a51f5aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3b7fe700 1 -- 172.21.15.31:0/1898599397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a51c98c00 con 0x555a51f30c00 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.425+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.426+0000 7f5a3b7fe700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f5a990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c09a1c0 tx=0x7f5a2c0165f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8fc97c03aae2a16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.426+0000 7f5a1affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.426+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c09a230 con 0x555a51f30c00 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.426+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5a2c07b070 con 0x555a51f30c00 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.427+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a2c09e380 con 0x555a51f30c00 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.427+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0a25c0 con 0x555a51f30c00 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.427+0000 7f5a55857b80 10 client.26929 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.427+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x555a51cae250 con 0x555a51f30c00 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.427+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c09e380 con 0x555a51f30c00 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5a2c099190 con 0x555a51f30c00 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 10 client.26929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 20 client.26929 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51fb99f0 0x555a51fbbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 4 client.26929 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:34:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a51e7dad0 con 0x555a51fb99f0 2023-10-06T21:34:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.428+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51fb99f0 0x555a51fbbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.429+0000 7f5a1affd700 10 client.26929 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.429+0000 7f5a3affd700 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51fb99f0 0x555a51fbbde0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e965a2b279c6f3b server_cookie=aae85ab64d98171b in_seq=0 out_seq=0 2023-10-06T21:34:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:27 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:27 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:27 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:34:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.528+0000 7f5a1affd700 1 -- 172.21.15.31:0/1898599397 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a51e7dad0 con 0x555a51fb99f0 2023-10-06T21:34:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.528+0000 7f5a1affd700 10 client.26929 handle_command_reply: tid=0 2023-10-06T21:34:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.530+0000 7f5a55857b80 1 client.26929 shutdown 2023-10-06T21:34:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 msgr2=0x555a51f5a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f5a990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c09a1c0 tx=0x7f5a2c0165f0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 shutdown_connections 2023-10-06T21:34:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a51f3c3d0 0x555a51fb8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555a51f30c00 0x555a51f5a990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555a51fb99f0 0x555a51fbbde0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 --2- 172.21.15.31:0/1898599397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a51f31520 0x555a51f5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 >> 172.21.15.31:0/1898599397 conn(0x555a51ea6200 msgr2=0x555a51f344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a1affd700 0 client.26929 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 shutdown_connections 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.534+0000 7f5a55857b80 1 -- 172.21.15.31:0/1898599397 wait complete. 2023-10-06T21:34:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.535+0000 7f5a55857b80 20 client.26929 trim_cache size 0 max 16384 2023-10-06T21:34:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:34:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:34:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:34:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 msgr2=0x7f5a280a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 0x7f5a280a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a3400b040 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 msgr2=0x7f5a3c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00e9c0 tx=0x7f5a2c00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 shutdown_connections 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5a280a1670 0x7f5a280a3b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a3c19dd20 0x7f5a3c19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a3c1010d0 0x7f5a3c199170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 --2- 172.21.15.31:0/2572647443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c1061f0 0x7f5a3c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 >> 172.21.15.31:0/2572647443 conn(0x7f5a3c0fc950 msgr2=0x7f5a3c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.536+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 shutdown_connections 2023-10-06T21:34:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:27.537+0000 7f5a41fff700 1 -- 172.21.15.31:0/2572647443 wait complete. 2023-10-06T21:34:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:27 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:27 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:27 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:34:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:27 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:27.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:27 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:27.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:27 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:34:27.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 1971 16.56 MB/sec warmup 16 sec latency 91.434 ms 2023-10-06T21:34:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:28 smithi031 ceph-mon[98539]: pgmap v425: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 11 MiB/s wr, 260 op/s 2023-10-06T21:34:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:28 smithi146 ceph-mon[102809]: pgmap v425: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 11 MiB/s wr, 260 op/s 2023-10-06T21:34:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:28 smithi142 ceph-mon[102951]: pgmap v425: 97 pgs: 97 active+clean; 330 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 11 MiB/s wr, 260 op/s 2023-10-06T21:34:28.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2102 15.68 MB/sec warmup 17 sec latency 71.704 ms 2023-10-06T21:34:29.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2246 15.20 MB/sec warmup 18 sec latency 81.255 ms 2023-10-06T21:34:30.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:30 smithi142 ceph-mon[102951]: pgmap v426: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 371 op/s 2023-10-06T21:34:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:30 smithi031 ceph-mon[98539]: pgmap v426: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 371 op/s 2023-10-06T21:34:30.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:30 smithi146 ceph-mon[102809]: pgmap v426: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 371 op/s 2023-10-06T21:34:30.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2358 14.64 MB/sec warmup 19 sec latency 72.801 ms 2023-10-06T21:34:31.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2477 14.25 MB/sec warmup 20 sec latency 82.244 ms 2023-10-06T21:34:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:32 smithi031 ceph-mon[98539]: pgmap v427: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2023-10-06T21:34:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:32 smithi146 ceph-mon[102809]: pgmap v427: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2023-10-06T21:34:32.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:32 smithi142 ceph-mon[102951]: pgmap v427: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2023-10-06T21:34:32.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2627 14.30 MB/sec warmup 21 sec latency 72.763 ms 2023-10-06T21:34:33.751 INFO:tasks.workunit.client.0.smithi031.stdout: 10 2820 14.41 MB/sec warmup 22 sec latency 77.143 ms 2023-10-06T21:34:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:34 smithi031 ceph-mon[98539]: pgmap v428: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.5 MiB/s wr, 247 op/s 2023-10-06T21:34:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:34 smithi146 ceph-mon[102809]: pgmap v428: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.5 MiB/s wr, 247 op/s 2023-10-06T21:34:34.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:34 smithi142 ceph-mon[102951]: pgmap v428: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.5 MiB/s wr, 247 op/s 2023-10-06T21:34:34.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3069 14.56 MB/sec warmup 23 sec latency 71.040 ms 2023-10-06T21:34:35.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3178 14.42 MB/sec warmup 24 sec latency 169.449 ms 2023-10-06T21:34:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:36 smithi031 ceph-mon[98539]: pgmap v429: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:36 smithi146 ceph-mon[102809]: pgmap v429: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:36 smithi142 ceph-mon[102951]: pgmap v429: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:36.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3293 14.08 MB/sec warmup 25 sec latency 584.236 ms 2023-10-06T21:34:37.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3450 14.31 MB/sec warmup 26 sec latency 77.288 ms 2023-10-06T21:34:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:38 smithi031 ceph-mon[98539]: pgmap v430: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:38 smithi146 ceph-mon[102809]: pgmap v430: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:38 smithi142 ceph-mon[102951]: pgmap v430: 97 pgs: 97 active+clean; 337 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2023-10-06T21:34:38.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3553 13.92 MB/sec warmup 27 sec latency 109.451 ms 2023-10-06T21:34:39.752 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3642 13.58 MB/sec warmup 28 sec latency 72.065 ms 2023-10-06T21:34:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:40 smithi142 ceph-mon[102951]: pgmap v431: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-06T21:34:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:40 smithi031 ceph-mon[98539]: pgmap v431: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-06T21:34:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:40 smithi146 ceph-mon[102809]: pgmap v431: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-06T21:34:40.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3761 13.47 MB/sec warmup 29 sec latency 154.809 ms 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.752+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2894217192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 msgr2=0x7fa7ac073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.752+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2894217192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac073660 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009990 tx=0x7fa7a8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2894217192 shutdown_connections 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2894217192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 0x7fa7ac10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2894217192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac073660 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2894217192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac10d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2894217192 >> 172.21.15.31:0/2894217192 conn(0x7fa7ac0fc940 msgr2=0x7fa7ac0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2894217192 shutdown_connections 2023-10-06T21:34:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.753+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2894217192 wait complete. 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 Processor -- start 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 -- start start 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 0x7fa7ac19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac19e2f0 con 0x7fa7ac073280 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac19e460 con 0x7fa7ac073ba0 2023-10-06T21:34:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac19e620 con 0x7fa7ac10d6f0 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.754+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50954/0 (socket says 172.21.15.31:50954) 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 -- 172.21.15.31:0/2693683311 learned_addr learned my addr 172.21.15.31:0/2693683311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 0x7fa7ac19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 -- 172.21.15.31:0/2693683311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 msgr2=0x7fa7ac19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 0x7fa7ac19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 -- 172.21.15.31:0/2693683311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 msgr2=0x7fa7ac199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 -- 172.21.15.31:0/2693683311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80095f0 con 0x7fa7ac073280 2023-10-06T21:34:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.755+0000 7fa7b159b700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.756+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2693683311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a801d070 con 0x7fa7ac073280 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.756+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac19e8a0 con 0x7fa7ac073280 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.756+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2693683311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a8022470 con 0x7fa7ac073280 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.756+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2693683311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8017650 con 0x7fa7ac073280 2023-10-06T21:34:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.756+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1a6140 con 0x7fa7ac073280 2023-10-06T21:34:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.759+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2693683311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa7a80225e0 con 0x7fa7ac073280 2023-10-06T21:34:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.760+0000 7fa7c7109b80 1 Processor -- start 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.760+0000 7fa7a27fc700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 0x7fa7980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.760+0000 7fa7c7109b80 1 -- start start 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7a27fc700 1 -- 172.21.15.31:0/2693683311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fa7a80ce5c0 con 0x7fa7ac073280 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 0x7fa7980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512f520 0x560145139e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014513a3d0 0x56014513c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560145098400 con 0x56014513a3d0 2023-10-06T21:34:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560145089bc0 con 0x56014512f520 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601450874e0 con 0x56014512ec00 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57502/0 (socket says 172.21.15.31:57502) 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b159b700 1 -- 172.21.15.31:0/3296174438 learned_addr learned my addr 172.21.15.31:0/3296174438 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512f520 0x560145139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.761+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014513a3d0 0x56014513c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 -- 172.21.15.31:0/3296174438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512f520 msgr2=0x560145139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512f520 0x560145139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 -- 172.21.15.31:0/3296174438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014513a3d0 msgr2=0x56014513c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014513a3d0 0x56014513c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 -- 172.21.15.31:0/3296174438 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560144eadd80 con 0x56014512ec00 2023-10-06T21:34:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 0x7fa7980a3ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac06a4d0 tx=0x7fa7a400fc00 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7b159b700 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004800 tx=0x7fa7a809b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c12aaa22aebee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7977fe700 1 -- 172.21.15.31:0/3296174438 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a809aec0 con 0x56014512ec00 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7977fe700 1 -- 172.21.15.31:0/3296174438 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a80228c0 con 0x56014512ec00 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.762+0000 7fa7977fe700 1 -- 172.21.15.31:0/3296174438 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a809ba20 con 0x56014512ec00 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 -- 172.21.15.31:0/3296174438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 msgr2=0x56014512efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004800 tx=0x7fa7a809b3b0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 -- 172.21.15.31:0/3296174438 shutdown_connections 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014513a3d0 0x56014513c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x56014512efe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/3296174438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512f520 0x560145139e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 -- 172.21.15.31:0/3296174438 >> 172.21.15.31:0/3296174438 conn(0x5601450a4200 msgr2=0x56014512b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 -- 172.21.15.31:0/3296174438 shutdown_connections 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.763+0000 7fa7c7109b80 1 -- 172.21.15.31:0/3296174438 wait complete. 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 -- start start 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 Processor -- start 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 0x56014513a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x56014513acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014513a3d0 0x56014513b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560145082dd0 con 0x56014512f520 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014501f6b0 con 0x56014512ec00 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014501f350 con 0x56014513a3d0 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.764+0000 7fa7c7109b80 10 client.0 _ll_register_callbacks cb 0x560144e7c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 0x56014513a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 0x56014513a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:60188/0 (socket says 172.21.15.31:60188) 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x56014513acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b159b700 1 -- 172.21.15.31:0/2033206000 learned_addr learned my addr 172.21.15.31:0/2033206000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014513a3d0 0x56014513b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/2033206000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014513a3d0 msgr2=0x56014513b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014513a3d0 0x56014513b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 msgr2=0x56014513a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 0x56014513a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.765+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/2033206000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1a6140 con 0x56014512f520 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x56014513acf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012cd0 tx=0x7fa79c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf2d7d3d8a75162 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa794ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c017070 con 0x56014512f520 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fa79c004050 con 0x56014512f520 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560145122d50 con 0x56014512f520 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa79c004210 con 0x56014512f520 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa7c7109b80 10 client.17316 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.766+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560144e96c00 con 0x56014512f520 2023-10-06T21:34:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560145122d50 con 0x56014512f520 2023-10-06T21:34:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa79c003680 con 0x56014512f520 2023-10-06T21:34:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 20 client.17316 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560145149ba0 0x56014514bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 4 client.17316 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56014507bdb0 con 0x560145149ba0 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.767+0000 7fa7b159b700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560145149ba0 0x56014514bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.768+0000 7fa7b159b700 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560145149ba0 0x56014514bfc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac0f000d69a08c server_cookie=580362af2c7bb61c in_seq=0 out_seq=0 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.768+0000 7fa794ff9700 10 client.17316 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.768+0000 7fa794ff9700 1 -- 172.21.15.31:0/2033206000 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56014507bdb0 con 0x560145149ba0 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.768+0000 7fa794ff9700 10 client.17316 handle_command_reply: tid=0 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.768+0000 7fa7c7109b80 1 client.17316 shutdown 2023-10-06T21:34:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 msgr2=0x56014513acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x56014513acf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012cd0 tx=0x7fa79c009d40 comp rx=0 tx=0).stop 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 shutdown_connections 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014513a3d0 0x56014513b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014512ec00 0x56014513a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x560145149ba0 0x56014514bfc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/2033206000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x56014513acf0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa794ff9700 0 client.17316 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 >> 172.21.15.31:0/2033206000 conn(0x5601450a4200 msgr2=0x56014512ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 shutdown_connections 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 1 -- 172.21.15.31:0/2033206000 wait complete. 2023-10-06T21:34:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.769+0000 7fa7c7109b80 20 client.17316 trim_cache size 0 max 16384 2023-10-06T21:34:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.784+0000 7fa7c7109b80 1 Processor -- start 2023-10-06T21:34:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.784+0000 7fa7c7109b80 1 -- start start 2023-10-06T21:34:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x560145138c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x560145136c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560144f038c0 con 0x56014512f520 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560145145930 con 0x56014513a3d0 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560145129e80 con 0x56014512ec00 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50970/0 (socket says 172.21.15.31:50970) 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/874490105 learned_addr learned my addr 172.21.15.31:0/874490105 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b1d9c700 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x560145136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b159b700 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x560145138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/874490105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 msgr2=0x560145138c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x560145138c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/874490105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 msgr2=0x560145136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x560145136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.785+0000 7fa7b0d9a700 1 -- 172.21.15.31:0/874490105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560145150c30 con 0x56014512f520 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.786+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0039e0 tx=0x7fa79c005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f684f1b54ca84c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.786+0000 7fa797fff700 1 -- 172.21.15.31:0/874490105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c017070 con 0x56014512f520 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.786+0000 7fa797fff700 1 -- 172.21.15.31:0/874490105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa79c018040 con 0x56014512f520 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.786+0000 7fa797fff700 1 -- 172.21.15.31:0/874490105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00fab0 con 0x56014512f520 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- 172.21.15.31:0/874490105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 msgr2=0x560145136740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0039e0 tx=0x7fa79c005860 comp rx=0 tx=0).stop 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- 172.21.15.31:0/874490105 shutdown_connections 2023-10-06T21:34:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x560145136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512ec00 0x560145138c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/874490105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512f520 0x560145136740 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- 172.21.15.31:0/874490105 >> 172.21.15.31:0/874490105 conn(0x5601450a4200 msgr2=0x5601451347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- 172.21.15.31:0/874490105 shutdown_connections 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- 172.21.15.31:0/874490105 wait complete. 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 -- start start 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.787+0000 7fa7c7109b80 1 Processor -- start 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512f520 0x56014513cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x56014519ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014519cf50 con 0x56014512ec00 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014519d090 con 0x56014513a3d0 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014519d1d0 con 0x56014512f520 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7c7109b80 10 client.0 _ll_register_callbacks cb 0x7fa79c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50974/0 (socket says 172.21.15.31:50974) 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 -- 172.21.15.31:0/4271001847 learned_addr learned my addr 172.21.15.31:0/4271001847 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512f520 0x56014513cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 -- 172.21.15.31:0/4271001847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512f520 msgr2=0x56014513cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512f520 0x56014513cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 -- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 msgr2=0x56014519ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x56014519ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.788+0000 7fa7b159b700 1 -- 172.21.15.31:0/4271001847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560144e96c00 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa7b159b700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa7a809b4f0 tx=0x7fa7a800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3d6c7a16447d05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a809b9c0 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fa7a809bc60 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7a80228c0 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a809f6b0 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa7c7109b80 10 client.17322 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560144ed41f0 con 0x56014512ec00 2023-10-06T21:34:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a80228c0 con 0x56014512ec00 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.789+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa7a80cbec0 con 0x56014512ec00 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 10 client.17322 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 20 client.17322 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56014519e580 0x5601451a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 4 client.17322 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56014507bad0 con 0x56014519e580 2023-10-06T21:34:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56014519e580 0x5601451a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa7b0d9a700 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56014519e580 0x5601451a09a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662fd609845c6c4f server_cookie=91402b7dd3114f in_seq=0 out_seq=0 2023-10-06T21:34:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.790+0000 7fa794ff9700 10 client.17322 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa794ff9700 1 -- 172.21.15.31:0/4271001847 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56014507bad0 con 0x56014519e580 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa794ff9700 10 client.17322 handle_command_reply: tid=0 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 client.17322 shutdown 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 msgr2=0x56014513c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa7a809b4f0 tx=0x7fa7a800f830 comp rx=0 tx=0).stop 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 shutdown_connections 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56014513a3d0 0x56014519ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56014519e580 0x5601451a09a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56014512ec00 0x56014513c9b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 --2- 172.21.15.31:0/4271001847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56014512f520 0x56014513cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa794ff9700 0 client.17322 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.921+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 >> 172.21.15.31:0/4271001847 conn(0x5601450a4200 msgr2=0x560145130bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.922+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 shutdown_connections 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.922+0000 7fa7c7109b80 1 -- 172.21.15.31:0/4271001847 wait complete. 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.922+0000 7fa7c7109b80 20 client.17322 trim_cache size 0 max 16384 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:34:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:34:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:34:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 msgr2=0x7fa7980a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 0x7fa7980a3ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac06a4d0 tx=0x7fa7a400fc00 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 msgr2=0x7fa7ac199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8004730 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 shutdown_connections 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7ac10d6f0 0x7fa7ac19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fa7980a15f0 0x7fa7980a3ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac073280 0x7fa7ac199100 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 --2- 172.21.15.31:0/2693683311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7ac073ba0 0x7fa7ac199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.923+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 >> 172.21.15.31:0/2693683311 conn(0x7fa7ac0fc940 msgr2=0x7fa7ac10f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.924+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 shutdown_connections 2023-10-06T21:34:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:40.924+0000 7fa7b37ff700 1 -- 172.21.15.31:0/2693683311 wait complete. 2023-10-06T21:34:41.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3849 13.17 MB/sec warmup 30 sec latency 302.361 ms 2023-10-06T21:34:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:42 smithi031 ceph-mon[98539]: pgmap v432: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 3.9 MiB/s wr, 149 op/s 2023-10-06T21:34:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:42 smithi146 ceph-mon[102809]: pgmap v432: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 3.9 MiB/s wr, 149 op/s 2023-10-06T21:34:42.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:42 smithi142 ceph-mon[102951]: pgmap v432: 97 pgs: 97 active+clean; 355 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 3.9 MiB/s wr, 149 op/s 2023-10-06T21:34:42.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 3918 12.85 MB/sec warmup 31 sec latency 340.186 ms 2023-10-06T21:34:43.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4001 12.49 MB/sec warmup 32 sec latency 131.609 ms 2023-10-06T21:34:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:44 smithi031 ceph-mon[98539]: pgmap v433: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 235 op/s 2023-10-06T21:34:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:44 smithi146 ceph-mon[102809]: pgmap v433: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 235 op/s 2023-10-06T21:34:44.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:44 smithi142 ceph-mon[102951]: pgmap v433: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 235 op/s 2023-10-06T21:34:44.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4100 12.15 MB/sec warmup 33 sec latency 81.355 ms 2023-10-06T21:34:45.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4208 11.86 MB/sec warmup 34 sec latency 83.495 ms 2023-10-06T21:34:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:46 smithi031 ceph-mon[98539]: pgmap v434: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:46 smithi146 ceph-mon[102809]: pgmap v434: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:46.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:46 smithi142 ceph-mon[102951]: pgmap v434: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:46.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4318 11.58 MB/sec warmup 35 sec latency 87.586 ms 2023-10-06T21:34:47.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4433 11.41 MB/sec warmup 36 sec latency 86.479 ms 2023-10-06T21:34:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:48 smithi031 ceph-mon[98539]: pgmap v435: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:48.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:48 smithi146 ceph-mon[102809]: pgmap v435: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:48.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:48 smithi142 ceph-mon[102951]: pgmap v435: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 3.7 MiB/s wr, 156 op/s 2023-10-06T21:34:48.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4552 11.20 MB/sec warmup 37 sec latency 86.821 ms 2023-10-06T21:34:49.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4697 11.18 MB/sec warmup 38 sec latency 85.826 ms 2023-10-06T21:34:50.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:50 smithi142 ceph-mon[102951]: pgmap v436: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:34:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:50 smithi031 ceph-mon[98539]: pgmap v436: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:34:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:50 smithi146 ceph-mon[102809]: pgmap v436: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:34:50.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4836 11.20 MB/sec warmup 39 sec latency 69.689 ms 2023-10-06T21:34:51.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 4931 10.96 MB/sec warmup 40 sec latency 245.113 ms 2023-10-06T21:34:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:52 smithi031 ceph-mon[98539]: pgmap v437: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-10-06T21:34:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:52 smithi146 ceph-mon[102809]: pgmap v437: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-10-06T21:34:52.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:52 smithi142 ceph-mon[102951]: pgmap v437: 97 pgs: 97 active+clean; 349 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 2.7 MiB/s wr, 184 op/s 2023-10-06T21:34:52.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5049 10.75 MB/sec warmup 41 sec latency 85.827 ms 2023-10-06T21:34:53.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5188 10.65 MB/sec warmup 42 sec latency 71.314 ms 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.151+0000 7faf46d9d700 1 -- 172.21.15.31:0/987923997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40100f20 msgr2=0x7faf40101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.151+0000 7faf46d9d700 1 --2- 172.21.15.31:0/987923997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40100f20 0x7faf40101300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b0a0 tx=0x7faf3800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.151+0000 7faf46d9d700 1 -- 172.21.15.31:0/987923997 shutdown_connections 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.151+0000 7faf46d9d700 1 --2- 172.21.15.31:0/987923997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40106040 0x7faf4010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.152+0000 7faf46d9d700 1 --2- 172.21.15.31:0/987923997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40100f20 0x7faf40101300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.152+0000 7faf46d9d700 1 --2- 172.21.15.31:0/987923997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401018d0 0x7faf40105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.152+0000 7faf46d9d700 1 -- 172.21.15.31:0/987923997 >> 172.21.15.31:0/987923997 conn(0x7faf400fc7c0 msgr2=0x7faf400febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.152+0000 7faf46d9d700 1 -- 172.21.15.31:0/987923997 shutdown_connections 2023-10-06T21:34:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.152+0000 7faf46d9d700 1 -- 172.21.15.31:0/987923997 wait complete. 2023-10-06T21:34:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.153+0000 7faf46d9d700 1 Processor -- start 2023-10-06T21:34:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 -- start start 2023-10-06T21:34:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40100f20 0x7faf4019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 0x7faf40194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf40195340 con 0x7faf401948c0 2023-10-06T21:34:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf40195480 con 0x7faf40106040 2023-10-06T21:34:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.154+0000 7faf46d9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf40195640 con 0x7faf40100f20 2023-10-06T21:34:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56746/0 (socket says 172.21.15.31:56746) 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 -- 172.21.15.31:0/1649595553 learned_addr learned my addr 172.21.15.31:0/1649595553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf45d9b700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40100f20 0x7faf4019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4659c700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 0x7faf40194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 -- 172.21.15.31:0/1649595553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40100f20 msgr2=0x7faf4019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40100f20 0x7faf4019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 -- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 msgr2=0x7faf40194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 0x7faf40194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4559a700 1 -- 172.21.15.31:0/1649595553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38009d00 con 0x7faf40106040 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.155+0000 7faf4659c700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 0x7faf40194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:34:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf4559a700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c9e0 tx=0x7faf3400ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf32ffd700 1 -- 172.21.15.31:0/1649595553 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34007aa0 con 0x7faf40106040 2023-10-06T21:34:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf32ffd700 1 -- 172.21.15.31:0/1649595553 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf34007c00 con 0x7faf40106040 2023-10-06T21:34:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf32ffd700 1 -- 172.21.15.31:0/1649595553 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34018660 con 0x7faf40106040 2023-10-06T21:34:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401a16c0 con 0x7faf40106040 2023-10-06T21:34:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.156+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf401a1bd0 con 0x7faf40106040 2023-10-06T21:34:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.161+0000 7faf32ffd700 1 -- 172.21.15.31:0/1649595553 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faf3400f450 con 0x7faf40106040 2023-10-06T21:34:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.162+0000 7faf5b971b80 1 Processor -- start 2023-10-06T21:34:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 -- start start 2023-10-06T21:34:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf32ffd700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 0x7faf2c0a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3333ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3334530 0x561bc3336920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc3292400 con 0x561bc3329680 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc3283bc0 con 0x561bc3328d60 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf5b971b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc32814e0 con 0x561bc3334530 2023-10-06T21:34:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.163+0000 7faf32ffd700 1 -- 172.21.15.31:0/1649595553 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7faf340cd1b0 con 0x7faf40106040 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56750/0 (socket says 172.21.15.31:56750) 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 -- 172.21.15.31:0/1393555034 learned_addr learned my addr 172.21.15.31:0/1393555034 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf4659c700 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3334530 0x561bc3336920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf4659c700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 0x7faf2c0a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 -- 172.21.15.31:0/1393555034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3334530 msgr2=0x561bc3336920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3334530 0x561bc3336920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 -- 172.21.15.31:0/1393555034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 msgr2=0x561bc3333ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3333ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.164+0000 7faf45d9b700 1 -- 172.21.15.31:0/1393555034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bc30a7d80 con 0x561bc3328d60 2023-10-06T21:34:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.165+0000 7faf45d9b700 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b0a0 tx=0x7faf3800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d61fbd4e230f0eac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.165+0000 7faf2bfff700 1 -- 172.21.15.31:0/1393555034 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3800e040 con 0x561bc3328d60 2023-10-06T21:34:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.165+0000 7faf2bfff700 1 -- 172.21.15.31:0/1393555034 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf38003a00 con 0x561bc3328d60 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.165+0000 7faf2bfff700 1 -- 172.21.15.31:0/1393555034 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3801c980 con 0x561bc3328d60 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf4659c700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 0x7faf2c0a7fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009f70 tx=0x7faf3c009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 -- 172.21.15.31:0/1393555034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 msgr2=0x561bc3329140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b0a0 tx=0x7faf3800bab0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 -- 172.21.15.31:0/1393555034 shutdown_connections 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3334530 0x561bc3336920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3329140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 --2- 172.21.15.31:0/1393555034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3333ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 -- 172.21.15.31:0/1393555034 >> 172.21.15.31:0/1393555034 conn(0x561bc329e200 msgr2=0x561bc33251b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 -- 172.21.15.31:0/1393555034 shutdown_connections 2023-10-06T21:34:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.166+0000 7faf5b971b80 1 -- 172.21.15.31:0/1393555034 wait complete. 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.167+0000 7faf5b971b80 1 -- start start 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 Processor -- start 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3331850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3334530 0x561bc3331d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc327cdd0 con 0x561bc3334530 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc32196b0 con 0x561bc3328d60 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc3219350 con 0x561bc3329680 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.168+0000 7faf5b971b80 10 client.0 _ll_register_callbacks cb 0x561bc30767a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.169+0000 7faf45d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.169+0000 7faf45d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56766/0 (socket says 172.21.15.31:56766) 2023-10-06T21:34:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.169+0000 7faf45d9b700 1 -- 172.21.15.31:0/3520179458 learned_addr learned my addr 172.21.15.31:0/3520179458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 -- 172.21.15.31:0/3520179458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 msgr2=0x561bc3331850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3331850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 -- 172.21.15.31:0/3520179458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3334530 msgr2=0x561bc3331d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3334530 0x561bc3331d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 -- 172.21.15.31:0/3520179458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4010e430 con 0x561bc3328d60 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf45d9b700 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf38000f80 tx=0x7faf38006110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d8ffb5dd972351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.170+0000 7faf297fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:34:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3800e070 con 0x561bc3328d60 2023-10-06T21:34:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7faf380078d0 con 0x561bc3328d60 2023-10-06T21:34:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bc331ceb0 con 0x561bc3328d60 2023-10-06T21:34:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf3801c4e0 con 0x561bc3328d60 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf5b971b80 10 client.27011 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.172+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561bc3090c00 con 0x561bc3328d60 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561bc331ceb0 con 0x561bc3328d60 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7faf38019030 con 0x561bc3328d60 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 10 client.27011 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 10 client.27011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 20 client.27011 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc3343d20 0x561bc3346140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 4 client.27011 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.173+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bc3275db0 con 0x561bc3343d20 2023-10-06T21:34:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.174+0000 7faf4559a700 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc3343d20 0x561bc3346140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.175+0000 7faf4559a700 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc3343d20 0x561bc3346140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c053099d95efb5cb server_cookie=721eb5d7b3ef9418 in_seq=0 out_seq=0 2023-10-06T21:34:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.175+0000 7faf297fa700 10 client.27011 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.176+0000 7faf297fa700 1 -- 172.21.15.31:0/3520179458 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561bc3275db0 con 0x561bc3343d20 2023-10-06T21:34:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.176+0000 7faf297fa700 10 client.27011 handle_command_reply: tid=0 2023-10-06T21:34:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.176+0000 7faf5b971b80 1 client.27011 shutdown 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 msgr2=0x561bc3331310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf38000f80 tx=0x7faf38006110 comp rx=0 tx=0).stop 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 shutdown_connections 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc3343d20 0x561bc3346140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3334530 0x561bc3331d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3328d60 0x561bc3331310 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3520179458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3331850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 >> 172.21.15.31:0/3520179458 conn(0x561bc329e200 msgr2=0x561bc3324da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf297fa700 0 client.27011 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 shutdown_connections 2023-10-06T21:34:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.177+0000 7faf5b971b80 1 -- 172.21.15.31:0/3520179458 wait complete. 2023-10-06T21:34:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.178+0000 7faf5b971b80 20 client.27011 trim_cache size 0 max 16384 2023-10-06T21:34:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.204+0000 7faf5b971b80 1 Processor -- start 2023-10-06T21:34:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.204+0000 7faf5b971b80 1 -- start start 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3328d60 0x561bc333b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc332e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc30fd8c0 con 0x561bc3329680 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc333b580 con 0x561bc3334530 2023-10-06T21:34:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf5b971b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc332f870 con 0x561bc3328d60 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf4559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf4559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38572/0 (socket says 172.21.15.31:38572) 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf45d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3328d60 0x561bc333b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.205+0000 7faf4559a700 1 -- 172.21.15.31:0/3224628942 learned_addr learned my addr 172.21.15.31:0/3224628942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4659c700 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc332e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4559a700 1 -- 172.21.15.31:0/3224628942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3328d60 msgr2=0x561bc333b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4559a700 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3328d60 0x561bc333b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4559a700 1 -- 172.21.15.31:0/3224628942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 msgr2=0x561bc332e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4559a700 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc332e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.206+0000 7faf4559a700 1 -- 172.21.15.31:0/3224628942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bc30ce1f0 con 0x561bc3329680 2023-10-06T21:34:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.207+0000 7faf4559a700 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7faf34096b30 tx=0x7faf3400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ff5ea737448a33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.208+0000 7faf30ff9700 1 -- 172.21.15.31:0/3224628942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340adea0 con 0x561bc3329680 2023-10-06T21:34:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.208+0000 7faf30ff9700 1 -- 172.21.15.31:0/3224628942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf3409a7c0 con 0x561bc3329680 2023-10-06T21:34:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.208+0000 7faf30ff9700 1 -- 172.21.15.31:0/3224628942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34098460 con 0x561bc3329680 2023-10-06T21:34:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 -- 172.21.15.31:0/3224628942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 msgr2=0x561bc3330ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7faf34096b30 tx=0x7faf3400f9f0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 -- 172.21.15.31:0/3224628942 shutdown_connections 2023-10-06T21:34:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc332e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3328d60 0x561bc333b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 --2- 172.21.15.31:0/3224628942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3329680 0x561bc3330ad0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.209+0000 7faf5b971b80 1 -- 172.21.15.31:0/3224628942 >> 172.21.15.31:0/3224628942 conn(0x561bc329e200 msgr2=0x561bc332c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.210+0000 7faf5b971b80 1 -- 172.21.15.31:0/3224628942 shutdown_connections 2023-10-06T21:34:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.210+0000 7faf5b971b80 1 -- 172.21.15.31:0/3224628942 wait complete. 2023-10-06T21:34:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.210+0000 7faf5b971b80 1 -- start start 2023-10-06T21:34:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.210+0000 7faf5b971b80 1 Processor -- start 2023-10-06T21:34:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3328d60 0x561bc338fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3390160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc333b1a0 con 0x561bc3328d60 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc333b310 con 0x561bc3334530 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bc33a0370 con 0x561bc3329680 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.211+0000 7faf5b971b80 10 client.0 _ll_register_callbacks cb 0x7faf38009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:56792/0 (socket says 172.21.15.31:56792) 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 -- 172.21.15.31:0/2364088745 learned_addr learned my addr 172.21.15.31:0/2364088745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:34:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4559a700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3390160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 -- 172.21.15.31:0/2364088745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 msgr2=0x561bc3390160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3390160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 -- 172.21.15.31:0/2364088745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3328d60 msgr2=0x561bc338fc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3328d60 0x561bc338fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.212+0000 7faf4659c700 1 -- 172.21.15.31:0/2364088745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf401a1bd0 con 0x561bc3334530 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf4659c700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00fb50 tx=0x7faf3c0097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce89c56cb3dcfb35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf297fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:34:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c018070 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf38003a00 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8637+0+0 (secure 0 0 0) 0x7faf3c021a20 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf3c021be0 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf5b971b80 10 client.27017 fetch_fsmap learned FSMap version 42 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.213+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561bc30a6250 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf3c035020 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7faf3c00e3b0 con 0x561bc3334530 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 10 client.27017 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 10 client.27017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 20 client.27017 populate_metadata read hostname 'smithi031' 2023-10-06T21:34:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc33a1b50 0x561bc33a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 4 client.27017 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:34:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bc3275ad0 con 0x561bc33a1b50 2023-10-06T21:34:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.214+0000 7faf45d9b700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc33a1b50 0x561bc33a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.215+0000 7faf297fa700 10 client.27017 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.215+0000 7faf45d9b700 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc33a1b50 0x561bc33a3f70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874dec3a7c6a8ebe server_cookie=7347682d5cf52425 in_seq=0 out_seq=0 2023-10-06T21:34:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf297fa700 1 -- 172.21.15.31:0/2364088745 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bc3275ad0 con 0x561bc33a1b50 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf297fa700 10 client.27017 handle_command_reply: tid=0 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 client.27017 shutdown 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 msgr2=0x561bc33906a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00fb50 tx=0x7faf3c0097b0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 shutdown_connections 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bc3334530 0x561bc33906a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561bc33a1b50 0x561bc33a3f70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bc3328d60 0x561bc338fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf297fa700 0 client.27017 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:34:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 --2- 172.21.15.31:0/2364088745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561bc3329680 0x561bc3390160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.260+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 >> 172.21.15.31:0/2364088745 conn(0x561bc329e200 msgr2=0x561bc332abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.261+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 shutdown_connections 2023-10-06T21:34:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.261+0000 7faf5b971b80 1 -- 172.21.15.31:0/2364088745 wait complete. 2023-10-06T21:34:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.261+0000 7faf5b971b80 20 client.27017 trim_cache size 0 max 16384 2023-10-06T21:34:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:34:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:34:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:34:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:34:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 msgr2=0x7faf2c0a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 0x7faf2c0a7fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009f70 tx=0x7faf3c009260 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 msgr2=0x7faf4019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c9e0 tx=0x7faf3400ccf0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 shutdown_connections 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7faf2c0a5b00 0x7faf2c0a7fc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf401948c0 0x7faf40194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf40100f20 0x7faf4019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 --2- 172.21.15.31:0/1649595553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf40106040 0x7faf4019ad90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 >> 172.21.15.31:0/1649595553 conn(0x7faf400fc7c0 msgr2=0x7faf401048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.262+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 shutdown_connections 2023-10-06T21:34:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:34:54.263+0000 7faf46d9d700 1 -- 172.21.15.31:0/1649595553 wait complete. 2023-10-06T21:34:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:54 smithi031 ceph-mon[98539]: pgmap v438: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 294 op/s 2023-10-06T21:34:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:54 smithi146 ceph-mon[102809]: pgmap v438: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 294 op/s 2023-10-06T21:34:54.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:54 smithi142 ceph-mon[102951]: pgmap v438: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 294 op/s 2023-10-06T21:34:54.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5311 10.49 MB/sec warmup 43 sec latency 90.832 ms 2023-10-06T21:34:55.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5435 10.35 MB/sec warmup 44 sec latency 70.747 ms 2023-10-06T21:34:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:56 smithi031 ceph-mon[98539]: pgmap v439: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:56 smithi146 ceph-mon[102809]: pgmap v439: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:56 smithi142 ceph-mon[102951]: pgmap v439: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:56.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5564 10.17 MB/sec warmup 45 sec latency 90.918 ms 2023-10-06T21:34:57.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5696 10.08 MB/sec warmup 46 sec latency 76.417 ms 2023-10-06T21:34:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:34:58 smithi031 ceph-mon[98539]: pgmap v440: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:34:58 smithi146 ceph-mon[102809]: pgmap v440: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:34:58 smithi142 ceph-mon[102951]: pgmap v440: 97 pgs: 97 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-10-06T21:34:58.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5803 9.92 MB/sec warmup 47 sec latency 67.658 ms 2023-10-06T21:34:59.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 5932 9.84 MB/sec warmup 48 sec latency 72.092 ms 2023-10-06T21:35:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:00 smithi142 ceph-mon[102951]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-10-06T21:35:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:00 smithi031 ceph-mon[98539]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-10-06T21:35:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:00 smithi146 ceph-mon[102809]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-10-06T21:35:00.753 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6085 9.95 MB/sec warmup 49 sec latency 97.175 ms 2023-10-06T21:35:01.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6277 10.04 MB/sec warmup 50 sec latency 71.974 ms 2023-10-06T21:35:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:02 smithi031 ceph-mon[98539]: pgmap v442: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-10-06T21:35:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:02 smithi146 ceph-mon[102809]: pgmap v442: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-10-06T21:35:02.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:02 smithi142 ceph-mon[102951]: pgmap v442: 97 pgs: 97 active+clean; 345 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2023-10-06T21:35:02.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6455 10.05 MB/sec warmup 51 sec latency 68.691 ms 2023-10-06T21:35:03.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6566 10.02 MB/sec warmup 52 sec latency 68.287 ms 2023-10-06T21:35:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:04 smithi031 ceph-mon[98539]: pgmap v443: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.0 MiB/s wr, 296 op/s 2023-10-06T21:35:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:04 smithi146 ceph-mon[102809]: pgmap v443: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.0 MiB/s wr, 296 op/s 2023-10-06T21:35:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:04 smithi142 ceph-mon[102951]: pgmap v443: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.0 MiB/s wr, 296 op/s 2023-10-06T21:35:04.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6663 9.97 MB/sec warmup 53 sec latency 82.354 ms 2023-10-06T21:35:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:05 smithi031 ceph-mon[98539]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:35:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:05 smithi146 ceph-mon[102809]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:35:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:05 smithi142 ceph-mon[102951]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:35:05.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6828 10.10 MB/sec warmup 54 sec latency 67.259 ms 2023-10-06T21:35:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:06 smithi031 ceph-mon[98539]: pgmap v445: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:06 smithi146 ceph-mon[102809]: pgmap v445: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:06 smithi142 ceph-mon[102951]: pgmap v445: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:06.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 6984 10.21 MB/sec warmup 55 sec latency 71.428 ms 2023-10-06T21:35:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 -- 172.21.15.31:0/864035751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c105ef0 msgr2=0x7ff57c1062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 --2- 172.21.15.31:0/864035751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c105ef0 0x7ff57c1062d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0099c0 tx=0x7ff56c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 -- 172.21.15.31:0/864035751 shutdown_connections 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 --2- 172.21.15.31:0/864035751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c100930 0x7ff57c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 --2- 172.21.15.31:0/864035751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c105ef0 0x7ff57c1062d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 --2- 172.21.15.31:0/864035751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c0fff70 0x7ff57c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 -- 172.21.15.31:0/864035751 >> 172.21.15.31:0/864035751 conn(0x7ff57c076b70 msgr2=0x7ff57c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 -- 172.21.15.31:0/864035751 shutdown_connections 2023-10-06T21:35:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.485+0000 7ff580bff700 1 -- 172.21.15.31:0/864035751 wait complete. 2023-10-06T21:35:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.487+0000 7ff580bff700 1 Processor -- start 2023-10-06T21:35:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.487+0000 7ff580bff700 1 -- start start 2023-10-06T21:35:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c0fff70 0x7ff57c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 0x7ff57c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19e2f0 con 0x7ff57c105ef0 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19e460 con 0x7ff57c0fff70 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff580bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19e620 con 0x7ff57c100930 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff579d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c0fff70 0x7ff57c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff579d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:40164/0 (socket says 172.21.15.31:40164) 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff579d9b700 1 -- 172.21.15.31:0/3053255195 learned_addr learned my addr 172.21.15.31:0/3053255195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.488+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 0x7ff57c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 -- 172.21.15.31:0/3053255195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c0fff70 msgr2=0x7ff57c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c0fff70 0x7ff57c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 -- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 msgr2=0x7ff57c19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 0x7ff57c19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 -- 172.21.15.31:0/3053255195 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0095f0 con 0x7ff57c100930 2023-10-06T21:35:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 0x7ff57c19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.489+0000 7ff579d9b700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff570009a20 tx=0x7ff570009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.490+0000 7ff56b7fe700 1 -- 172.21.15.31:0/3053255195 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570012070 con 0x7ff57c100930 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.490+0000 7ff56b7fe700 1 -- 172.21.15.31:0/3053255195 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff57000d650 con 0x7ff57c100930 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.490+0000 7ff56b7fe700 1 -- 172.21.15.31:0/3053255195 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570016ca0 con 0x7ff57c100930 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.490+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c19e900 con 0x7ff57c100930 2023-10-06T21:35:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.490+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c1a6140 con 0x7ff57c100930 2023-10-06T21:35:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.495+0000 7ff56b7fe700 1 -- 172.21.15.31:0/3053255195 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff57000c970 con 0x7ff57c100930 2023-10-06T21:35:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.496+0000 7ff594533b80 1 Processor -- start 2023-10-06T21:35:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 -- start start 2023-10-06T21:35:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef6f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceeec5400 con 0x55eceef64410 2023-10-06T21:35:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceeeb6bc0 con 0x55eceef64d30 2023-10-06T21:35:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.497+0000 7ff594533b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceeeb44e0 con 0x55eceef6fbe0 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff57ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff57ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33772/0 (socket says 172.21.15.31:33772) 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff57ad9d700 1 -- 172.21.15.31:0/1465727186 learned_addr learned my addr 172.21.15.31:0/1465727186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff56b7fe700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 0x7ff5640a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff57a59c700 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef6f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.498+0000 7ff56b7fe700 1 -- 172.21.15.31:0/3053255195 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8681+0+0 (secure 0 0 0) 0x7ff5700cc5f0 con 0x7ff57c100930 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 -- 172.21.15.31:0/1465727186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 msgr2=0x55eceef71fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef71fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 -- 172.21.15.31:0/1465727186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 msgr2=0x55eceef6f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef6f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 -- 172.21.15.31:0/1465727186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eceecdad80 con 0x55eceef64410 2023-10-06T21:35:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.499+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 0x7ff5640a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.500+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff574007530 tx=0x7ff574007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb02ab726c89d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.500+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 0x7ff5640a3c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5740090a0 tx=0x7ff574015000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.501+0000 7ff568ff9700 1 -- 172.21.15.31:0/1465727186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400e040 con 0x55eceef64410 2023-10-06T21:35:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.501+0000 7ff568ff9700 1 -- 172.21.15.31:0/1465727186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff57400f040 con 0x55eceef64410 2023-10-06T21:35:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.501+0000 7ff568ff9700 1 -- 172.21.15.31:0/1465727186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574020b10 con 0x55eceef64410 2023-10-06T21:35:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.502+0000 7ff594533b80 1 -- 172.21.15.31:0/1465727186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 msgr2=0x55eceef647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.502+0000 7ff594533b80 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff574007530 tx=0x7ff574007840 comp rx=0 tx=0).stop 2023-10-06T21:35:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.504+0000 7ff594533b80 1 -- 172.21.15.31:0/1465727186 shutdown_connections 2023-10-06T21:35:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.504+0000 7ff594533b80 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef71fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.504+0000 7ff594533b80 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef647f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.504+0000 7ff594533b80 1 --2- 172.21.15.31:0/1465727186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef6f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.504+0000 7ff594533b80 1 -- 172.21.15.31:0/1465727186 >> 172.21.15.31:0/1465727186 conn(0x55eceeed1200 msgr2=0x55eceef57ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.506+0000 7ff594533b80 1 -- 172.21.15.31:0/1465727186 shutdown_connections 2023-10-06T21:35:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.506+0000 7ff594533b80 1 -- 172.21.15.31:0/1465727186 wait complete. 2023-10-06T21:35:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.506+0000 7ff594533b80 1 -- start start 2023-10-06T21:35:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.507+0000 7ff594533b80 1 Processor -- start 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64d30 0x55eceef6cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceeeafdd0 con 0x55eceef64d30 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceee4c6b0 con 0x55eceef64410 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceee4c350 con 0x55eceef6fbe0 2023-10-06T21:35:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.508+0000 7ff594533b80 10 client.0 _ll_register_callbacks cb 0x55eceeca97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.509+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.509+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:36194/0 (socket says 172.21.15.31:36194) 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.509+0000 7ff57a59c700 1 -- 172.21.15.31:0/774669154 learned_addr learned my addr 172.21.15.31:0/774669154 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57a59c700 1 -- 172.21.15.31:0/774669154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 msgr2=0x55eceef6d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57a59c700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef6d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57a59c700 1 -- 172.21.15.31:0/774669154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64d30 msgr2=0x55eceef6cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57a59c700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64d30 0x55eceef6cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.510+0000 7ff57a59c700 1 -- 172.21.15.31:0/774669154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574009140 con 0x55eceef64410 2023-10-06T21:35:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.512+0000 7ff57a59c700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0099c0 tx=0x7ff56c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c70eb3dc13da44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.512+0000 7ff561ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:35:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.512+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c01d050 con 0x55eceef64410 2023-10-06T21:35:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.512+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8681+0+0 (secure 0 0 0) 0x7ff56c0054e0 con 0x55eceef64410 2023-10-06T21:35:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.512+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eceef4fd50 con 0x55eceef64410 2023-10-06T21:35:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.513+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff56c003760 con 0x55eceef64410 2023-10-06T21:35:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.513+0000 7ff594533b80 10 client.27026 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.513+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55eceecc3c00 con 0x55eceef64410 2023-10-06T21:35:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eceef4fd50 con 0x55eceef64410 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff56c00f6f0 con 0x55eceef64410 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 10 client.27026 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 10 client.27026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 20 client.27026 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceef7f9b0 0x55eceef81dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 4 client.27026 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eceeea8db0 con 0x55eceef7f9b0 2023-10-06T21:35:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.514+0000 7ff579d9b700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceef7f9b0 0x55eceef81dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.516+0000 7ff579d9b700 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceef7f9b0 0x55eceef81dd0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457a3cbaa26a4b46 server_cookie=82f28a92770757b6 in_seq=0 out_seq=0 2023-10-06T21:35:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.516+0000 7ff561ffb700 10 client.27026 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.516+0000 7ff561ffb700 1 -- 172.21.15.31:0/774669154 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eceeea8db0 con 0x55eceef7f9b0 2023-10-06T21:35:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.516+0000 7ff561ffb700 10 client.27026 handle_command_reply: tid=0 2023-10-06T21:35:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 client.27026 shutdown 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 msgr2=0x55eceef6cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0099c0 tx=0x7ff56c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 shutdown_connections 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceef6d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64410 0x55eceef6cab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceef7f9b0 0x55eceef81dd0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 --2- 172.21.15.31:0/774669154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64d30 0x55eceef6cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 >> 172.21.15.31:0/774669154 conn(0x55eceeed1200 msgr2=0x55eceef57b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.517+0000 7ff561ffb700 0 client.27026 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.518+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 shutdown_connections 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.518+0000 7ff594533b80 1 -- 172.21.15.31:0/774669154 wait complete. 2023-10-06T21:35:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.518+0000 7ff594533b80 20 client.27026 trim_cache size 0 max 16384 2023-10-06T21:35:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 Processor -- start 2023-10-06T21:35:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 -- start start 2023-10-06T21:35:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef64d30 0x55eceef76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef6fbe0 0x55eceef6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceed308c0 con 0x55eceef64410 2023-10-06T21:35:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceef7cf30 con 0x55eceef6fbe0 2023-10-06T21:35:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.541+0000 7ff594533b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceef65530 con 0x55eceef64d30 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33788/0 (socket says 172.21.15.31:33788) 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 -- 172.21.15.31:0/2939140578 learned_addr learned my addr 172.21.15.31:0/2939140578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff579d9b700 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef64d30 0x55eceef76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef6fbe0 0x55eceef6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 -- 172.21.15.31:0/2939140578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef64d30 msgr2=0x55eceef76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef64d30 0x55eceef76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 -- 172.21.15.31:0/2939140578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef6fbe0 msgr2=0x55eceef6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef6fbe0 0x55eceef6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.542+0000 7ff57a59c700 1 -- 172.21.15.31:0/2939140578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eceed011f0 con 0x55eceef64410 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.543+0000 7ff57a59c700 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009d10 tx=0x7ff56c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd22ff37d4bae182 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.543+0000 7ff5697fa700 1 -- 172.21.15.31:0/2939140578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c01d070 con 0x55eceef64410 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.543+0000 7ff5697fa700 1 -- 172.21.15.31:0/2939140578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff56c01e040 con 0x55eceef64410 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.544+0000 7ff5697fa700 1 -- 172.21.15.31:0/2939140578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0039c0 con 0x55eceef64410 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.544+0000 7ff594533b80 1 -- 172.21.15.31:0/2939140578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 msgr2=0x55eceef67480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.544+0000 7ff594533b80 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009d10 tx=0x7ff56c009290 comp rx=0 tx=0).stop 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.545+0000 7ff594533b80 1 -- 172.21.15.31:0/2939140578 shutdown_connections 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.545+0000 7ff594533b80 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef6fbe0 0x55eceef6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.545+0000 7ff594533b80 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef67480 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.545+0000 7ff594533b80 1 --2- 172.21.15.31:0/2939140578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef64d30 0x55eceef76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.545+0000 7ff594533b80 1 -- 172.21.15.31:0/2939140578 >> 172.21.15.31:0/2939140578 conn(0x55eceeed1200 msgr2=0x55eceef67de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.546+0000 7ff594533b80 1 -- 172.21.15.31:0/2939140578 shutdown_connections 2023-10-06T21:35:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.546+0000 7ff594533b80 1 -- 172.21.15.31:0/2939140578 wait complete. 2023-10-06T21:35:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.547+0000 7ff594533b80 1 -- start start 2023-10-06T21:35:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.547+0000 7ff594533b80 1 Processor -- start 2023-10-06T21:35:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef74c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceefd27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceef75590 con 0x55eceef64410 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceefd2ce0 con 0x55eceef64d30 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eceefd2e20 con 0x55eceef6fbe0 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.548+0000 7ff594533b80 10 client.0 _ll_register_callbacks cb 0x7ff56c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33792/0 (socket says 172.21.15.31:33792) 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57a59c700 1 -- 172.21.15.31:0/749304991 learned_addr learned my addr 172.21.15.31:0/749304991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef74c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceefd27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 -- 172.21.15.31:0/749304991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 msgr2=0x55eceefd27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceefd27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 -- 172.21.15.31:0/749304991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 msgr2=0x55eceef74740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff579d9b700 1 -- 172.21.15.31:0/749304991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eceecc3c00 con 0x55eceef64d30 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57a59c700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.549+0000 7ff57ad9d700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceefd27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff579d9b700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef74c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff5700c9c40 tx=0x7ff570000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494421588ac33ed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff561ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570009e50 con 0x55eceef64d30 2023-10-06T21:35:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff57400f040 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8681+0+0 (secure 0 0 0) 0x7ff570099bd0 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5700994d0 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff594533b80 10 client.27032 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.550+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55eceecd9250 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff57400f040 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff5700a9070 con 0x55eceef64d30 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 10 client.27032 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 20 client.27032 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceefd44d0 0x55eceefd68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 4 client.27032 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:35:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eceeea8ad0 con 0x55eceefd44d0 2023-10-06T21:35:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.551+0000 7ff57a59c700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceefd44d0 0x55eceefd68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.552+0000 7ff57a59c700 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceefd44d0 0x55eceefd68f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688ad917c53b7ac5 server_cookie=28e010abf94d82f2 in_seq=0 out_seq=0 2023-10-06T21:35:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.552+0000 7ff561ffb700 10 client.27032 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.602+0000 7ff561ffb700 1 -- 172.21.15.31:0/749304991 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eceeea8ad0 con 0x55eceefd44d0 2023-10-06T21:35:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.602+0000 7ff561ffb700 10 client.27032 handle_command_reply: tid=0 2023-10-06T21:35:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.602+0000 7ff594533b80 1 client.27032 shutdown 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.602+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 msgr2=0x55eceef74c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.602+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef74c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff5700c9c40 tx=0x7ff570000bc0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 shutdown_connections 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eceef6fbe0 0x55eceefd27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55eceefd44d0 0x55eceefd68f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eceef64410 0x55eceef74740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 --2- 172.21.15.31:0/749304991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eceef64d30 0x55eceef74c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 >> 172.21.15.31:0/749304991 conn(0x55eceeed1200 msgr2=0x55eceef57250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff561ffb700 0 client.27032 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 shutdown_connections 2023-10-06T21:35:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 1 -- 172.21.15.31:0/749304991 wait complete. 2023-10-06T21:35:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.603+0000 7ff594533b80 20 client.27032 trim_cache size 0 max 16384 2023-10-06T21:35:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:35:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:35:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:35:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 msgr2=0x7ff5640a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 0x7ff5640a3c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5740090a0 tx=0x7ff574015000 comp rx=0 tx=0).stop 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 msgr2=0x7ff57c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff570009a20 tx=0x7ff570009d30 comp rx=0 tx=0).stop 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 shutdown_connections 2023-10-06T21:35:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff5640a17c0 0x7ff5640a3c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff57c105ef0 0x7ff57c19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff57c0fff70 0x7ff57c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 --2- 172.21.15.31:0/3053255195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff57c100930 0x7ff57c199640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 >> 172.21.15.31:0/3053255195 conn(0x7ff57c076b70 msgr2=0x7ff57c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.604+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 shutdown_connections 2023-10-06T21:35:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:07.605+0000 7ff580bff700 1 -- 172.21.15.31:0/3053255195 wait complete. 2023-10-06T21:35:07.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7109 10.12 MB/sec warmup 56 sec latency 84.001 ms 2023-10-06T21:35:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:08 smithi031 ceph-mon[98539]: pgmap v446: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:08 smithi146 ceph-mon[102809]: pgmap v446: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:08 smithi142 ceph-mon[102951]: pgmap v446: 97 pgs: 97 active+clean; 365 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 224 op/s 2023-10-06T21:35:08.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7193 10.03 MB/sec warmup 57 sec latency 71.137 ms 2023-10-06T21:35:09.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7289 9.98 MB/sec warmup 58 sec latency 65.002 ms 2023-10-06T21:35:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:10 smithi142 ceph-mon[102951]: pgmap v447: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:10 smithi031 ceph-mon[98539]: pgmap v447: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:10 smithi146 ceph-mon[102809]: pgmap v447: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:10.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7370 9.83 MB/sec warmup 59 sec latency 65.128 ms 2023-10-06T21:35:11.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7455 9.80 MB/sec warmup 60 sec latency 361.701 ms 2023-10-06T21:35:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:12 smithi031 ceph-mon[98539]: pgmap v448: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:12 smithi146 ceph-mon[102809]: pgmap v448: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:12.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:12 smithi142 ceph-mon[102951]: pgmap v448: 97 pgs: 97 active+clean; 382 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.9 MiB/s wr, 219 op/s 2023-10-06T21:35:12.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7561 9.66 MB/sec warmup 61 sec latency 70.036 ms 2023-10-06T21:35:13.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7671 9.56 MB/sec warmup 62 sec latency 81.853 ms 2023-10-06T21:35:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:14 smithi031 ceph-mon[98539]: pgmap v449: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:14 smithi031 ceph-mon[98539]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:35:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:14 smithi146 ceph-mon[102809]: pgmap v449: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:14 smithi146 ceph-mon[102809]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:35:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:14 smithi142 ceph-mon[102951]: pgmap v449: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:14.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:14 smithi142 ceph-mon[102951]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:35:14.754 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7782 9.45 MB/sec warmup 63 sec latency 82.971 ms 2023-10-06T21:35:15.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7890 9.38 MB/sec warmup 64 sec latency 82.086 ms 2023-10-06T21:35:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:16 smithi031 ceph-mon[98539]: pgmap v451: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:16 smithi146 ceph-mon[102809]: pgmap v451: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:16 smithi142 ceph-mon[102951]: pgmap v451: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:16.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 7971 9.29 MB/sec warmup 65 sec latency 363.941 ms 2023-10-06T21:35:17.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8092 9.22 MB/sec warmup 66 sec latency 66.548 ms 2023-10-06T21:35:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:18 smithi031 ceph-mon[98539]: pgmap v452: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:18 smithi146 ceph-mon[102809]: pgmap v452: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:18 smithi142 ceph-mon[102951]: pgmap v452: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-06T21:35:18.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8233 9.27 MB/sec warmup 67 sec latency 70.711 ms 2023-10-06T21:35:19.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8348 9.20 MB/sec warmup 68 sec latency 79.828 ms 2023-10-06T21:35:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:20 smithi142 ceph-mon[102951]: pgmap v453: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:20 smithi031 ceph-mon[98539]: pgmap v453: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:20.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:20 smithi146 ceph-mon[102809]: pgmap v453: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:20.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8466 9.10 MB/sec warmup 69 sec latency 90.717 ms 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 -- 172.21.15.31:0/4160475504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0864101a80 msgr2=0x7f0864105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 --2- 172.21.15.31:0/4160475504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0864101a80 0x7f0864105ac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0858009990 tx=0x7f0858009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 -- 172.21.15.31:0/4160475504 shutdown_connections 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 --2- 172.21.15.31:0/4160475504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08641061f0 0x7f086410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 --2- 172.21.15.31:0/4160475504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f08641014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 --2- 172.21.15.31:0/4160475504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0864101a80 0x7f0864105ac0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.829+0000 7f0869fff700 1 -- 172.21.15.31:0/4160475504 >> 172.21.15.31:0/4160475504 conn(0x7f08640fc950 msgr2=0x7f08640fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.830+0000 7f0869fff700 1 -- 172.21.15.31:0/4160475504 shutdown_connections 2023-10-06T21:35:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.830+0000 7f0869fff700 1 -- 172.21.15.31:0/4160475504 wait complete. 2023-10-06T21:35:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.831+0000 7f0869fff700 1 Processor -- start 2023-10-06T21:35:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.831+0000 7f0869fff700 1 -- start start 2023-10-06T21:35:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08641061f0 0x7f08641996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086419dd30 0x7f086419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086419e7b0 con 0x7f086419dd30 2023-10-06T21:35:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086419e920 con 0x7f08641010d0 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0869fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086419eae0 con 0x7f08641061f0 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f08637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f08637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:59568/0 (socket says 172.21.15.31:59568) 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f08637fe700 1 -- 172.21.15.31:0/752442668 learned_addr learned my addr 172.21.15.31:0/752442668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0863fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086419dd30 0x7f086419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f0862ffd700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08641061f0 0x7f08641996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f08637fe700 1 -- 172.21.15.31:0/752442668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08641061f0 msgr2=0x7f08641996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.832+0000 7f08637fe700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08641061f0 0x7f08641996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f08637fe700 1 -- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086419dd30 msgr2=0x7f086419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f08637fe700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086419dd30 0x7f086419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f08637fe700 1 -- 172.21.15.31:0/752442668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580095f0 con 0x7f08641010d0 2023-10-06T21:35:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f08637fe700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9a0 tx=0x7f085400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f0860ff9700 1 -- 172.21.15.31:0/752442668 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400ca10 con 0x7f08641010d0 2023-10-06T21:35:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.833+0000 7f0860ff9700 1 -- 172.21.15.31:0/752442668 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0854010430 con 0x7f08641010d0 2023-10-06T21:35:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.834+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08641a5fe0 con 0x7f08641010d0 2023-10-06T21:35:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.834+0000 7f0860ff9700 1 -- 172.21.15.31:0/752442668 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854018680 con 0x7f08641010d0 2023-10-06T21:35:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.834+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08641a6460 con 0x7f08641010d0 2023-10-06T21:35:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.839+0000 7f0860ff9700 1 -- 172.21.15.31:0/752442668 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f08540105a0 con 0x7f08641010d0 2023-10-06T21:35:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.840+0000 7f087d93cb80 1 Processor -- start 2023-10-06T21:35:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 -- start start 2023-10-06T21:35:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e12fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36d66400 con 0x55ca36e10be0 2023-10-06T21:35:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36d57bc0 con 0x55ca36dfcc00 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f087d93cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36d554e0 con 0x55ca36dfd520 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f08637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.841+0000 7f0863fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:59584/0 (socket says 172.21.15.31:59584) 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e12fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 -- 172.21.15.31:0/641700008 learned_addr learned my addr 172.21.15.31:0/641700008 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 -- 172.21.15.31:0/641700008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 msgr2=0x55ca36e106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 -- 172.21.15.31:0/641700008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 msgr2=0x55ca36e12fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e12fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0863fff700 1 -- 172.21.15.31:0/641700008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca36b7bd80 con 0x55ca36dfcc00 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.842+0000 7f0862ffd700 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e12fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0863fff700 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f085c011680 tx=0x7f085c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa787a5aed9f4a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0860ff9700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 0x7f084c0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0849ffb700 1 -- 172.21.15.31:0/641700008 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00e040 con 0x55ca36dfcc00 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0849ffb700 1 -- 172.21.15.31:0/641700008 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f085c00f040 con 0x55ca36dfcc00 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0849ffb700 1 -- 172.21.15.31:0/641700008 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c0046e0 con 0x55ca36dfcc00 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f0860ff9700 1 -- 172.21.15.31:0/752442668 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0854014070 con 0x7f08641010d0 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.843+0000 7f08637fe700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 0x7f084c0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 -- 172.21.15.31:0/641700008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 msgr2=0x55ca36dfcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f085c011680 tx=0x7f085c009920 comp rx=0 tx=0).stop 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 -- 172.21.15.31:0/641700008 shutdown_connections 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e12fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfcc00 0x55ca36dfcfe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 --2- 172.21.15.31:0/641700008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 -- 172.21.15.31:0/641700008 >> 172.21.15.31:0/641700008 conn(0x55ca36d72200 msgr2=0x55ca36df9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 -- 172.21.15.31:0/641700008 shutdown_connections 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.844+0000 7f087d93cb80 1 -- 172.21.15.31:0/641700008 wait complete. 2023-10-06T21:35:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.845+0000 7f08637fe700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 0x7f084c0a3c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08540ca9b0 tx=0x7f08540d0b60 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.845+0000 7f087d93cb80 1 -- start start 2023-10-06T21:35:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.845+0000 7f087d93cb80 1 Processor -- start 2023-10-06T21:35:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e10670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36e10be0 0x55ca36e0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36d50dd0 con 0x55ca36dfcc00 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ced6b0 con 0x55ca36e10be0 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ced350 con 0x55ca36dfd520 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f087d93cb80 10 client.0 _ll_register_callbacks cb 0x55ca36b4a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46828/0 (socket says 172.21.15.31:46828) 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f0862ffd700 1 -- 172.21.15.31:0/3691354511 learned_addr learned my addr 172.21.15.31:0/3691354511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.846+0000 7f0863fff700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e10670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0863fff700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36e10be0 0x55ca36e0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 -- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36e10be0 msgr2=0x55ca36e0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36e10be0 0x55ca36e0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 -- 172.21.15.31:0/3691354511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 msgr2=0x55ca36e10670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e10670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 -- 172.21.15.31:0/3691354511 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086410e5e0 con 0x55ca36dfd520 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0863fff700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0858009990 tx=0x7f0858009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d4aeb9d8e520de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.847+0000 7f0842ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.848+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085801d070 con 0x55ca36dfd520 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.848+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0858004500 con 0x55ca36dfd520 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.848+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca36df0d50 con 0x55ca36dfd520 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.849+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08580046c0 con 0x55ca36dfd520 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.849+0000 7f087d93cb80 10 client.26989 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.849+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f085c011330 con 0x55ca36dfd520 2023-10-06T21:35:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.849+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca36df0d50 con 0x55ca36dfd520 2023-10-06T21:35:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f085800f7b0 con 0x55ca36dfd520 2023-10-06T21:35:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 10 client.26989 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 10 client.26989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 20 client.26989 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e20820 0x55ca36e22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 4 client.26989 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca36d49db0 con 0x55ca36e20820 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.850+0000 7f0863fff700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e20820 0x55ca36e22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.851+0000 7f0863fff700 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e20820 0x55ca36e22c40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb7b469c26d7a56 server_cookie=1bb012b01a761693 in_seq=0 out_seq=0 2023-10-06T21:35:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.851+0000 7f0842ffd700 10 client.26989 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f0842ffd700 1 -- 172.21.15.31:0/3691354511 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ca36d49db0 con 0x55ca36e20820 2023-10-06T21:35:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f0842ffd700 10 client.26989 handle_command_reply: tid=0 2023-10-06T21:35:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 client.26989 shutdown 2023-10-06T21:35:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 msgr2=0x55ca36e0d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0858009990 tx=0x7f0858009910 comp rx=0 tx=0).stop 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 shutdown_connections 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36e10be0 0x55ca36e0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.852+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e20820 0x55ca36e22c40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3691354511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfd520 0x55ca36e0d8f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f0842ffd700 0 client.26989 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 >> 172.21.15.31:0/3691354511 conn(0x55ca36d72200 msgr2=0x55ca36df8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 shutdown_connections 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 1 -- 172.21.15.31:0/3691354511 wait complete. 2023-10-06T21:35:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.853+0000 7f087d93cb80 20 client.26989 trim_cache size 0 max 16384 2023-10-06T21:35:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.878+0000 7f087d93cb80 1 Processor -- start 2023-10-06T21:35:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.878+0000 7f087d93cb80 1 -- start start 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.878+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 0x55ca36e12fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.878+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e10210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f087d93cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36bd18c0 con 0x55ca36e10be0 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f087d93cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36dfdb60 con 0x55ca36dfd520 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f087d93cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36dffaa0 con 0x55ca36dfcc00 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f0863fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 0x55ca36e12fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f0863fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 0x55ca36e12fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46832/0 (socket says 172.21.15.31:46832) 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f0863fff700 1 -- 172.21.15.31:0/3335000149 learned_addr learned my addr 172.21.15.31:0/3335000149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f0863fff700 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.879+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e10210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 -- 172.21.15.31:0/3335000149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 msgr2=0x55ca36e12fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 0x55ca36e12fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 -- 172.21.15.31:0/3335000149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 msgr2=0x55ca36e0ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e0ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 -- 172.21.15.31:0/3335000149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca36b64c00 con 0x55ca36dfd520 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.880+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e10210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0858006010 tx=0x7f0858009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba18b0846d799fe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.881+0000 7f084a7fc700 1 -- 172.21.15.31:0/3335000149 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085801d040 con 0x55ca36dfd520 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.881+0000 7f084a7fc700 1 -- 172.21.15.31:0/3335000149 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f085801e040 con 0x55ca36dfd520 2023-10-06T21:35:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.881+0000 7f084a7fc700 1 -- 172.21.15.31:0/3335000149 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580249f0 con 0x55ca36dfd520 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.881+0000 7f087d93cb80 1 -- 172.21.15.31:0/3335000149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 msgr2=0x55ca36e10210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.881+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e10210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0858006010 tx=0x7f0858009290 comp rx=0 tx=0).stop 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 -- 172.21.15.31:0/3335000149 shutdown_connections 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36e10be0 0x55ca36e0ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36dfcc00 0x55ca36e12fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3335000149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e10210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 -- 172.21.15.31:0/3335000149 >> 172.21.15.31:0/3335000149 conn(0x55ca36d72200 msgr2=0x55ca36e08c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 -- 172.21.15.31:0/3335000149 shutdown_connections 2023-10-06T21:35:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 -- 172.21.15.31:0/3335000149 wait complete. 2023-10-06T21:35:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.882+0000 7f087d93cb80 1 -- start start 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 Processor -- start 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 0x55ca36e6d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e6dbe0 con 0x55ca36dfcc00 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e6dd50 con 0x55ca36dfd520 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e6df10 con 0x55ca36e10be0 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.883+0000 7f087d93cb80 10 client.0 _ll_register_callbacks cb 0x7f0858005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:59610/0 (socket says 172.21.15.31:59610) 2023-10-06T21:35:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 -- 172.21.15.31:0/3327688260 learned_addr learned my addr 172.21.15.31:0/3327688260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0863fff700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 0x55ca36e6d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 -- 172.21.15.31:0/3327688260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 msgr2=0x55ca36e6d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 0x55ca36e6d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 -- 172.21.15.31:0/3327688260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 msgr2=0x55ca36e15650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0862ffd700 1 -- 172.21.15.31:0/3327688260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c011330 con 0x55ca36dfd520 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.884+0000 7f0863fff700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 0x55ca36e6d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f0862ffd700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0858005580 tx=0x7f085800f940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8624093a3725ade7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f0842ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:35:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580249a0 con 0x55ca36dfd520 2023-10-06T21:35:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0858000e40 con 0x55ca36dfd520 2023-10-06T21:35:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f085c00f040 con 0x55ca36dfd520 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f085800b2d0 con 0x55ca36dfd520 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f087d93cb80 10 client.27050 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.885+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ca36ba21f0 con 0x55ca36dfd520 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f085c00f040 con 0x55ca36dfd520 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0858004c20 con 0x55ca36dfd520 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 10 client.27050 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 20 client.27050 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e752f0 0x55ca36e77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 4 client.27050 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:35:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.886+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca36d49ad0 con 0x55ca36e752f0 2023-10-06T21:35:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.887+0000 7f0863fff700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e752f0 0x55ca36e77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.887+0000 7f0842ffd700 10 client.27050 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.887+0000 7f0863fff700 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e752f0 0x55ca36e77710 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b76fab232b757e server_cookie=d6418d257a30699a in_seq=0 out_seq=0 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f0842ffd700 1 -- 172.21.15.31:0/3327688260 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca36d49ad0 con 0x55ca36e752f0 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f0842ffd700 10 client.27050 handle_command_reply: tid=0 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 client.27050 shutdown 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 msgr2=0x55ca36e15b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0858005580 tx=0x7f085800f940 comp rx=0 tx=0).stop 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 shutdown_connections 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ca36e10be0 0x55ca36e6d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.929+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ca36e752f0 0x55ca36e77710 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca36dfcc00 0x55ca36e15650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 1 --2- 172.21.15.31:0/3327688260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca36dfd520 0x55ca36e15b90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 >> 172.21.15.31:0/3327688260 conn(0x55ca36d72200 msgr2=0x55ca36dfea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f0842ffd700 0 client.27050 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 shutdown_connections 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 1 -- 172.21.15.31:0/3327688260 wait complete. 2023-10-06T21:35:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.930+0000 7f087d93cb80 20 client.27050 trim_cache size 0 max 16384 2023-10-06T21:35:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:35:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:35:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:35:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.931+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 msgr2=0x7f084c0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.931+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 0x7f084c0a3c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08540ca9b0 tx=0x7f08540d0b60 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.931+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 msgr2=0x7f0864199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.931+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9a0 tx=0x7f085400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 shutdown_connections 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f084c0a1740 0x7f084c0a3c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086419dd30 0x7f086419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08641010d0 0x7f0864199180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 --2- 172.21.15.31:0/752442668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08641061f0 0x7f08641996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 >> 172.21.15.31:0/752442668 conn(0x7f08640fc950 msgr2=0x7f08640fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 shutdown_connections 2023-10-06T21:35:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:20.932+0000 7f0869fff700 1 -- 172.21.15.31:0/752442668 wait complete. 2023-10-06T21:35:21.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8585 8.99 MB/sec warmup 70 sec latency 100.177 ms 2023-10-06T21:35:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:22 smithi031 ceph-mon[98539]: pgmap v454: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:22 smithi146 ceph-mon[102809]: pgmap v454: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:22.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:22 smithi142 ceph-mon[102951]: pgmap v454: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 219 op/s 2023-10-06T21:35:22.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8669 8.92 MB/sec warmup 71 sec latency 438.492 ms 2023-10-06T21:35:23.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8797 8.86 MB/sec warmup 72 sec latency 65.962 ms 2023-10-06T21:35:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:24 smithi031 ceph-mon[98539]: pgmap v455: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-10-06T21:35:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:24 smithi146 ceph-mon[102809]: pgmap v455: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-10-06T21:35:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:24 smithi142 ceph-mon[102951]: pgmap v455: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-10-06T21:35:24.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 8921 8.77 MB/sec warmup 73 sec latency 75.450 ms 2023-10-06T21:35:25.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9058 8.75 MB/sec warmup 74 sec latency 70.969 ms 2023-10-06T21:35:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:26 smithi031 ceph-mon[98539]: pgmap v456: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-10-06T21:35:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:26 smithi146 ceph-mon[102809]: pgmap v456: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-10-06T21:35:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:26 smithi142 ceph-mon[102951]: pgmap v456: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-10-06T21:35:26.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9144 8.67 MB/sec warmup 75 sec latency 415.411 ms 2023-10-06T21:35:27.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9256 8.64 MB/sec warmup 76 sec latency 77.041 ms 2023-10-06T21:35:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:28 smithi031 ceph-mon[98539]: pgmap v457: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2023-10-06T21:35:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:28 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:28 smithi146 ceph-mon[102809]: pgmap v457: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2023-10-06T21:35:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:28 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:28 smithi142 ceph-mon[102951]: pgmap v457: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2023-10-06T21:35:28.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:28 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:28.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9373 8.61 MB/sec warmup 77 sec latency 84.498 ms 2023-10-06T21:35:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:29 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:29 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:29 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:35:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:29 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:29 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:29 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:35:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:29 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:29 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:29 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:35:29.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9525 8.63 MB/sec warmup 78 sec latency 67.418 ms 2023-10-06T21:35:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:30 smithi031 ceph-mon[98539]: pgmap v458: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-10-06T21:35:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:30 smithi146 ceph-mon[102809]: pgmap v458: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-10-06T21:35:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:30 smithi142 ceph-mon[102951]: pgmap v458: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-10-06T21:35:30.755 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9684 8.71 MB/sec warmup 79 sec latency 394.210 ms 2023-10-06T21:35:31.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9846 8.71 MB/sec warmup 80 sec latency 61.605 ms 2023-10-06T21:35:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:32 smithi031 ceph-mon[98539]: pgmap v459: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2023-10-06T21:35:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:32 smithi146 ceph-mon[102809]: pgmap v459: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2023-10-06T21:35:32.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:32 smithi142 ceph-mon[102951]: pgmap v459: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2023-10-06T21:35:32.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 9996 8.70 MB/sec warmup 81 sec latency 65.709 ms 2023-10-06T21:35:33.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10090 8.70 MB/sec warmup 82 sec latency 64.292 ms 2023-10-06T21:35:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/3550206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4105e20 msgr2=0x7fc3e40682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/3550206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4105e20 0x7fc3e40682d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009990 tx=0x7fc3d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/3550206576 shutdown_connections 2023-10-06T21:35:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/3550206576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/3550206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4105e20 0x7fc3e40682d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/3550206576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4068810 0x7fc3e4068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.171+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/3550206576 >> 172.21.15.31:0/3550206576 conn(0x7fc3e4076b70 msgr2=0x7fc3e4076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.172+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/3550206576 shutdown_connections 2023-10-06T21:35:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.172+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/3550206576 wait complete. 2023-10-06T21:35:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.173+0000 7fc3ea1ff700 1 Processor -- start 2023-10-06T21:35:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 -- start start 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 0x7fc3e41a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41a25a0 con 0x7fc3e4105e20 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41a2710 con 0x7fc3e41038b0 2023-10-06T21:35:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3ea1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41a28d0 con 0x7fc3e4068810 2023-10-06T21:35:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:54696/0 (socket says 172.21.15.31:54696) 2023-10-06T21:35:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3e37fe700 1 -- 172.21.15.31:0/1444318355 learned_addr learned my addr 172.21.15.31:0/1444318355 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.174+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 0x7fc3e41a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 -- 172.21.15.31:0/1444318355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 msgr2=0x7fc3e419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 -- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 msgr2=0x7fc3e41a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 0x7fc3e41a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 -- 172.21.15.31:0/1444318355 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40095f0 con 0x7fc3e4068810 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 0x7fc3e41a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.175+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005740 tx=0x7fc3d4005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.176+0000 7fc3e0ff9700 1 -- 172.21.15.31:0/1444318355 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d401d070 con 0x7fc3e4068810 2023-10-06T21:35:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.176+0000 7fc3e0ff9700 1 -- 172.21.15.31:0/1444318355 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3d4022470 con 0x7fc3e4068810 2023-10-06T21:35:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.176+0000 7fc3e0ff9700 1 -- 172.21.15.31:0/1444318355 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017650 con 0x7fc3e4068810 2023-10-06T21:35:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.176+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e41a2bb0 con 0x7fc3e4068810 2023-10-06T21:35:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.176+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e41aa3f0 con 0x7fc3e4068810 2023-10-06T21:35:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.181+0000 7fc3e0ff9700 1 -- 172.21.15.31:0/1444318355 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc3d400fb00 con 0x7fc3e4068810 2023-10-06T21:35:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.182+0000 7fc3fdc1bb80 1 Processor -- start 2023-10-06T21:35:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 -- start start 2023-10-06T21:35:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 0x557c307d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c30734400 con 0x557c307cb520 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c30725bc0 con 0x557c307d63d0 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.183+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c307234e0 con 0x557c307cac00 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:54706/0 (socket says 172.21.15.31:54706) 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 -- 172.21.15.31:0/2958078218 learned_addr learned my addr 172.21.15.31:0/2958078218 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 -- 172.21.15.31:0/2958078218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 msgr2=0x557c307d87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 0x557c307d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 -- 172.21.15.31:0/2958078218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 msgr2=0x557c307d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 0x557c307d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e3fff700 1 -- 172.21.15.31:0/2958078218 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c30549d80 con 0x557c307cac00 2023-10-06T21:35:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e0ff9700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 0x7fc3cc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:35:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.184+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 0x557c307d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc003e00 tx=0x7fc3dc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67442473b85b560e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3e0ff9700 1 -- 172.21.15.31:0/1444318355 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc3d40ce730 con 0x7fc3e4068810 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3c9ffb700 1 -- 172.21.15.31:0/2958078218 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0092d0 con 0x557c307cac00 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3c9ffb700 1 -- 172.21.15.31:0/2958078218 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3dc00f040 con 0x557c307cac00 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3c9ffb700 1 -- 172.21.15.31:0/2958078218 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc007920 con 0x557c307cac00 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.185+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 0x7fc3cc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2958078218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 msgr2=0x557c307cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3e37fe700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 0x7fc3cc0a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ce990 tx=0x7fc3d4005fd0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc003e00 tx=0x7fc3dc009920 comp rx=0 tx=0).stop 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2958078218 shutdown_connections 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cac00 0x557c307cafe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2958078218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cb520 0x557c307d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.186+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2958078218 >> 172.21.15.31:0/2958078218 conn(0x557c30740200 msgr2=0x557c307c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.187+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2958078218 shutdown_connections 2023-10-06T21:35:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.187+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2958078218 wait complete. 2023-10-06T21:35:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.187+0000 7fc3fdc1bb80 1 -- start start 2023-10-06T21:35:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 Processor -- start 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3071edd0 con 0x557c307cac00 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c306bb6b0 con 0x557c307d63d0 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c306bb350 con 0x557c307cb520 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.188+0000 7fc3fdc1bb80 10 client.0 _ll_register_callbacks cb 0x557c305187a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:54708/0 (socket says 172.21.15.31:54708) 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/2708973105 learned_addr learned my addr 172.21.15.31:0/2708973105 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 msgr2=0x557c307d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/2708973105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 msgr2=0x557c307d5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307d5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.189+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/2708973105 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc003ae0 con 0x557c307cb520 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010700 tx=0x7fc3d8010ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ed4becc8722d202 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:35:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8011970 con 0x557c307cb520 2023-10-06T21:35:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc3d8017070 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c307bed50 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.190+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d8011ad0 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 10 client.27004 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557c30532c00 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c307bed50 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc3d80233d0 con 0x557c307cb520 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 10 client.27004 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 10 client.27004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 20 client.27004 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c307e5fb0 0x557c307e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 4 client.27004 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:35:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.191+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c30717db0 con 0x557c307e5fb0 2023-10-06T21:35:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.192+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c307e5fb0 0x557c307e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.192+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c307e5fb0 0x557c307e83d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa37a3a6cceff2b server_cookie=63af53ef88a3803a in_seq=0 out_seq=0 2023-10-06T21:35:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.192+0000 7fc3c2ffd700 10 client.27004 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.193+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/2708973105 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c30717db0 con 0x557c307e5fb0 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.193+0000 7fc3c2ffd700 10 client.27004 handle_command_reply: tid=0 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.193+0000 7fc3fdc1bb80 1 client.27004 shutdown 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 msgr2=0x557c307d30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010700 tx=0x7fc3d8010ac0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 shutdown_connections 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c307d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c307e5fb0 0x557c307e83d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307d5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/2708973105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c307d30b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 >> 172.21.15.31:0/2708973105 conn(0x557c30740200 msgr2=0x557c307c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3c2ffd700 0 client.27004 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 shutdown_connections 2023-10-06T21:35:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/2708973105 wait complete. 2023-10-06T21:35:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.194+0000 7fc3fdc1bb80 20 client.27004 trim_cache size 0 max 16384 2023-10-06T21:35:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.220+0000 7fc3fdc1bb80 1 Processor -- start 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.220+0000 7fc3fdc1bb80 1 -- start start 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307d63d0 0x557c307d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3059f8c0 con 0x557c307cac00 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c307c7dc0 con 0x557c307cb520 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c307cef40 con 0x557c307d63d0 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53462/0 (socket says 172.21.15.31:53462) 2023-10-06T21:35:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/1619075360 learned_addr learned my addr 172.21.15.31:0/1619075360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307d63d0 0x557c307d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/1619075360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307d63d0 msgr2=0x557c307d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307d63d0 0x557c307d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.221+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/1619075360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 msgr2=0x557c307cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/1619075360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c305701f0 con 0x557c307cb520 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80106d0 tx=0x7fc3d800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f466405ef8a5a3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3ca7fc700 1 -- 172.21.15.31:0/1619075360 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8004020 con 0x557c307cb520 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3ca7fc700 1 -- 172.21.15.31:0/1619075360 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3d8015070 con 0x557c307cb520 2023-10-06T21:35:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.222+0000 7fc3ca7fc700 1 -- 172.21.15.31:0/1619075360 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80119d0 con 0x557c307cb520 2023-10-06T21:35:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/1619075360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 msgr2=0x557c307d5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80106d0 tx=0x7fc3d800b690 comp rx=0 tx=0).stop 2023-10-06T21:35:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/1619075360 shutdown_connections 2023-10-06T21:35:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307d63d0 0x557c307d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/1619075360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307cb520 0x557c307d5960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/1619075360 >> 172.21.15.31:0/1619075360 conn(0x557c30740200 msgr2=0x557c307ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/1619075360 shutdown_connections 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.223+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/1619075360 wait complete. 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.224+0000 7fc3fdc1bb80 1 -- start start 2023-10-06T21:35:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.224+0000 7fc3fdc1bb80 1 Processor -- start 2023-10-06T21:35:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c308322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c30835b70 con 0x557c307cac00 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c308327f0 con 0x557c307d63d0 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c308329b0 con 0x557c307cb520 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.225+0000 7fc3fdc1bb80 10 client.0 _ll_register_callbacks cb 0x7fc3d800bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:54722/0 (socket says 172.21.15.31:54722) 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/489562329 learned_addr learned my addr 172.21.15.31:0/489562329 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c308322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 msgr2=0x557c308322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c308322b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/489562329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 msgr2=0x557c307dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.226+0000 7fc3e2ffd700 1 -- 172.21.15.31:0/489562329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c30532c00 con 0x557c307cb520 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.227+0000 7fc3e2ffd700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010700 tx=0x7fc3d8023480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c6507dbe8422b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.227+0000 7fc3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.227+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8004020 con 0x557c307cb520 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.227+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc3d8017040 con 0x557c307cb520 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.227+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3dc00f040 con 0x557c307cb520 2023-10-06T21:35:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.228+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d8004180 con 0x557c307cb520 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.228+0000 7fc3fdc1bb80 10 client.27010 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.228+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557c30548250 con 0x557c307cb520 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.228+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3dc00f040 con 0x557c307cb520 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc3d802a030 con 0x557c307cb520 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 10 client.27010 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 10 client.27010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 20 client.27010 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c3084bd70 0x557c3084e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 4 client.27010 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:35:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c30717ad0 con 0x557c3084bd70 2023-10-06T21:35:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.229+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c3084bd70 0x557c3084e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.230+0000 7fc3c2ffd700 10 client.27010 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.230+0000 7fc3e3fff700 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c3084bd70 0x557c3084e190 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8faf2edbada03ce0 server_cookie=d9554a57f7b5f4db in_seq=0 out_seq=0 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.267+0000 7fc3c2ffd700 1 -- 172.21.15.31:0/489562329 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c30717ad0 con 0x557c3084bd70 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.267+0000 7fc3c2ffd700 10 client.27010 handle_command_reply: tid=0 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.267+0000 7fc3fdc1bb80 1 client.27010 shutdown 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.267+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 msgr2=0x557c30831d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.267+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010700 tx=0x7fc3d8023480 comp rx=0 tx=0).stop 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 shutdown_connections 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c307d63d0 0x557c308322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557c3084bd70 0x557c3084e190 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c307cac00 0x557c307dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 --2- 172.21.15.31:0/489562329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557c307cb520 0x557c30831d70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 >> 172.21.15.31:0/489562329 conn(0x557c30740200 msgr2=0x557c307c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3c2ffd700 0 client.27010 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 shutdown_connections 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 1 -- 172.21.15.31:0/489562329 wait complete. 2023-10-06T21:35:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.268+0000 7fc3fdc1bb80 20 client.27010 trim_cache size 0 max 16384 2023-10-06T21:35:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:35:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:35:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:35:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:35:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 msgr2=0x7fc3cc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 0x7fc3cc0a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ce990 tx=0x7fc3d4005fd0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 msgr2=0x7fc3e419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005740 tx=0x7fc3d4005770 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 shutdown_connections 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc3cc0a1670 0x7fc3cc0a3b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e4105e20 0x7fc3e41a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e4068810 0x7fc3e419d3b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 --2- 172.21.15.31:0/1444318355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e41038b0 0x7fc3e419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 >> 172.21.15.31:0/1444318355 conn(0x7fc3e4076b70 msgr2=0x7fc3e40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 shutdown_connections 2023-10-06T21:35:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:34.269+0000 7fc3ea1ff700 1 -- 172.21.15.31:0/1444318355 wait complete. 2023-10-06T21:35:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:34 smithi031 ceph-mon[98539]: pgmap v460: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2023-10-06T21:35:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:34 smithi146 ceph-mon[102809]: pgmap v460: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2023-10-06T21:35:34.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:34 smithi142 ceph-mon[102951]: pgmap v460: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2023-10-06T21:35:34.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10211 8.72 MB/sec warmup 83 sec latency 80.977 ms 2023-10-06T21:35:35.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10372 8.78 MB/sec warmup 84 sec latency 67.309 ms 2023-10-06T21:35:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:36 smithi031 ceph-mon[98539]: pgmap v461: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:36 smithi146 ceph-mon[102809]: pgmap v461: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:36.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:36 smithi142 ceph-mon[102951]: pgmap v461: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:36.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10509 8.80 MB/sec warmup 85 sec latency 67.153 ms 2023-10-06T21:35:37.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10615 8.77 MB/sec warmup 86 sec latency 64.444 ms 2023-10-06T21:35:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:38 smithi031 ceph-mon[98539]: pgmap v462: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:38 smithi146 ceph-mon[102809]: pgmap v462: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:38 smithi142 ceph-mon[102951]: pgmap v462: 97 pgs: 97 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.8 MiB/s wr, 157 op/s 2023-10-06T21:35:38.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10730 8.76 MB/sec warmup 87 sec latency 65.602 ms 2023-10-06T21:35:39.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10842 8.74 MB/sec warmup 88 sec latency 66.918 ms 2023-10-06T21:35:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:40 smithi142 ceph-mon[102951]: pgmap v463: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2023-10-06T21:35:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:40 smithi031 ceph-mon[98539]: pgmap v463: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2023-10-06T21:35:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:40 smithi146 ceph-mon[102809]: pgmap v463: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2023-10-06T21:35:40.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 10943 8.66 MB/sec warmup 89 sec latency 73.528 ms 2023-10-06T21:35:41.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11046 8.60 MB/sec warmup 90 sec latency 66.926 ms 2023-10-06T21:35:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:42 smithi031 ceph-mon[98539]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.7 MiB/s wr, 167 op/s 2023-10-06T21:35:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:42 smithi146 ceph-mon[102809]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.7 MiB/s wr, 167 op/s 2023-10-06T21:35:42.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:42 smithi142 ceph-mon[102951]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.7 MiB/s wr, 167 op/s 2023-10-06T21:35:42.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11161 8.56 MB/sec warmup 91 sec latency 74.618 ms 2023-10-06T21:35:43.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11272 8.50 MB/sec warmup 92 sec latency 70.990 ms 2023-10-06T21:35:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:44 smithi031 ceph-mon[98539]: pgmap v465: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2023-10-06T21:35:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:44 smithi146 ceph-mon[102809]: pgmap v465: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2023-10-06T21:35:44.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:44 smithi142 ceph-mon[102951]: pgmap v465: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2023-10-06T21:35:44.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11393 8.49 MB/sec warmup 93 sec latency 69.586 ms 2023-10-06T21:35:45.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11509 8.45 MB/sec warmup 94 sec latency 63.119 ms 2023-10-06T21:35:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:46 smithi031 ceph-mon[98539]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:46 smithi146 ceph-mon[102809]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:46.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:46 smithi142 ceph-mon[102951]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:46.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11648 8.47 MB/sec warmup 95 sec latency 75.041 ms 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1557894370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 msgr2=0x7eff14107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1557894370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff14107340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff10009990 tx=0x7eff10009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1557894370 shutdown_connections 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1557894370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 0x7eff1410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1557894370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff14107340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1557894370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 0x7eff14068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.502+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1557894370 >> 172.21.15.31:0/1557894370 conn(0x7eff14076b70 msgr2=0x7eff14076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.503+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1557894370 shutdown_connections 2023-10-06T21:35:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.503+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1557894370 wait complete. 2023-10-06T21:35:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.504+0000 7eff1c7ff700 1 Processor -- start 2023-10-06T21:35:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.504+0000 7eff1c7ff700 1 -- start start 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 0x7eff14196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 0x7eff14197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1419c130 con 0x7eff14068c90 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1419c2a0 con 0x7eff14106f60 2023-10-06T21:35:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1c7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1419c460 con 0x7eff140682d0 2023-10-06T21:35:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.505+0000 7eff1ad9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:36248/0 (socket says 172.21.15.31:36248) 2023-10-06T21:35:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 -- 172.21.15.31:0/1838345602 learned_addr learned my addr 172.21.15.31:0/1838345602 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1a59b700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 0x7eff14196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff19d9a700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 0x7eff14197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 -- 172.21.15.31:0/1838345602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 msgr2=0x7eff14196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 0x7eff14196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 -- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 msgr2=0x7eff14197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 0x7eff14197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.506+0000 7eff1ad9c700 1 -- 172.21.15.31:0/1838345602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009520 con 0x7eff14106f60 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00efc0 tx=0x7eff0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff0b7fe700 1 -- 172.21.15.31:0/1838345602 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c004bb0 con 0x7eff14106f60 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff0b7fe700 1 -- 172.21.15.31:0/1838345602 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff0c00f040 con 0x7eff14106f60 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff0b7fe700 1 -- 172.21.15.31:0/1838345602 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00ceb0 con 0x7eff14106f60 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff100095f0 con 0x7eff14106f60 2023-10-06T21:35:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.507+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff140fff70 con 0x7eff14106f60 2023-10-06T21:35:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.512+0000 7eff0b7fe700 1 -- 172.21.15.31:0/1838345602 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7eff0c0135c0 con 0x7eff14106f60 2023-10-06T21:35:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 Processor -- start 2023-10-06T21:35:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 -- start start 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d318377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d3184da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31798400 con 0x564d3184b650 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31789bc0 con 0x564d31837410 2023-10-06T21:35:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.513+0000 7eff301a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d317874e0 con 0x564d31837d30 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.514+0000 7eff1a59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.514+0000 7eff1a59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44144/0 (socket says 172.21.15.31:44144) 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.514+0000 7eff1a59b700 1 -- 172.21.15.31:0/3143518359 learned_addr learned my addr 172.21.15.31:0/3143518359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff1a59b700 1 -- 172.21.15.31:0/3143518359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 msgr2=0x564d318377f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff1a59b700 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d318377f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff1a59b700 1 -- 172.21.15.31:0/3143518359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 msgr2=0x564d3184da40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:35:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff0b7fe700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 0x7eff000a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff1a59b700 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d3184da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff1a59b700 1 -- 172.21.15.31:0/3143518359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d315add80 con 0x564d31837d30 2023-10-06T21:35:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.515+0000 7eff0b7fe700 1 -- 172.21.15.31:0/1838345602 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7eff0c0d0680 con 0x7eff14106f60 2023-10-06T21:35:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.516+0000 7eff1a59b700 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff10015000 tx=0x7eff10005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4202c0ccfd75ec91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.516+0000 7eff08ff9700 1 -- 172.21.15.31:0/3143518359 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001d070 con 0x564d31837d30 2023-10-06T21:35:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.516+0000 7eff08ff9700 1 -- 172.21.15.31:0/3143518359 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1001e040 con 0x564d31837d30 2023-10-06T21:35:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.516+0000 7eff08ff9700 1 -- 172.21.15.31:0/3143518359 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000b710 con 0x564d31837d30 2023-10-06T21:35:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.516+0000 7eff19d9a700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 0x7eff000a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 -- 172.21.15.31:0/3143518359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 msgr2=0x564d31834be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff10015000 tx=0x7eff10005e20 comp rx=0 tx=0).stop 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 -- 172.21.15.31:0/3143518359 shutdown_connections 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d3184da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d318377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 --2- 172.21.15.31:0/3143518359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d31834be0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 -- 172.21.15.31:0/3143518359 >> 172.21.15.31:0/3143518359 conn(0x564d317a4200 msgr2=0x564d3182aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 -- 172.21.15.31:0/3143518359 shutdown_connections 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff19d9a700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 0x7eff000a3bb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff14198760 tx=0x7eff04009970 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.517+0000 7eff301a0b80 1 -- 172.21.15.31:0/3143518359 wait complete. 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.518+0000 7eff301a0b80 1 -- start start 2023-10-06T21:35:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.518+0000 7eff301a0b80 1 Processor -- start 2023-10-06T21:35:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d3184ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d31856ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31782dd0 con 0x564d3184b650 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3171f6b0 con 0x564d31837410 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3171f350 con 0x564d31837d30 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.519+0000 7eff301a0b80 10 client.0 _ll_register_callbacks cb 0x564d3157c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44158/0 (socket says 172.21.15.31:44158) 2023-10-06T21:35:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 -- 172.21.15.31:0/453937598 learned_addr learned my addr 172.21.15.31:0/453937598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff1a59b700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d3184ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 -- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 msgr2=0x564d3184ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d3184ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 -- 172.21.15.31:0/453937598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 msgr2=0x564d31856ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d31856ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.520+0000 7eff19d9a700 1 -- 172.21.15.31:0/453937598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1410e5c0 con 0x564d31837d30 2023-10-06T21:35:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.521+0000 7eff1a59b700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d3184ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.521+0000 7eff19d9a700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff04005950 tx=0x7eff04009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a79ad836af8dc82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.521+0000 7efefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:35:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.521+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400f850 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.522+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7eff0400f9b0 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.522+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d31822d50 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.522+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff0400fb70 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.522+0000 7eff301a0b80 10 client.27022 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.522+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x564d31596c00 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564d31822d50 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7eff04027420 con 0x564d31837d30 2023-10-06T21:35:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 20 client.27022 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d3185b010 0x564d3185d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 4 client.27022 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:35:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.523+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3177bdb0 con 0x564d3185b010 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.524+0000 7eff1a59b700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d3185b010 0x564d3185d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.525+0000 7eff1a59b700 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d3185b010 0x564d3185d430 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ba4d1e4954b275 server_cookie=33f55a0611fdd054 in_seq=0 out_seq=0 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.525+0000 7efefdffb700 10 client.27022 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.526+0000 7efefdffb700 1 -- 172.21.15.31:0/453937598 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d3177bdb0 con 0x564d3185b010 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.526+0000 7efefdffb700 10 client.27022 handle_command_reply: tid=0 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.526+0000 7eff301a0b80 1 client.27022 shutdown 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.526+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 msgr2=0x564d3184bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.526+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff04005950 tx=0x7eff04009260 comp rx=0 tx=0).stop 2023-10-06T21:35:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 shutdown_connections 2023-10-06T21:35:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d3185b010 0x564d3185d430 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d3184b650 0x564d31856ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837410 0x564d3184ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 --2- 172.21.15.31:0/453937598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d31837d30 0x564d3184bf70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 >> 172.21.15.31:0/453937598 conn(0x564d317a4200 msgr2=0x564d3182ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7efefdffb700 0 client.27022 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 shutdown_connections 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.527+0000 7eff301a0b80 1 -- 172.21.15.31:0/453937598 wait complete. 2023-10-06T21:35:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.528+0000 7eff301a0b80 20 client.27022 trim_cache size 0 max 16384 2023-10-06T21:35:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.553+0000 7eff301a0b80 1 Processor -- start 2023-10-06T21:35:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.553+0000 7eff301a0b80 1 -- start start 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.553+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31850a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.553+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.553+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d3182f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff301a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d316038c0 con 0x564d31837410 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff301a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31850f70 con 0x564d31837d30 2023-10-06T21:35:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff301a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31832830 con 0x564d3184b650 2023-10-06T21:35:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff19d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff19d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:36282/0 (socket says 172.21.15.31:36282) 2023-10-06T21:35:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff19d9a700 1 -- 172.21.15.31:0/938828854 learned_addr learned my addr 172.21.15.31:0/938828854 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.554+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d3182f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff1a59b700 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31850a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 -- 172.21.15.31:0/938828854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 msgr2=0x564d3182f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d3182f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 -- 172.21.15.31:0/938828854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 msgr2=0x564d31850a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31850a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 -- 172.21.15.31:0/938828854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d315d41f0 con 0x564d31837d30 2023-10-06T21:35:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.555+0000 7eff19d9a700 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff04027800 tx=0x7eff04009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d16b28445fbdb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.556+0000 7eff097fa700 1 -- 172.21.15.31:0/938828854 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04027420 con 0x564d31837d30 2023-10-06T21:35:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.556+0000 7eff097fa700 1 -- 172.21.15.31:0/938828854 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff04027580 con 0x564d31837d30 2023-10-06T21:35:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.556+0000 7eff097fa700 1 -- 172.21.15.31:0/938828854 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04039680 con 0x564d31837d30 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 -- 172.21.15.31:0/938828854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 msgr2=0x564d31833a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff04027800 tx=0x7eff04009c30 comp rx=0 tx=0).stop 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 -- 172.21.15.31:0/938828854 shutdown_connections 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d3182f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31850a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 --2- 172.21.15.31:0/938828854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31833a90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 -- 172.21.15.31:0/938828854 >> 172.21.15.31:0/938828854 conn(0x564d317a4200 msgr2=0x564d3184ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 -- 172.21.15.31:0/938828854 shutdown_connections 2023-10-06T21:35:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.557+0000 7eff301a0b80 1 -- 172.21.15.31:0/938828854 wait complete. 2023-10-06T21:35:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.558+0000 7eff301a0b80 1 -- start start 2023-10-06T21:35:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.558+0000 7eff301a0b80 1 Processor -- start 2023-10-06T21:35:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31830a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d31830fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31831510 con 0x564d31837410 2023-10-06T21:35:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31831680 con 0x564d31837d30 2023-10-06T21:35:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d31831840 con 0x564d3184b650 2023-10-06T21:35:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.559+0000 7eff301a0b80 10 client.0 _ll_register_callbacks cb 0x7eff04012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff19d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31830a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42406/0 (socket says 172.21.15.31:42406) 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 -- 172.21.15.31:0/329655113 learned_addr learned my addr 172.21.15.31:0/329655113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d31830fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 -- 172.21.15.31:0/329655113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 msgr2=0x564d31830fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d31830fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 -- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 msgr2=0x564d31830a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31830a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1a59b700 1 -- 172.21.15.31:0/329655113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d31596c00 con 0x564d31837410 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff1ad9c700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d31830fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.560+0000 7eff19d9a700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31830a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.561+0000 7eff1a59b700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7eff100098e0 tx=0x7eff10003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a805e8339a7535 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.561+0000 7efefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:35:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.561+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001d040 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.561+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7eff10031040 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.561+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff1001e040 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff10003a00 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7eff301a0b80 10 client.17385 fetch_fsmap learned FSMap version 42 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x564d315ac250 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1001e040 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 42) v1 ==== 7922+0+0 (secure 0 0 0) 0x7eff100053d0 con 0x564d31837410 2023-10-06T21:35:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7eff301a0b80 10 client.17385 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7eff301a0b80 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.562+0000 7eff301a0b80 20 client.17385 populate_metadata read hostname 'smithi031' 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.563+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d318af450 0x564d318b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.563+0000 7eff301a0b80 4 client.17385 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.563+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564d3177bad0 con 0x564d318af450 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.563+0000 7eff19d9a700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d318af450 0x564d318b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.564+0000 7eff19d9a700 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d318af450 0x564d318b1870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab2edf587813cc1 server_cookie=b8f83ec67fe483cd in_seq=0 out_seq=0 2023-10-06T21:35:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.564+0000 7efefdffb700 10 client.17385 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.708+0000 7efefdffb700 1 -- 172.21.15.31:0/329655113 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564d3177bad0 con 0x564d318af450 2023-10-06T21:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.708+0000 7efefdffb700 10 client.17385 handle_command_reply: tid=0 2023-10-06T21:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.708+0000 7eff301a0b80 1 client.17385 shutdown 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.708+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 msgr2=0x564d31830550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.708+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7eff100098e0 tx=0x7eff10003ea0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 shutdown_connections 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d3184b650 0x564d31830fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564d318af450 0x564d318b1870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564d31837410 0x564d31830550 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 --2- 172.21.15.31:0/329655113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d31837d30 0x564d31830a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 >> 172.21.15.31:0/329655113 conn(0x564d317a4200 msgr2=0x564d3182f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7efefdffb700 0 client.17385 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 shutdown_connections 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 1 -- 172.21.15.31:0/329655113 wait complete. 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.709+0000 7eff301a0b80 20 client.17385 trim_cache size 0 max 16384 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:35:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 msgr2=0x7eff000a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 0x7eff000a3bb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff14198760 tx=0x7eff04009970 comp rx=0 tx=0).stop 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 msgr2=0x7eff1419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00efc0 tx=0x7eff0c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 shutdown_connections 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff14106f60 0x7eff1419baf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff140682d0 0x7eff14196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7eff000a16f0 0x7eff000a3bb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 --2- 172.21.15.31:0/1838345602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff14068c90 0x7eff14197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.710+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 >> 172.21.15.31:0/1838345602 conn(0x7eff14076b70 msgr2=0x7eff140fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.712+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 shutdown_connections 2023-10-06T21:35:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:35:47.712+0000 7eff1c7ff700 1 -- 172.21.15.31:0/1838345602 wait complete. 2023-10-06T21:35:47.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11761 8.45 MB/sec warmup 96 sec latency 204.626 ms 2023-10-06T21:35:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:48 smithi031 ceph-mon[98539]: pgmap v467: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:48 smithi146 ceph-mon[102809]: pgmap v467: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:48.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:48 smithi142 ceph-mon[102951]: pgmap v467: 97 pgs: 97 active+clean; 419 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 178 op/s 2023-10-06T21:35:48.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11874 8.42 MB/sec warmup 97 sec latency 75.432 ms 2023-10-06T21:35:49.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 11989 8.35 MB/sec warmup 98 sec latency 68.839 ms 2023-10-06T21:35:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:50 smithi031 ceph-mon[98539]: pgmap v468: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2023-10-06T21:35:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:50 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:35:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:50 smithi146 ceph-mon[102809]: pgmap v468: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2023-10-06T21:35:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:50 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:35:50.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:50 smithi142 ceph-mon[102951]: pgmap v468: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2023-10-06T21:35:50.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:50 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:35:50.756 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12119 8.31 MB/sec warmup 99 sec latency 74.200 ms 2023-10-06T21:35:51.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12240 8.24 MB/sec warmup 100 sec latency 70.302 ms 2023-10-06T21:35:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:52 smithi031 ceph-mon[98539]: pgmap v469: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-10-06T21:35:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:52 smithi146 ceph-mon[102809]: pgmap v469: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-10-06T21:35:52.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:52 smithi142 ceph-mon[102951]: pgmap v469: 97 pgs: 97 active+clean; 405 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2023-10-06T21:35:52.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12365 8.20 MB/sec warmup 101 sec latency 74.624 ms 2023-10-06T21:35:53.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12493 8.17 MB/sec warmup 102 sec latency 70.844 ms 2023-10-06T21:35:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:54 smithi031 ceph-mon[98539]: pgmap v470: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-10-06T21:35:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:54 smithi146 ceph-mon[102809]: pgmap v470: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-10-06T21:35:54.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:54 smithi142 ceph-mon[102951]: pgmap v470: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-10-06T21:35:54.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12581 8.12 MB/sec warmup 103 sec latency 460.535 ms 2023-10-06T21:35:55.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12688 8.08 MB/sec warmup 104 sec latency 140.551 ms 2023-10-06T21:35:56.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:56 smithi031 ceph-mon[98539]: pgmap v471: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:56 smithi146 ceph-mon[102809]: pgmap v471: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:56 smithi142 ceph-mon[102951]: pgmap v471: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:56.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12789 8.05 MB/sec warmup 105 sec latency 317.016 ms 2023-10-06T21:35:57.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 12943 8.11 MB/sec warmup 106 sec latency 75.340 ms 2023-10-06T21:35:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:35:58 smithi031 ceph-mon[98539]: pgmap v472: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:35:58 smithi146 ceph-mon[102809]: pgmap v472: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:35:58 smithi142 ceph-mon[102951]: pgmap v472: 97 pgs: 97 active+clean; 404 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-10-06T21:35:58.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13107 8.14 MB/sec warmup 107 sec latency 71.248 ms 2023-10-06T21:35:59.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13301 8.20 MB/sec warmup 108 sec latency 72.207 ms 2023-10-06T21:36:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:00 smithi031 ceph-mon[98539]: pgmap v473: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-10-06T21:36:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:00 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:00 smithi146 ceph-mon[102809]: pgmap v473: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-10-06T21:36:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:00 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:00 smithi142 ceph-mon[102951]: pgmap v473: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-10-06T21:36:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:00 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:00.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13425 8.19 MB/sec warmup 109 sec latency 118.501 ms 2023-10-06T21:36:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 -- 172.21.15.31:0/1571564828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c073250 msgr2=0x7fb56c073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 --2- 172.21.15.31:0/1571564828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c073250 0x7fb56c073630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5680098f0 tx=0x7fb568009c00 comp rx=0 tx=0).stop 2023-10-06T21:36:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 -- 172.21.15.31:0/1571564828 shutdown_connections 2023-10-06T21:36:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 --2- 172.21.15.31:0/1571564828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c111ce0 0x7fb56c1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 --2- 172.21.15.31:0/1571564828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c073250 0x7fb56c073630 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 --2- 172.21.15.31:0/1571564828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 0x7fb56c1117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 -- 172.21.15.31:0/1571564828 >> 172.21.15.31:0/1571564828 conn(0x7fb56c0fc7c0 msgr2=0x7fb56c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 -- 172.21.15.31:0/1571564828 shutdown_connections 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.934+0000 7fb571b9d700 1 -- 172.21.15.31:0/1571564828 wait complete. 2023-10-06T21:36:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 Processor -- start 2023-10-06T21:36:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 -- start start 2023-10-06T21:36:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 0x7fb56c10ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 0x7fb56c10d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c1a0f60 con 0x7fb56c073250 2023-10-06T21:36:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c1a10a0 con 0x7fb56c073b70 2023-10-06T21:36:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.936+0000 7fb571b9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c1a11e0 con 0x7fb56c111ce0 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb570b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 0x7fb56c10ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb57139c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb57139c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51884/0 (socket says 172.21.15.31:51884) 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb57139c700 1 -- 172.21.15.31:0/2981679660 learned_addr learned my addr 172.21.15.31:0/2981679660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb563fff700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 0x7fb56c10d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb570b9b700 1 -- 172.21.15.31:0/2981679660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 msgr2=0x7fb56c1a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb570b9b700 1 -- 172.21.15.31:0/2981679660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 msgr2=0x7fb56c10d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.937+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 0x7fb56c10d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.938+0000 7fb570b9b700 1 -- 172.21.15.31:0/2981679660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c01a600 con 0x7fb56c073250 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.938+0000 7fb57139c700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.938+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 0x7fb56c10ccc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb568009810 tx=0x7fb56800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.938+0000 7fb561ffb700 1 -- 172.21.15.31:0/2981679660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56801c070 con 0x7fb56c073250 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.939+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb568009520 con 0x7fb56c073250 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.939+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1a16d0 con 0x7fb56c073250 2023-10-06T21:36:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.939+0000 7fb561ffb700 1 -- 172.21.15.31:0/2981679660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb568005040 con 0x7fb56c073250 2023-10-06T21:36:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.939+0000 7fb561ffb700 1 -- 172.21.15.31:0/2981679660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568003970 con 0x7fb56c073250 2023-10-06T21:36:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.943+0000 7fb561ffb700 1 -- 172.21.15.31:0/2981679660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb56800b7f0 con 0x7fb56c073250 2023-10-06T21:36:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.944+0000 7fb5866d9b80 1 Processor -- start 2023-10-06T21:36:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.944+0000 7fb5866d9b80 1 -- start start 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de1d60 0x55e083de2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de2680 0x55e083decff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083d4b400 con 0x55e083ded530 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083d3cbc0 con 0x55e083de2680 2023-10-06T21:36:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.945+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083d3a4e0 con 0x55e083de1d60 2023-10-06T21:36:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43430/0 (socket says 172.21.15.31:43430) 2023-10-06T21:36:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 -- 172.21.15.31:0/1253130329 learned_addr learned my addr 172.21.15.31:0/1253130329 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb561ffb700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 0x7fb5540a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb561ffb700 1 -- 172.21.15.31:0/2981679660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb5680cdc00 con 0x7fb56c073250 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb57139c700 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de1d60 0x55e083de2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 -- 172.21.15.31:0/1253130329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de1d60 msgr2=0x55e083de2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de1d60 0x55e083de2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 -- 172.21.15.31:0/1253130329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de2680 msgr2=0x55e083decff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de2680 0x55e083decff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb563fff700 1 -- 172.21.15.31:0/1253130329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e083b60d80 con 0x55e083ded530 2023-10-06T21:36:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.946+0000 7fb57139c700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 0x7fb5540a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.947+0000 7fb563fff700 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb55c01af60 tx=0x7fb55c01af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d4e7f0d03cb82d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.947+0000 7fb552ffd700 1 -- 172.21.15.31:0/1253130329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004110 con 0x55e083ded530 2023-10-06T21:36:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.947+0000 7fb552ffd700 1 -- 172.21.15.31:0/1253130329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55c004830 con 0x55e083ded530 2023-10-06T21:36:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.947+0000 7fb552ffd700 1 -- 172.21.15.31:0/1253130329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0053e0 con 0x55e083ded530 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb57139c700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 0x7fb5540a3bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5640075c0 tx=0x7fb564004ad0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1253130329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 msgr2=0x55e083def920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb55c01af60 tx=0x7fb55c01af90 comp rx=0 tx=0).stop 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1253130329 shutdown_connections 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083def920 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de1d60 0x55e083de2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1253130329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de2680 0x55e083decff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1253130329 >> 172.21.15.31:0/1253130329 conn(0x55e083d57200 msgr2=0x55e083dde1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1253130329 shutdown_connections 2023-10-06T21:36:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.948+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1253130329 wait complete. 2023-10-06T21:36:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.949+0000 7fb5866d9b80 1 -- start start 2023-10-06T21:36:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.949+0000 7fb5866d9b80 1 Processor -- start 2023-10-06T21:36:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083dea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083deac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083d35dd0 con 0x55e083de2680 2023-10-06T21:36:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083cd26b0 con 0x55e083de1d60 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083cd2350 con 0x55e083ded530 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb5866d9b80 10 client.0 _ll_register_callbacks cb 0x55e083b2f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb570b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb570b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57832/0 (socket says 172.21.15.31:57832) 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.950+0000 7fb570b9b700 1 -- 172.21.15.31:0/2520166125 learned_addr learned my addr 172.21.15.31:0/2520166125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb57139c700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083deac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 -- 172.21.15.31:0/2520166125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 msgr2=0x55e083deac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083deac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 -- 172.21.15.31:0/2520166125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 msgr2=0x55e083dea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083dea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 -- 172.21.15.31:0/2520166125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c01ab90 con 0x55e083de1d60 2023-10-06T21:36:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb568097610 tx=0x7fb56809a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67fba8b1d63d591 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.951+0000 7fb54bfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56809aba0 con 0x55e083de1d60 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb56809ecf0 con 0x55e083de1d60 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e083dd5eb0 con 0x55e083de1d60 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5680a2ea0 con 0x55e083de1d60 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb5866d9b80 10 client.27089 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.952+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e083b49c00 con 0x55e083de1d60 2023-10-06T21:36:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e083dd5eb0 con 0x55e083de1d60 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb5680a2a10 con 0x55e083de1d60 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 10 client.27089 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 10 client.27089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 20 client.27089 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e0de40 0x55e083e10260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 4 client.27089 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e083d2edb0 con 0x55e083e0de40 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.953+0000 7fb563fff700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e0de40 0x55e083e10260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.954+0000 7fb563fff700 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e0de40 0x55e083e10260 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7ffefba87e1976 server_cookie=b322636ae0d342de in_seq=0 out_seq=0 2023-10-06T21:36:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.954+0000 7fb54bfff700 10 client.27089 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb54bfff700 1 -- 172.21.15.31:0/2520166125 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e083d2edb0 con 0x55e083e0de40 2023-10-06T21:36:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb54bfff700 10 client.27089 handle_command_reply: tid=0 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 client.27089 shutdown 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 msgr2=0x55e083dea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb568097610 tx=0x7fb56809a500 comp rx=0 tx=0).stop 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 shutdown_connections 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083deac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083dea210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e0de40 0x55e083e10260 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2520166125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083dea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 >> 172.21.15.31:0/2520166125 conn(0x55e083d57200 msgr2=0x55e083dddd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.955+0000 7fb54bfff700 0 client.27089 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.956+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 shutdown_connections 2023-10-06T21:36:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.956+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2520166125 wait complete. 2023-10-06T21:36:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.956+0000 7fb5866d9b80 20 client.27089 trim_cache size 0 max 16384 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.984+0000 7fb5866d9b80 1 Processor -- start 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 -- start start 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083de7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083bb68c0 con 0x55e083ded530 2023-10-06T21:36:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083df4610 con 0x55e083de1d60 2023-10-06T21:36:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083ddcf90 con 0x55e083de2680 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb57139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083de7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.985+0000 7fb563fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51920/0 (socket says 172.21.15.31:51920) 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 -- 172.21.15.31:0/2908600685 learned_addr learned my addr 172.21.15.31:0/2908600685 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 -- 172.21.15.31:0/2908600685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 msgr2=0x55e083df40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 -- 172.21.15.31:0/2908600685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 msgr2=0x55e083de7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083de7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 -- 172.21.15.31:0/2908600685 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e083b871f0 con 0x55e083de2680 2023-10-06T21:36:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb570b9b700 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.986+0000 7fb563fff700 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb55c01aa70 tx=0x7fb55c01aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d130055a0aa20114 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.987+0000 7fb5537fe700 1 -- 172.21.15.31:0/2908600685 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004ac0 con 0x55e083de2680 2023-10-06T21:36:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.987+0000 7fb5537fe700 1 -- 172.21.15.31:0/2908600685 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55c004c20 con 0x55e083de2680 2023-10-06T21:36:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.987+0000 7fb5537fe700 1 -- 172.21.15.31:0/2908600685 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037530 con 0x55e083de2680 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2908600685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 msgr2=0x55e083de99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb55c01aa70 tx=0x7fb55c01aa00 comp rx=0 tx=0).stop 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2908600685 shutdown_connections 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083ded530 0x55e083de7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/2908600685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083de2680 0x55e083de99d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2908600685 >> 172.21.15.31:0/2908600685 conn(0x55e083d57200 msgr2=0x55e083de54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2908600685 shutdown_connections 2023-10-06T21:36:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.988+0000 7fb5866d9b80 1 -- 172.21.15.31:0/2908600685 wait complete. 2023-10-06T21:36:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.989+0000 7fb5866d9b80 1 -- start start 2023-10-06T21:36:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.989+0000 7fb5866d9b80 1 Processor -- start 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083df1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083df2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083e60d30 con 0x55e083de2680 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083e60ea0 con 0x55e083de1d60 2023-10-06T21:36:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083e61060 con 0x55e083ded530 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb5866d9b80 10 client.0 _ll_register_callbacks cb 0x55e083b2f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb570b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb570b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57844/0 (socket says 172.21.15.31:57844) 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb570b9b700 1 -- 172.21.15.31:0/1511180082 learned_addr learned my addr 172.21.15.31:0/1511180082 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.990+0000 7fb563fff700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083df1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb57139c700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083df2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 -- 172.21.15.31:0/1511180082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 msgr2=0x55e083df2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083df2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 -- 172.21.15.31:0/1511180082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 msgr2=0x55e083df1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083df1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 -- 172.21.15.31:0/1511180082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c02e040 con 0x55e083de1d60 2023-10-06T21:36:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb570b9b700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5680a2d70 tx=0x7fb56809a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81980f844613b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb54bfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56809aba0 con 0x55e083de1d60 2023-10-06T21:36:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb55c004830 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.991+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fb5680a2ea0 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb568003c90 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 10 client.27095 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e083b5f250 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5680a5080 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb568099d10 con 0x55e083de1d60 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 10 client.27095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 20 client.27095 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e622d0 0x55e083e646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 4 client.27095 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.992+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e083d2ead0 con 0x55e083e622d0 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.993+0000 7fb563fff700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e622d0 0x55e083e646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.993+0000 7fb563fff700 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e622d0 0x55e083e646c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ecd3173e7eab0 server_cookie=6ea8b4b7d1401ef1 in_seq=0 out_seq=0 2023-10-06T21:36:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:00.993+0000 7fb54bfff700 10 client.27095 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb54bfff700 1 -- 172.21.15.31:0/1511180082 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e083d2ead0 con 0x55e083e622d0 2023-10-06T21:36:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb54bfff700 10 client.27095 handle_command_reply: tid=0 2023-10-06T21:36:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 client.27095 shutdown 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 msgr2=0x55e083df16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5680a2d70 tx=0x7fb56809a410 comp rx=0 tx=0).stop 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 shutdown_connections 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e083ded530 0x55e083df2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e083de1d60 0x55e083df16c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e083e622d0 0x55e083e646c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 --2- 172.21.15.31:0/1511180082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e083de2680 0x55e083df1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 >> 172.21.15.31:0/1511180082 conn(0x55e083d57200 msgr2=0x55e083de3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.041+0000 7fb54bfff700 0 client.27095 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.042+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 shutdown_connections 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.042+0000 7fb5866d9b80 1 -- 172.21.15.31:0/1511180082 wait complete. 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.042+0000 7fb5866d9b80 20 client.27095 trim_cache size 0 max 16384 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:36:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 msgr2=0x7fb5540a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 0x7fb5540a3bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5640075c0 tx=0x7fb564004ad0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 msgr2=0x7fb56c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 0x7fb56c10ccc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb568009810 tx=0x7fb56800fa00 comp rx=0 tx=0).stop 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 shutdown_connections 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb56c111ce0 0x7fb56c1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fb5540a16f0 0x7fb5540a3bb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb56c073250 0x7fb56c10ccc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 --2- 172.21.15.31:0/2981679660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb56c073b70 0x7fb56c10d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 >> 172.21.15.31:0/2981679660 conn(0x7fb56c0fc7c0 msgr2=0x7fb56c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.043+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 shutdown_connections 2023-10-06T21:36:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:01.044+0000 7fb571b9d700 1 -- 172.21.15.31:0/2981679660 wait complete. 2023-10-06T21:36:01.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13513 8.15 MB/sec warmup 110 sec latency 102.006 ms 2023-10-06T21:36:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:02 smithi031 ceph-mon[98539]: pgmap v474: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-10-06T21:36:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:02 smithi146 ceph-mon[102809]: pgmap v474: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-10-06T21:36:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:02 smithi142 ceph-mon[102951]: pgmap v474: 97 pgs: 97 active+clean; 410 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-10-06T21:36:02.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13590 8.13 MB/sec warmup 111 sec latency 300.245 ms 2023-10-06T21:36:03.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13728 8.17 MB/sec warmup 112 sec latency 346.313 ms 2023-10-06T21:36:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:04 smithi031 ceph-mon[98539]: pgmap v475: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-10-06T21:36:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:04 smithi146 ceph-mon[102809]: pgmap v475: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-10-06T21:36:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:04 smithi142 ceph-mon[102951]: pgmap v475: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-10-06T21:36:04.757 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13843 8.19 MB/sec warmup 113 sec latency 391.666 ms 2023-10-06T21:36:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:05 smithi031 ceph-mon[98539]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:36:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:05 smithi146 ceph-mon[102809]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:36:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:05 smithi142 ceph-mon[102951]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:36:05.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 13964 8.18 MB/sec warmup 114 sec latency 67.025 ms 2023-10-06T21:36:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:06 smithi031 ceph-mon[98539]: pgmap v477: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:06 smithi146 ceph-mon[102809]: pgmap v477: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:06 smithi142 ceph-mon[102951]: pgmap v477: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:06.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14057 8.15 MB/sec warmup 115 sec latency 379.178 ms 2023-10-06T21:36:07.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14143 8.11 MB/sec warmup 116 sec latency 65.476 ms 2023-10-06T21:36:08.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:08 smithi031 ceph-mon[98539]: pgmap v478: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:08 smithi146 ceph-mon[102809]: pgmap v478: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:08.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:08 smithi142 ceph-mon[102951]: pgmap v478: 97 pgs: 97 active+clean; 424 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-06T21:36:08.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14252 8.11 MB/sec warmup 117 sec latency 65.403 ms 2023-10-06T21:36:09.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14355 8.06 MB/sec warmup 118 sec latency 68.387 ms 2023-10-06T21:36:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:10 smithi142 ceph-mon[102951]: pgmap v479: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:10 smithi031 ceph-mon[98539]: pgmap v479: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:10 smithi146 ceph-mon[102809]: pgmap v479: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:10.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14477 8.04 MB/sec warmup 119 sec latency 63.768 ms 2023-10-06T21:36:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:12 smithi031 ceph-mon[98539]: pgmap v480: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:12 smithi146 ceph-mon[102809]: pgmap v480: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:12.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:12 smithi142 ceph-mon[102951]: pgmap v480: 97 pgs: 97 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.1 MiB/s wr, 198 op/s 2023-10-06T21:36:12.758 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14714 4.52 MB/sec execute 1 sec latency 77.133 ms 2023-10-06T21:36:13.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14833 5.58 MB/sec execute 2 sec latency 76.976 ms 2023-10-06T21:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 -- 172.21.15.31:0/3799234047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274103320 msgr2=0x7fd274103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 --2- 172.21.15.31:0/3799234047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274103320 0x7fd274103700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd270009990 tx=0x7fd270009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 -- 172.21.15.31:0/3799234047 shutdown_connections 2023-10-06T21:36:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 --2- 172.21.15.31:0/3799234047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274108440 0x7fd27410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 --2- 172.21.15.31:0/3799234047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274103320 0x7fd274103700 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 --2- 172.21.15.31:0/3799234047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd274103cd0 0x7fd274107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 -- 172.21.15.31:0/3799234047 >> 172.21.15.31:0/3799234047 conn(0x7fd2740feba0 msgr2=0x7fd274100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 -- 172.21.15.31:0/3799234047 shutdown_connections 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.280+0000 7fd27bdff700 1 -- 172.21.15.31:0/3799234047 wait complete. 2023-10-06T21:36:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 Processor -- start 2023-10-06T21:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 -- start start 2023-10-06T21:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274103320 0x7fd27410fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 0x7fd27410ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419f650 con 0x7fd274103320 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419f7c0 con 0x7fd27410cfb0 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.282+0000 7fd27bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419f980 con 0x7fd274108440 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd279b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274103320 0x7fd27410fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:38338/0 (socket says 172.21.15.31:38338) 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 -- 172.21.15.31:0/2384804534 learned_addr learned my addr 172.21.15.31:0/2384804534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27939a700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 0x7fd27410ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 -- 172.21.15.31:0/2384804534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 msgr2=0x7fd27410ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 0x7fd27410ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 -- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274103320 msgr2=0x7fd27410fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.283+0000 7fd27a39c700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274103320 0x7fd27410fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd27a39c700 1 -- 172.21.15.31:0/2384804534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2700095f0 con 0x7fd27410cfb0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd27939a700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 0x7fd27410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd27a39c700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007ee0 tx=0x7fd26c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd26affd700 1 -- 172.21.15.31:0/2384804534 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c017070 con 0x7fd27410cfb0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27419fc60 con 0x7fd27410cfb0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd26affd700 1 -- 172.21.15.31:0/2384804534 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26c00f040 con 0x7fd27410cfb0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.284+0000 7fd26affd700 1 -- 172.21.15.31:0/2384804534 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c005570 con 0x7fd27410cfb0 2023-10-06T21:36:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.285+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2741a0170 con 0x7fd27410cfb0 2023-10-06T21:36:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.290+0000 7fd26affd700 1 -- 172.21.15.31:0/2384804534 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd26c004ad0 con 0x7fd27410cfb0 2023-10-06T21:36:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.291+0000 7fd28f6bfb80 1 Processor -- start 2023-10-06T21:36:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.292+0000 7fd28f6bfb80 1 -- start start 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bcdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bce520 0x561572bd8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572b37400 con 0x561572bce520 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572b28bc0 con 0x561572bcdc00 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572b264e0 con 0x561572bd93d0 2023-10-06T21:36:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd26affd700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 0x7fd2600a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd26affd700 1 -- 172.21.15.31:0/2384804534 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd26c0cc7f0 con 0x7fd27410cfb0 2023-10-06T21:36:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd27939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd27939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36238/0 (socket says 172.21.15.31:36238) 2023-10-06T21:36:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd27939a700 1 -- 172.21.15.31:0/1791819688 learned_addr learned my addr 172.21.15.31:0/1791819688 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.293+0000 7fd27939a700 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bcdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd279b9b700 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bce520 0x561572bd8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 -- 172.21.15.31:0/1791819688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 msgr2=0x561572bcdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bcdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 -- 172.21.15.31:0/1791819688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bce520 msgr2=0x561572bd8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bce520 0x561572bd8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 -- 172.21.15.31:0/1791819688 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56157294cd80 con 0x561572bd93d0 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27a39c700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 0x7fd2600a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.294+0000 7fd27939a700 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd264015ab0 tx=0x7fd264009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8887934d579352d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.295+0000 7fd25ffff700 1 -- 172.21.15.31:0/1791819688 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400e5a0 con 0x561572bd93d0 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.295+0000 7fd25ffff700 1 -- 172.21.15.31:0/1791819688 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26400eb90 con 0x561572bd93d0 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.295+0000 7fd25ffff700 1 -- 172.21.15.31:0/1791819688 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640189e0 con 0x561572bd93d0 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1791819688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 msgr2=0x561572bdb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd264015ab0 tx=0x7fd264009900 comp rx=0 tx=0).stop 2023-10-06T21:36:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd27a39c700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 0x7fd2600a3bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd26c099400 tx=0x7fd26c007f10 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1791819688 shutdown_connections 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bd93d0 0x561572bdb7c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bcdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1791819688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bce520 0x561572bd8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1791819688 >> 172.21.15.31:0/1791819688 conn(0x561572b43200 msgr2=0x561572bca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1791819688 shutdown_connections 2023-10-06T21:36:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.296+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1791819688 wait complete. 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.297+0000 7fd28f6bfb80 1 -- start start 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.297+0000 7fd28f6bfb80 1 Processor -- start 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572b21dd0 con 0x561572bcdc00 2023-10-06T21:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572abe6b0 con 0x561572bd93d0 2023-10-06T21:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572abe350 con 0x561572bce520 2023-10-06T21:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.298+0000 7fd28f6bfb80 10 client.0 _ll_register_callbacks cb 0x56157291b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33898/0 (socket says 172.21.15.31:33898) 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 -- 172.21.15.31:0/858623989 learned_addr learned my addr 172.21.15.31:0/858623989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd279b9b700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 -- 172.21.15.31:0/858623989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 msgr2=0x561572bd6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 -- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 msgr2=0x561572bd6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 -- 172.21.15.31:0/858623989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd264015760 con 0x561572bcdc00 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.299+0000 7fd27939a700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd264001eb0 tx=0x7fd264009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb2bcc3204610f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.300+0000 7fd25d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.300+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264018650 con 0x561572bcdc00 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.300+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561572bc1d50 con 0x561572bcdc00 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.300+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd264000e40 con 0x561572bcdc00 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26400e9e0 con 0x561572bcdc00 2023-10-06T21:36:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 10 client.17412 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561572935c00 con 0x561572bcdc00 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561572bc1d50 con 0x561572bcdc00 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd264017790 con 0x561572bcdc00 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 10 client.17412 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 10 client.17412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 20 client.17412 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.301+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572be8f80 0x561572beb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.302+0000 7fd28f6bfb80 4 client.17412 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:36:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.302+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561572b1adb0 con 0x561572be8f80 2023-10-06T21:36:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.302+0000 7fd279b9b700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572be8f80 0x561572beb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.303+0000 7fd279b9b700 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572be8f80 0x561572beb3a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdfd8fe22bf055b server_cookie=db5d523caa126e23 in_seq=0 out_seq=0 2023-10-06T21:36:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.303+0000 7fd25d7fa700 10 client.17412 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd25d7fa700 1 -- 172.21.15.31:0/858623989 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561572b1adb0 con 0x561572be8f80 2023-10-06T21:36:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd25d7fa700 10 client.17412 handle_command_reply: tid=0 2023-10-06T21:36:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 client.17412 shutdown 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 msgr2=0x561572bd8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd264001eb0 tx=0x7fd264009340 comp rx=0 tx=0).stop 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 shutdown_connections 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572be8f80 0x561572beb3a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd8e80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/858623989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd25d7fa700 0 client.17412 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 >> 172.21.15.31:0/858623989 conn(0x561572b43200 msgr2=0x561572bc9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 shutdown_connections 2023-10-06T21:36:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.304+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/858623989 wait complete. 2023-10-06T21:36:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.305+0000 7fd28f6bfb80 20 client.17412 trim_cache size 0 max 16384 2023-10-06T21:36:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 Processor -- start 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 -- start start 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572be51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bd93d0 0x561572bd8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615729a28c0 con 0x561572bd93d0 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572be0450 con 0x561572bcdc00 2023-10-06T21:36:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.331+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572bdbb20 con 0x561572bce520 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd279b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572be51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:38370/0 (socket says 172.21.15.31:38370) 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 -- 172.21.15.31:0/588958474 learned_addr learned my addr 172.21.15.31:0/588958474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bd93d0 0x561572bd8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 -- 172.21.15.31:0/588958474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 msgr2=0x561572be51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572be51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 -- 172.21.15.31:0/588958474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bd93d0 msgr2=0x561572bd8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.332+0000 7fd27939a700 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bd93d0 0x561572bd8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.333+0000 7fd27939a700 1 -- 172.21.15.31:0/588958474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615729731f0 con 0x561572bcdc00 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.333+0000 7fd27939a700 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd264009250 tx=0x7fd264006d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bc0cb67addb628 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.333+0000 7fd268bff700 1 -- 172.21.15.31:0/588958474 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400e9e0 con 0x561572bcdc00 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.333+0000 7fd268bff700 1 -- 172.21.15.31:0/588958474 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd264029030 con 0x561572bcdc00 2023-10-06T21:36:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.333+0000 7fd268bff700 1 -- 172.21.15.31:0/588958474 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264018a00 con 0x561572bcdc00 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/588958474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 msgr2=0x561572bdff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd264009250 tx=0x7fd264006d30 comp rx=0 tx=0).stop 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/588958474 shutdown_connections 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bd93d0 0x561572bd8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bcdc00 0x561572bdff10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/588958474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572be51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/588958474 >> 172.21.15.31:0/588958474 conn(0x561572b43200 msgr2=0x561572befcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/588958474 shutdown_connections 2023-10-06T21:36:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.334+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/588958474 wait complete. 2023-10-06T21:36:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.335+0000 7fd28f6bfb80 1 -- start start 2023-10-06T21:36:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.335+0000 7fd28f6bfb80 1 Processor -- start 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572bd40f0 con 0x561572bcdc00 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572bd28c0 con 0x561572bd93d0 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561572bd2a80 con 0x561572bce520 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd28f6bfb80 10 client.0 _ll_register_callbacks cb 0x7fd26400fd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd27939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd27939a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33904/0 (socket says 172.21.15.31:33904) 2023-10-06T21:36:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd27939a700 1 -- 172.21.15.31:0/1992415125 learned_addr learned my addr 172.21.15.31:0/1992415125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd27939a700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.336+0000 7fd279b9b700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 -- 172.21.15.31:0/1992415125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 msgr2=0x561572bd3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 -- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 msgr2=0x561572bd2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 -- 172.21.15.31:0/1992415125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561572935c00 con 0x561572bcdc00 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd27939a700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd264009890 tx=0x7fd264017d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616d9d63c13b1571 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.337+0000 7fd25d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264017850 con 0x561572bcdc00 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fd264000e40 con 0x561572bcdc00 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd26400eb90 con 0x561572bcdc00 2023-10-06T21:36:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26400e9e0 con 0x561572bcdc00 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd28f6bfb80 10 client.17418 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.338+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56157294b250 con 0x561572bcdc00 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26400eb90 con 0x561572bcdc00 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd2640464f0 con 0x561572bcdc00 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 10 client.17418 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 20 client.17418 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572c3d6a0 0x561572c3fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 4 client.17418 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:36:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561572b1aad0 con 0x561572c3d6a0 2023-10-06T21:36:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.339+0000 7fd279b9b700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572c3d6a0 0x561572c3fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.340+0000 7fd279b9b700 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572c3d6a0 0x561572c3fa90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6389c3d24542f54 server_cookie=d84dd742dbaaf0bb in_seq=0 out_seq=0 2023-10-06T21:36:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.340+0000 7fd25d7fa700 10 client.17418 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.386+0000 7fd25d7fa700 1 -- 172.21.15.31:0/1992415125 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561572b1aad0 con 0x561572c3d6a0 2023-10-06T21:36:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.386+0000 7fd25d7fa700 10 client.17418 handle_command_reply: tid=0 2023-10-06T21:36:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.386+0000 7fd28f6bfb80 1 client.17418 shutdown 2023-10-06T21:36:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.386+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 msgr2=0x561572bd34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.386+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd264009890 tx=0x7fd264017d20 comp rx=0 tx=0).stop 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 shutdown_connections 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561572bd93d0 0x561572bd2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x561572c3d6a0 0x561572c3fa90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561572bcdc00 0x561572bd34e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 --2- 172.21.15.31:0/1992415125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561572bce520 0x561572bd3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 >> 172.21.15.31:0/1992415125 conn(0x561572b43200 msgr2=0x561572be51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd25d7fa700 0 client.17418 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 shutdown_connections 2023-10-06T21:36:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 1 -- 172.21.15.31:0/1992415125 wait complete. 2023-10-06T21:36:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.387+0000 7fd28f6bfb80 20 client.17418 trim_cache size 0 max 16384 2023-10-06T21:36:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:36:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:36:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:36:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:36:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.388+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 msgr2=0x7fd2600a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.388+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 0x7fd2600a3bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd26c099400 tx=0x7fd26c007f10 comp rx=0 tx=0).stop 2023-10-06T21:36:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.388+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 msgr2=0x7fd27410d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.388+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007ee0 tx=0x7fd26c00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 shutdown_connections 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27410cfb0 0x7fd27410d460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd2600a16f0 0x7fd2600a3bb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd274103320 0x7fd27410fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 --2- 172.21.15.31:0/2384804534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274108440 0x7fd27410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 >> 172.21.15.31:0/2384804534 conn(0x7fd2740feba0 msgr2=0x7fd274106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 shutdown_connections 2023-10-06T21:36:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:14.389+0000 7fd27bdff700 1 -- 172.21.15.31:0/2384804534 wait complete. 2023-10-06T21:36:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:14 smithi031 ceph-mon[98539]: pgmap v481: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:14.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:14 smithi031 ceph-mon[98539]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:36:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:14 smithi146 ceph-mon[102809]: pgmap v481: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:14 smithi146 ceph-mon[102809]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:36:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:14 smithi142 ceph-mon[102951]: pgmap v481: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:14 smithi142 ceph-mon[102951]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:36:14.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 14952 5.35 MB/sec execute 3 sec latency 80.214 ms 2023-10-06T21:36:15.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15073 6.20 MB/sec execute 4 sec latency 72.246 ms 2023-10-06T21:36:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:16 smithi031 ceph-mon[98539]: pgmap v483: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:16 smithi146 ceph-mon[102809]: pgmap v483: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:16 smithi142 ceph-mon[102951]: pgmap v483: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:16.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15186 5.92 MB/sec execute 5 sec latency 73.159 ms 2023-10-06T21:36:17.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15314 6.45 MB/sec execute 6 sec latency 69.338 ms 2023-10-06T21:36:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:18 smithi031 ceph-mon[98539]: pgmap v484: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:18 smithi146 ceph-mon[102809]: pgmap v484: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:18 smithi142 ceph-mon[102951]: pgmap v484: 97 pgs: 97 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 201 op/s 2023-10-06T21:36:18.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15442 6.25 MB/sec execute 7 sec latency 64.733 ms 2023-10-06T21:36:19.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15559 5.75 MB/sec execute 8 sec latency 73.531 ms 2023-10-06T21:36:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:20 smithi142 ceph-mon[102951]: pgmap v485: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:20 smithi031 ceph-mon[98539]: pgmap v485: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:20.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:20 smithi146 ceph-mon[102809]: pgmap v485: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:20.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15688 5.65 MB/sec execute 9 sec latency 64.467 ms 2023-10-06T21:36:21.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15807 5.21 MB/sec execute 10 sec latency 64.666 ms 2023-10-06T21:36:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:22 smithi031 ceph-mon[98539]: pgmap v486: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:22 smithi146 ceph-mon[102809]: pgmap v486: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:22.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:22 smithi142 ceph-mon[102951]: pgmap v486: 97 pgs: 97 active+clean; 456 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-10-06T21:36:22.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 15940 5.34 MB/sec execute 11 sec latency 71.029 ms 2023-10-06T21:36:23.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16050 5.17 MB/sec execute 12 sec latency 67.884 ms 2023-10-06T21:36:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:24 smithi031 ceph-mon[98539]: pgmap v487: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-10-06T21:36:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:24 smithi146 ceph-mon[102809]: pgmap v487: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-10-06T21:36:24.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:24 smithi142 ceph-mon[102951]: pgmap v487: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-10-06T21:36:24.759 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16166 5.24 MB/sec execute 13 sec latency 68.267 ms 2023-10-06T21:36:25.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16292 5.28 MB/sec execute 14 sec latency 60.933 ms 2023-10-06T21:36:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:26 smithi031 ceph-mon[98539]: pgmap v488: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-10-06T21:36:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:26 smithi146 ceph-mon[102809]: pgmap v488: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-10-06T21:36:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:26 smithi142 ceph-mon[102951]: pgmap v488: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-10-06T21:36:26.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16455 5.78 MB/sec execute 15 sec latency 69.366 ms 2023-10-06T21:36:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.618+0000 7f191a3ff700 1 -- 172.21.15.31:0/3618285901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1914101a80 msgr2=0x7f1914105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.618+0000 7f191a3ff700 1 --2- 172.21.15.31:0/3618285901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1914101a80 0x7f1914105ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1908009990 tx=0x7f1908009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 -- 172.21.15.31:0/3618285901 shutdown_connections 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 --2- 172.21.15.31:0/3618285901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19141061f0 0x7f191410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 --2- 172.21.15.31:0/3618285901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f19141014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 --2- 172.21.15.31:0/3618285901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1914101a80 0x7f1914105ac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 -- 172.21.15.31:0/3618285901 >> 172.21.15.31:0/3618285901 conn(0x7f19140fc950 msgr2=0x7f19140fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 -- 172.21.15.31:0/3618285901 shutdown_connections 2023-10-06T21:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.619+0000 7f191a3ff700 1 -- 172.21.15.31:0/3618285901 wait complete. 2023-10-06T21:36:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.620+0000 7f191a3ff700 1 Processor -- start 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 -- start start 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f1914199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 0x7f1914199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f191419dcd0 0x7f191419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419e750 con 0x7f19141061f0 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419e890 con 0x7f191419dcd0 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.621+0000 7f191a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419ea00 con 0x7f19141010d0 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f19137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 0x7f1914199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f1914199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f19137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 0x7f1914199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49236/0 (socket says 172.21.15.31:49236) 2023-10-06T21:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f19137fe700 1 -- 172.21.15.31:0/255326827 learned_addr learned my addr 172.21.15.31:0/255326827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f191899c700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f191419dcd0 0x7f191419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 -- 172.21.15.31:0/255326827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f191419dcd0 msgr2=0x7f191419e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f191419dcd0 0x7f191419e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 -- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 msgr2=0x7f1914199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 0x7f1914199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 -- 172.21.15.31:0/255326827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009c40 con 0x7f19141010d0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.622+0000 7f1913fff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f1914199120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f190400e9f0 tx=0x7f190400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.623+0000 7f19117fa700 1 -- 172.21.15.31:0/255326827 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400c9c0 con 0x7f19141010d0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.623+0000 7f19117fa700 1 -- 172.21.15.31:0/255326827 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1904010430 con 0x7f19141010d0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.623+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19080095f0 con 0x7f19141010d0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.623+0000 7f19117fa700 1 -- 172.21.15.31:0/255326827 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904018680 con 0x7f19141010d0 2023-10-06T21:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.623+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191406aca0 con 0x7f19141010d0 2023-10-06T21:36:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.629+0000 7f19117fa700 1 -- 172.21.15.31:0/255326827 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f19040105a0 con 0x7f19141010d0 2023-10-06T21:36:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.629+0000 7f192dc83b80 1 Processor -- start 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 -- start start 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c13b400 con 0x55cc6c1dd3d0 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c12cbc0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.630+0000 7f192dc83b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c12a4e0 con 0x55cc6c1d2520 2023-10-06T21:36:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f1913fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41330/0 (socket says 172.21.15.31:41330) 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 -- 172.21.15.31:0/322251027 learned_addr learned my addr 172.21.15.31:0/322251027 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f19137fe700 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 -- 172.21.15.31:0/322251027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d2520 msgr2=0x55cc6c1dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 -- 172.21.15.31:0/322251027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 msgr2=0x55cc6c1df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f191899c700 1 -- 172.21.15.31:0/322251027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc6bf50d80 con 0x55cc6c1d1c00 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.631+0000 7f19117fa700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 0x7f18fc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f19137fe700 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:36:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f19117fa700 1 -- 172.21.15.31:0/255326827 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1904014070 con 0x7f19141010d0 2023-10-06T21:36:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f191899c700 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e730 tx=0x7f190c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941b1174a9c40cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f1913fff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 0x7f18fc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f18fa7fc700 1 -- 172.21.15.31:0/322251027 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00e040 con 0x55cc6c1d1c00 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f18fa7fc700 1 -- 172.21.15.31:0/322251027 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f190c00f040 con 0x55cc6c1d1c00 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.632+0000 7f18fa7fc700 1 -- 172.21.15.31:0/322251027 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c007500 con 0x55cc6c1d1c00 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 -- 172.21.15.31:0/322251027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 msgr2=0x55cc6c1d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e730 tx=0x7f190c009920 comp rx=0 tx=0).stop 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f1913fff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 0x7f18fc0a3b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19040ca9e0 tx=0x7f1904009790 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 -- 172.21.15.31:0/322251027 shutdown_connections 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1d1fe0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 --2- 172.21.15.31:0/322251027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.633+0000 7f192dc83b80 1 -- 172.21.15.31:0/322251027 >> 172.21.15.31:0/322251027 conn(0x55cc6c147200 msgr2=0x55cc6c1ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.634+0000 7f192dc83b80 1 -- 172.21.15.31:0/322251027 shutdown_connections 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.634+0000 7f192dc83b80 1 -- 172.21.15.31:0/322251027 wait complete. 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.635+0000 7f192dc83b80 1 -- start start 2023-10-06T21:36:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.635+0000 7f192dc83b80 1 Processor -- start 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1da6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c125dd0 con 0x55cc6c1d2520 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c0c26b0 con 0x55cc6c1dd3d0 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c0c2350 con 0x55cc6c1d1c00 2023-10-06T21:36:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f192dc83b80 10 client.0 _ll_register_callbacks cb 0x55cc6bf1f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:59622/0 (socket says 172.21.15.31:59622) 2023-10-06T21:36:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.636+0000 7f191899c700 1 -- 172.21.15.31:0/496743857 learned_addr learned my addr 172.21.15.31:0/496743857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1da6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 -- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 msgr2=0x55cc6c1da6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1da6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 -- 172.21.15.31:0/496743857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 msgr2=0x55cc6c1da020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 -- 172.21.15.31:0/496743857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c00e410 con 0x55cc6c1d1c00 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.637+0000 7f191899c700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f190c009f00 tx=0x7f190c0045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404ce2c16e439757 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.638+0000 7f18f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:36:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.638+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c009a10 con 0x55cc6c1d1c00 2023-10-06T21:36:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.638+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f190c007500 con 0x55cc6c1d1c00 2023-10-06T21:36:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.638+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc6c1c5d50 con 0x55cc6c1d1c00 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.638+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f190c0076c0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 10 client.27076 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cc6bf39c00 con 0x55cc6c1d1c00 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cc6c1c5d50 con 0x55cc6c1d1c00 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f190c0154e0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 10 client.27076 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 20 client.27076 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c1e3df0 0x55cc6c1e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 4 client.27076 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:36:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.639+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc6c11edb0 con 0x55cc6c1e3df0 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.640+0000 7f19137fe700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c1e3df0 0x55cc6c1e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.641+0000 7f19137fe700 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c1e3df0 0x55cc6c1e6210 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d61b232dca48b server_cookie=6e88a0dcddd15082 in_seq=0 out_seq=0 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.641+0000 7f18f37fe700 10 client.27076 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.641+0000 7f18f37fe700 1 -- 172.21.15.31:0/496743857 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc6c11edb0 con 0x55cc6c1e3df0 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.641+0000 7f18f37fe700 10 client.27076 handle_command_reply: tid=0 2023-10-06T21:36:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.641+0000 7f192dc83b80 1 client.27076 shutdown 2023-10-06T21:36:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 msgr2=0x55cc6c1dcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f190c009f00 tx=0x7f190c0045d0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 shutdown_connections 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1da6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1dcda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c1e3df0 0x55cc6c1e6210 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 --2- 172.21.15.31:0/496743857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1da020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 >> 172.21.15.31:0/496743857 conn(0x55cc6c147200 msgr2=0x55cc6c1cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f18f37fe700 0 client.27076 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 shutdown_connections 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.642+0000 7f192dc83b80 1 -- 172.21.15.31:0/496743857 wait complete. 2023-10-06T21:36:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.643+0000 7f192dc83b80 20 client.27076 trim_cache size 0 max 16384 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.670+0000 7f192dc83b80 1 Processor -- start 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.670+0000 7f192dc83b80 1 -- start start 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6bfa68c0 con 0x55cc6c1d2520 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c1cccf0 con 0x55cc6c1dd3d0 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f192dc83b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c1dfb80 con 0x55cc6c1d1c00 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:59628/0 (socket says 172.21.15.31:59628) 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f191899c700 1 -- 172.21.15.31:0/1204384619 learned_addr learned my addr 172.21.15.31:0/1204384619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.671+0000 7f191899c700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.672+0000 7f19137fe700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.672+0000 7f191899c700 1 -- 172.21.15.31:0/1204384619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 msgr2=0x55cc6c1d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f191899c700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1d72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f191899c700 1 -- 172.21.15.31:0/1204384619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 msgr2=0x55cc6c1dc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f191899c700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f191899c700 1 -- 172.21.15.31:0/1204384619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc6bf771f0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f19137fe700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f191899c700 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e250 tx=0x7f190c00e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb5346dc6394726 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f18faffd700 1 -- 172.21.15.31:0/1204384619 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0098a0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f18faffd700 1 -- 172.21.15.31:0/1204384619 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f190c009a00 con 0x55cc6c1d1c00 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.673+0000 7f18faffd700 1 -- 172.21.15.31:0/1204384619 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c005030 con 0x55cc6c1d1c00 2023-10-06T21:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 -- 172.21.15.31:0/1204384619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 msgr2=0x55cc6c1cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e250 tx=0x7f190c00e040 comp rx=0 tx=0).stop 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 -- 172.21.15.31:0/1204384619 shutdown_connections 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1cc7b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 --2- 172.21.15.31:0/1204384619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1d2520 0x55cc6c1dc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 -- 172.21.15.31:0/1204384619 >> 172.21.15.31:0/1204384619 conn(0x55cc6c147200 msgr2=0x55cc6c2067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 -- 172.21.15.31:0/1204384619 shutdown_connections 2023-10-06T21:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.675+0000 7f192dc83b80 1 -- 172.21.15.31:0/1204384619 wait complete. 2023-10-06T21:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.676+0000 7f192dc83b80 1 -- start start 2023-10-06T21:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.676+0000 7f192dc83b80 1 Processor -- start 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 0x55cc6c1e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c1e9d80 con 0x55cc6c1dd3d0 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c251890 con 0x55cc6c1d2520 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc6c2519d0 con 0x55cc6c1d1c00 2023-10-06T21:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.677+0000 7f192dc83b80 10 client.0 _ll_register_callbacks cb 0x7f190c004980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 0x55cc6c1e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f191899c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:59638/0 (socket says 172.21.15.31:59638) 2023-10-06T21:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f191899c700 1 -- 172.21.15.31:0/3334194725 learned_addr learned my addr 172.21.15.31:0/3334194725 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f191899c700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 -- 172.21.15.31:0/3334194725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 msgr2=0x55cc6c1ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 -- 172.21.15.31:0/3334194725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 msgr2=0x55cc6c1e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.678+0000 7f19137fe700 1 -- 172.21.15.31:0/3334194725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc6bf39c00 con 0x55cc6c1d2520 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f191899c700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f19137fe700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 0x55cc6c1e8fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55cc6c1db5d0 tx=0x7f1908009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb21102882d63258 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f18f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080255a0 con 0x55cc6c1d2520 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1908025990 con 0x55cc6c1d2520 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.679+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc6c11ead0 con 0x55cc6c1d2520 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.680+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1908025b50 con 0x55cc6c1d2520 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.680+0000 7f192dc83b80 10 client.27128 fetch_fsmap learned FSMap version 44 2023-10-06T21:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.680+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cc6bf4f250 con 0x55cc6c1d2520 2023-10-06T21:36:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.680+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1908034030 con 0x55cc6c1d2520 2023-10-06T21:36:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.680+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1908023050 con 0x55cc6c1d2520 2023-10-06T21:36:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 10 client.27128 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:36:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 20 client.27128 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c252ff0 0x55cc6c255410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 4 client.27128 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc6bf71f10 con 0x55cc6c252ff0 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.681+0000 7f191899c700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c252ff0 0x55cc6c255410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.682+0000 7f191899c700 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c252ff0 0x55cc6c255410 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6112b30e210997dd server_cookie=21301cb3b698cbba in_seq=0 out_seq=0 2023-10-06T21:36:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.682+0000 7f18f37fe700 10 client.27128 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.722+0000 7f18f37fe700 1 -- 172.21.15.31:0/3334194725 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc6bf71f10 con 0x55cc6c252ff0 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.722+0000 7f18f37fe700 10 client.27128 handle_command_reply: tid=0 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.722+0000 7f192dc83b80 1 client.27128 shutdown 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 msgr2=0x55cc6c1e8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 0x55cc6c1e8fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55cc6c1db5d0 tx=0x7f1908009290 comp rx=0 tx=0).stop 2023-10-06T21:36:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 shutdown_connections 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55cc6c252ff0 0x55cc6c255410 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc6c1dd3d0 0x55cc6c1e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc6c1d1c00 0x55cc6c1ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f18f37fe700 0 client.27128 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 --2- 172.21.15.31:0/3334194725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc6c1d2520 0x55cc6c1e8fa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 >> 172.21.15.31:0/3334194725 conn(0x55cc6c147200 msgr2=0x55cc6c1d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 shutdown_connections 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 1 -- 172.21.15.31:0/3334194725 wait complete. 2023-10-06T21:36:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.723+0000 7f192dc83b80 20 client.27128 trim_cache size 0 max 16384 2023-10-06T21:36:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:36:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:36:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:36:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 msgr2=0x7f18fc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 0x7f18fc0a3b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19040ca9e0 tx=0x7f1904009790 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 msgr2=0x7f1914199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f1914199120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f190400e9f0 tx=0x7f190400ed00 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 shutdown_connections 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f191419dcd0 0x7f191419e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19141010d0 0x7f1914199120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f18fc0a1670 0x7f18fc0a3b30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 --2- 172.21.15.31:0/255326827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19141061f0 0x7f1914199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.724+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 >> 172.21.15.31:0/255326827 conn(0x7f19140fc950 msgr2=0x7f19140fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.725+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 shutdown_connections 2023-10-06T21:36:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:27.725+0000 7f191a3ff700 1 -- 172.21.15.31:0/255326827 wait complete. 2023-10-06T21:36:27.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16629 6.29 MB/sec execute 16 sec latency 99.759 ms 2023-10-06T21:36:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:28 smithi031 ceph-mon[98539]: pgmap v489: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2023-10-06T21:36:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:28 smithi146 ceph-mon[102809]: pgmap v489: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2023-10-06T21:36:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:28 smithi142 ceph-mon[102951]: pgmap v489: 97 pgs: 97 active+clean; 456 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2023-10-06T21:36:28.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16797 6.48 MB/sec execute 17 sec latency 80.600 ms 2023-10-06T21:36:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:29 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:36:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:29 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:36:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:29 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:36:29.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 16904 6.54 MB/sec execute 18 sec latency 72.498 ms 2023-10-06T21:36:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:30 smithi031 ceph-mon[98539]: pgmap v490: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2023-10-06T21:36:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:30 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:36:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:30 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:36:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:30 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:36:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:30 smithi146 ceph-mon[102809]: pgmap v490: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2023-10-06T21:36:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:30 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:36:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:30 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:36:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:30 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:36:30.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:30 smithi142 ceph-mon[102951]: pgmap v490: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2023-10-06T21:36:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:30 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:36:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:30 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:36:30.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:30 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:36:30.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17002 6.48 MB/sec execute 19 sec latency 84.872 ms 2023-10-06T21:36:31.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17090 6.57 MB/sec execute 20 sec latency 319.061 ms 2023-10-06T21:36:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:32 smithi031 ceph-mon[98539]: pgmap v491: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-10-06T21:36:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:32 smithi146 ceph-mon[102809]: pgmap v491: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-10-06T21:36:32.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:32 smithi142 ceph-mon[102951]: pgmap v491: 97 pgs: 97 active+clean; 466 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2023-10-06T21:36:32.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17229 6.82 MB/sec execute 21 sec latency 68.588 ms 2023-10-06T21:36:33.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17381 7.00 MB/sec execute 22 sec latency 70.385 ms 2023-10-06T21:36:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:34 smithi031 ceph-mon[98539]: pgmap v492: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2023-10-06T21:36:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:34 smithi146 ceph-mon[102809]: pgmap v492: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2023-10-06T21:36:34.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:34 smithi142 ceph-mon[102951]: pgmap v492: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2023-10-06T21:36:34.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17490 7.05 MB/sec execute 23 sec latency 125.843 ms 2023-10-06T21:36:35.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17569 6.84 MB/sec execute 24 sec latency 417.474 ms 2023-10-06T21:36:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:36 smithi031 ceph-mon[98539]: pgmap v493: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:36 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:36 smithi146 ceph-mon[102809]: pgmap v493: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:36 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:36 smithi142 ceph-mon[102951]: pgmap v493: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:36.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:36 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:36:36.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17675 6.82 MB/sec execute 25 sec latency 71.479 ms 2023-10-06T21:36:37.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17763 6.64 MB/sec execute 26 sec latency 75.337 ms 2023-10-06T21:36:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:38 smithi031 ceph-mon[98539]: pgmap v494: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:38 smithi146 ceph-mon[102809]: pgmap v494: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:38 smithi142 ceph-mon[102951]: pgmap v494: 97 pgs: 97 active+clean; 475 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.6 MiB/s wr, 175 op/s 2023-10-06T21:36:38.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17867 6.53 MB/sec execute 27 sec latency 80.336 ms 2023-10-06T21:36:39.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 17983 6.50 MB/sec execute 28 sec latency 78.439 ms 2023-10-06T21:36:40.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:40 smithi142 ceph-mon[102951]: pgmap v495: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2023-10-06T21:36:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:40 smithi031 ceph-mon[98539]: pgmap v495: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2023-10-06T21:36:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:40 smithi146 ceph-mon[102809]: pgmap v495: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2023-10-06T21:36:40.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18065 6.34 MB/sec execute 29 sec latency 419.377 ms 2023-10-06T21:36:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 -- 172.21.15.31:0/4174952275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb0068c90 msgr2=0x7f0bb010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/4174952275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb0068c90 0x7f0bb010dd00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009f90 tx=0x7f0ba800b330 comp rx=0 tx=0).stop 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 -- 172.21.15.31:0/4174952275 shutdown_connections 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/4174952275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb0068c90 0x7f0bb010dd00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/4174952275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0106f60 0x7f0bb0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/4174952275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb00682d0 0x7f0bb0068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 -- 172.21.15.31:0/4174952275 >> 172.21.15.31:0/4174952275 conn(0x7f0bb0076b70 msgr2=0x7f0bb0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 -- 172.21.15.31:0/4174952275 shutdown_connections 2023-10-06T21:36:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.968+0000 7f0bb65ff700 1 -- 172.21.15.31:0/4174952275 wait complete. 2023-10-06T21:36:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.970+0000 7f0bb65ff700 1 Processor -- start 2023-10-06T21:36:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.970+0000 7f0bb65ff700 1 -- start start 2023-10-06T21:36:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.970+0000 7f0bb65ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.970+0000 7f0bb65ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb0068c90 0x7f0bb0072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.970+0000 7f0bb65ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0106f60 0x7f0bb006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0bb65ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006ee10 con 0x7f0bb0106f60 2023-10-06T21:36:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0bb65ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006ef80 con 0x7f0bb00682d0 2023-10-06T21:36:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0bb65ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006f140 con 0x7f0bb0068c90 2023-10-06T21:36:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0baffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0baffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58262/0 (socket says 172.21.15.31:58262) 2023-10-06T21:36:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.971+0000 7f0baffff700 1 -- 172.21.15.31:0/603656133 learned_addr learned my addr 172.21.15.31:0/603656133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 -- 172.21.15.31:0/603656133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb0068c90 msgr2=0x7f0bb0072b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb0068c90 0x7f0bb0072b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 -- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0106f60 msgr2=0x7f0bb006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0106f60 0x7f0bb006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 -- 172.21.15.31:0/603656133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba8009c40 con 0x7f0bb00682d0 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0baffff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9a0 tx=0x7f0ba000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0badffb700 1 -- 172.21.15.31:0/603656133 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000cbd0 con 0x7f0bb00682d0 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0badffb700 1 -- 172.21.15.31:0/603656133 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba000cd30 con 0x7f0bb00682d0 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0badffb700 1 -- 172.21.15.31:0/603656133 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0018680 con 0x7f0bb00682d0 2023-10-06T21:36:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb01ae5c0 con 0x7f0bb00682d0 2023-10-06T21:36:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.972+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01aea20 con 0x7f0bb00682d0 2023-10-06T21:36:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.976+0000 7f0badffb700 1 -- 172.21.15.31:0/603656133 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0ba0010430 con 0x7f0bb00682d0 2023-10-06T21:36:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.977+0000 7f0bc9e56b80 1 Processor -- start 2023-10-06T21:36:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 -- start start 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb70e400 con 0x5644cb7a5520 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb6ffbc0 con 0x5644cb7b03d0 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb6fd4e0 con 0x5644cb7a4c00 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.978+0000 7f0badffb700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 0x7f0b9c0a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0badffb700 1 -- 172.21.15.31:0/603656133 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0ba0014070 con 0x7f0bb00682d0 2023-10-06T21:36:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58264/0 (socket says 172.21.15.31:58264) 2023-10-06T21:36:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 -- 172.21.15.31:0/256625139 learned_addr learned my addr 172.21.15.31:0/256625139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0baffff700 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 -- 172.21.15.31:0/256625139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 msgr2=0x5644cb7a4fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7a4fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 -- 172.21.15.31:0/256625139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 msgr2=0x5644cb7afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0ba7fff700 1 -- 172.21.15.31:0/256625139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644cb523d80 con 0x5644cb7b03d0 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.979+0000 7f0baffff700 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.980+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009b00 tx=0x7f0b98009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da131a56d0059f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.980+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 0x7f0b9c0a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.980+0000 7f0ba67fc700 1 -- 172.21.15.31:0/256625139 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98004220 con 0x5644cb7b03d0 2023-10-06T21:36:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.980+0000 7f0ba67fc700 1 -- 172.21.15.31:0/256625139 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b9800ed90 con 0x5644cb7b03d0 2023-10-06T21:36:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.980+0000 7f0ba67fc700 1 -- 172.21.15.31:0/256625139 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98004e40 con 0x5644cb7b03d0 2023-10-06T21:36:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.981+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/256625139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 msgr2=0x5644cb7b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.981+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009b00 tx=0x7f0b98009e10 comp rx=0 tx=0).stop 2023-10-06T21:36:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 0x7f0b9c0a3bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009790 tx=0x7f0ba80096c0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/256625139 shutdown_connections 2023-10-06T21:36:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7b27c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/256625139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/256625139 >> 172.21.15.31:0/256625139 conn(0x5644cb71a200 msgr2=0x5644cb7a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/256625139 shutdown_connections 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.982+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/256625139 wait complete. 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.983+0000 7f0bc9e56b80 1 -- start start 2023-10-06T21:36:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.983+0000 7f0bc9e56b80 1 Processor -- start 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb6f8dd0 con 0x5644cb7a5520 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb6956b0 con 0x5644cb7b03d0 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb695350 con 0x5644cb7a4c00 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.984+0000 7f0bc9e56b80 10 client.0 _ll_register_callbacks cb 0x5644cb4f27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58266/0 (socket says 172.21.15.31:58266) 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/2488805790 learned_addr learned my addr 172.21.15.31:0/2488805790 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0baffff700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/2488805790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 msgr2=0x5644cb7afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/2488805790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 msgr2=0x5644cb7ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/2488805790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b980097b0 con 0x5644cb7b03d0 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.985+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007860 tx=0x7f0ba8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a928bbd4bdeca99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0b8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800e040 con 0x5644cb7b03d0 2023-10-06T21:36:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0ba800bbf0 con 0x5644cb7b03d0 2023-10-06T21:36:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644cb798d50 con 0x5644cb7b03d0 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ba8004e90 con 0x5644cb7b03d0 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0bc9e56b80 10 client.27140 fetch_fsmap learned FSMap version 45 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.986+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5644cb50cc00 con 0x5644cb7b03d0 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5644cb798d50 con 0x5644cb7b03d0 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0ba8005040 con 0x5644cb7b03d0 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 10 client.27140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 20 client.27140 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb7c87d0 0x5644cb7cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 4 client.27140 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.987+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644cb6f1db0 con 0x5644cb7c87d0 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.988+0000 7f0baffff700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb7c87d0 0x5644cb7cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.989+0000 7f0baffff700 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb7c87d0 0x5644cb7cabf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ae3b7598cd15f6 server_cookie=c32cd01bd9759323 in_seq=0 out_seq=0 2023-10-06T21:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.989+0000 7f0b8f7fe700 10 client.27140 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.990+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/2488805790 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5644cb6f1db0 con 0x5644cb7c87d0 2023-10-06T21:36:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.990+0000 7f0b8f7fe700 10 client.27140 handle_command_reply: tid=0 2023-10-06T21:36:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.990+0000 7f0bc9e56b80 1 client.27140 shutdown 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.990+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 msgr2=0x5644cb7ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.990+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007860 tx=0x7f0ba8009920 comp rx=0 tx=0).stop 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 shutdown_connections 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7b03d0 0x5644cb7ad7d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb7c87d0 0x5644cb7cabf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/2488805790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 >> 172.21.15.31:0/2488805790 conn(0x5644cb71a200 msgr2=0x5644cb7a0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0b8f7fe700 0 client.27140 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 shutdown_connections 2023-10-06T21:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/2488805790 wait complete. 2023-10-06T21:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:40.991+0000 7f0bc9e56b80 20 client.27140 trim_cache size 0 max 16384 2023-10-06T21:36:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.019+0000 7f0bc9e56b80 1 Processor -- start 2023-10-06T21:36:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.019+0000 7f0bc9e56b80 1 -- start start 2023-10-06T21:36:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb5798c0 con 0x5644cb7a5520 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb7a66c0 con 0x5644cb7a4c00 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb7a8f30 con 0x5644cb7b03d0 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0baffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bb4b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bb4b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:50010/0 (socket says 172.21.15.31:50010) 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/548337819 learned_addr learned my addr 172.21.15.31:0/548337819 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.020+0000 7f0baffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58276/0 (socket says 172.21.15.31:58276) 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/548337819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 msgr2=0x5644cb7ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/548337819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 msgr2=0x5644cb7bb670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 -- 172.21.15.31:0/548337819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644cb54a1f0 con 0x5644cb7b03d0 2023-10-06T21:36:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0baffff700 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80061e0 tx=0x7f0ba8007520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=962cf7e3ebdbb97e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.021+0000 7f0ba6ffd700 1 -- 172.21.15.31:0/548337819 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800e040 con 0x5644cb7b03d0 2023-10-06T21:36:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.022+0000 7f0ba6ffd700 1 -- 172.21.15.31:0/548337819 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba800be80 con 0x5644cb7b03d0 2023-10-06T21:36:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.022+0000 7f0ba6ffd700 1 -- 172.21.15.31:0/548337819 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8035070 con 0x5644cb7b03d0 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.022+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/548337819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 msgr2=0x5644cb7af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.022+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80061e0 tx=0x7f0ba8007520 comp rx=0 tx=0).stop 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/548337819 shutdown_connections 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7b03d0 0x5644cb7af940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a4c00 0x5644cb7ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/548337819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7a5520 0x5644cb7bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/548337819 >> 172.21.15.31:0/548337819 conn(0x5644cb71a200 msgr2=0x5644cb7ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/548337819 shutdown_connections 2023-10-06T21:36:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.023+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/548337819 wait complete. 2023-10-06T21:36:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.024+0000 7f0bc9e56b80 1 -- start start 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.024+0000 7f0bc9e56b80 1 Processor -- start 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7b03d0 0x5644cb7b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb7b7510 con 0x5644cb7b03d0 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb81b600 con 0x5644cb7a5520 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644cb81b790 con 0x5644cb7a4c00 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bc9e56b80 10 client.0 _ll_register_callbacks cb 0x5644cb4f27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58290/0 (socket says 172.21.15.31:58290) 2023-10-06T21:36:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0ba7fff700 1 -- 172.21.15.31:0/1377850189 learned_addr learned my addr 172.21.15.31:0/1377850189 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.025+0000 7f0bb4b9c700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7b03d0 0x5644cb7b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 -- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 msgr2=0x5644cb7b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 -- 172.21.15.31:0/1377850189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7b03d0 msgr2=0x5644cb7b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7b03d0 0x5644cb7b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0baffff700 1 -- 172.21.15.31:0/1377850189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01aea20 con 0x5644cb7a4c00 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.026+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0baffff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7b4440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00cab50 tx=0x7f0ba009a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e7ce5b759ccac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0b8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00b2ea0 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0ba009a3e0 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644cb6f1ad0 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.027+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ba009e530 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 10 client.27094 fetch_fsmap learned FSMap version 45 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5644cb50cc00 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba00b6020 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 45) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0ba00b1060 con 0x5644cb7a4c00 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 10 client.27094 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 10 client.27094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 20 client.27094 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb81d020 0x5644cb81f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 4 client.27094 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.028+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5644cb544f10 con 0x5644cb81d020 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.029+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb81d020 0x5644cb81f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.029+0000 7f0b8f7fe700 10 client.27094 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.029+0000 7f0ba7fff700 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb81d020 0x5644cb81f440 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276cc98190902fac server_cookie=17876f461b175605 in_seq=0 out_seq=0 2023-10-06T21:36:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.073+0000 7f0b8f7fe700 1 -- 172.21.15.31:0/1377850189 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5644cb544f10 con 0x5644cb81d020 2023-10-06T21:36:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.073+0000 7f0b8f7fe700 10 client.27094 handle_command_reply: tid=0 2023-10-06T21:36:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.073+0000 7f0bc9e56b80 1 client.27094 shutdown 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 msgr2=0x5644cb7b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7b4440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00cab50 tx=0x7f0ba009a240 comp rx=0 tx=0).stop 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 shutdown_connections 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5644cb81d020 0x5644cb81f440 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644cb7b03d0 0x5644cb7b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644cb7a4c00 0x5644cb7b4440 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 --2- 172.21.15.31:0/1377850189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cb7a5520 0x5644cb7b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 >> 172.21.15.31:0/1377850189 conn(0x5644cb71a200 msgr2=0x5644cb7bb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.074+0000 7f0b8f7fe700 0 client.27094 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.075+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 shutdown_connections 2023-10-06T21:36:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.075+0000 7f0bc9e56b80 1 -- 172.21.15.31:0/1377850189 wait complete. 2023-10-06T21:36:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.075+0000 7f0bc9e56b80 20 client.27094 trim_cache size 0 max 16384 2023-10-06T21:36:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:36:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:36:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:36:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 msgr2=0x7f0b9c0a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 0x7f0b9c0a3bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009790 tx=0x7f0ba80096c0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 msgr2=0x7f0bb0072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9a0 tx=0x7f0ba000ed60 comp rx=0 tx=0).stop 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 shutdown_connections 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0b9c0a16f0 0x7f0b9c0a3bb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0106f60 0x7f0bb006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb00682d0 0x7f0bb0072640 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 --2- 172.21.15.31:0/603656133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb0068c90 0x7f0bb0072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 >> 172.21.15.31:0/603656133 conn(0x7f0bb0076b70 msgr2=0x7f0bb0102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 shutdown_connections 2023-10-06T21:36:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:41.076+0000 7f0bb65ff700 1 -- 172.21.15.31:0/603656133 wait complete. 2023-10-06T21:36:41.760 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18141 6.24 MB/sec execute 30 sec latency 344.095 ms 2023-10-06T21:36:42.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:42 smithi031 ceph-mon[98539]: pgmap v496: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2023-10-06T21:36:42.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:42 smithi146 ceph-mon[102809]: pgmap v496: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2023-10-06T21:36:42.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:42 smithi142 ceph-mon[102951]: pgmap v496: 97 pgs: 97 active+clean; 486 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2023-10-06T21:36:42.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18245 6.16 MB/sec execute 31 sec latency 77.167 ms 2023-10-06T21:36:43.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18350 6.12 MB/sec execute 32 sec latency 68.805 ms 2023-10-06T21:36:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:44 smithi031 ceph-mon[98539]: pgmap v497: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 252 op/s 2023-10-06T21:36:44.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:44 smithi146 ceph-mon[102809]: pgmap v497: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 252 op/s 2023-10-06T21:36:44.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:44 smithi142 ceph-mon[102951]: pgmap v497: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 252 op/s 2023-10-06T21:36:44.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18476 6.19 MB/sec execute 33 sec latency 76.107 ms 2023-10-06T21:36:45.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18608 6.32 MB/sec execute 34 sec latency 69.583 ms 2023-10-06T21:36:46.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:46 smithi031 ceph-mon[98539]: pgmap v498: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:46.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:46 smithi146 ceph-mon[102809]: pgmap v498: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:46.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:46 smithi142 ceph-mon[102951]: pgmap v498: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:46.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18740 6.38 MB/sec execute 35 sec latency 64.916 ms 2023-10-06T21:36:47.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18878 6.38 MB/sec execute 36 sec latency 66.477 ms 2023-10-06T21:36:48.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:48 smithi031 ceph-mon[98539]: pgmap v499: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:48.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:48 smithi146 ceph-mon[102809]: pgmap v499: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:48.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:48 smithi142 ceph-mon[102951]: pgmap v499: 97 pgs: 97 active+clean; 480 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2023-10-06T21:36:48.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 18999 6.27 MB/sec execute 37 sec latency 75.086 ms 2023-10-06T21:36:49.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19116 6.22 MB/sec execute 38 sec latency 71.443 ms 2023-10-06T21:36:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:50 smithi142 ceph-mon[102951]: pgmap v500: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2023-10-06T21:36:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:50 smithi031 ceph-mon[98539]: pgmap v500: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2023-10-06T21:36:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:50 smithi146 ceph-mon[102809]: pgmap v500: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2023-10-06T21:36:50.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19236 6.11 MB/sec execute 39 sec latency 66.721 ms 2023-10-06T21:36:51.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19370 6.09 MB/sec execute 40 sec latency 74.821 ms 2023-10-06T21:36:52.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:52 smithi031 ceph-mon[98539]: pgmap v501: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2023-10-06T21:36:52.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:52 smithi146 ceph-mon[102809]: pgmap v501: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2023-10-06T21:36:52.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:52 smithi142 ceph-mon[102951]: pgmap v501: 97 pgs: 97 active+clean; 482 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2023-10-06T21:36:52.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19510 6.10 MB/sec execute 41 sec latency 71.921 ms 2023-10-06T21:36:53.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19630 6.07 MB/sec execute 42 sec latency 76.550 ms 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3504898407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a40ffc40 msgr2=0x7fe1a4103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3504898407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a40ffc40 0x7fe1a4103d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098f0 tx=0x7fe198009c00 comp rx=0 tx=0).stop 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3504898407 shutdown_connections 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3504898407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4104250 0x7fe1a41046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3504898407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a40ff320 0x7fe1a40ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3504898407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a40ffc40 0x7fe1a4103d10 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098f0 tx=0x7fe198009c00 comp rx=0 tx=0).stop 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.313+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3504898407 >> 172.21.15.31:0/3504898407 conn(0x7fe1a4076e70 msgr2=0x7fe1a40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.314+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3504898407 shutdown_connections 2023-10-06T21:36:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.314+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3504898407 wait complete. 2023-10-06T21:36:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.315+0000 7fe1a9bff700 1 Processor -- start 2023-10-06T21:36:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 -- start start 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 0x7fe1a4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a419dfe0 0x7fe1a419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419ea60 con 0x7fe1a419dfe0 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419eba0 con 0x7fe1a4104250 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.316+0000 7fe1a9bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419ed10 con 0x7fe1a40ff320 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58588/0 (socket says 172.21.15.31:58588) 2023-10-06T21:36:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/3801980559 learned_addr learned my addr 172.21.15.31:0/3801980559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 0x7fe1a4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/3801980559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 msgr2=0x7fe1a4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 0x7fe1a4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a419dfe0 msgr2=0x7fe1a419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a419dfe0 0x7fe1a419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/3801980559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198009520 con 0x7fe1a4104250 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.317+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 0x7fe1a4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.318+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe198009230 tx=0x7fe19800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.318+0000 7fe1a0ff9700 1 -- 172.21.15.31:0/3801980559 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19801c070 con 0x7fe1a4104250 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.318+0000 7fe1a0ff9700 1 -- 172.21.15.31:0/3801980559 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe19800bba0 con 0x7fe1a4104250 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.318+0000 7fe1a0ff9700 1 -- 172.21.15.31:0/3801980559 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980038e0 con 0x7fe1a4104250 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.318+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a406a8d0 con 0x7fe1a4104250 2023-10-06T21:36:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.319+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a406acd0 con 0x7fe1a4104250 2023-10-06T21:36:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.323+0000 7fe1a0ff9700 1 -- 172.21.15.31:0/3801980559 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe198020030 con 0x7fe1a4104250 2023-10-06T21:36:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.324+0000 7fe1bd5e5b80 1 Processor -- start 2023-10-06T21:36:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.324+0000 7fe1bd5e5b80 1 -- start start 2023-10-06T21:36:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409f7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409ecf400 con 0x556409f6ed30 2023-10-06T21:36:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409ec0bc0 con 0x556409f79be0 2023-10-06T21:36:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.325+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409ebe4e0 con 0x556409f6e410 2023-10-06T21:36:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43326/0 (socket says 172.21.15.31:43326) 2023-10-06T21:36:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a3fff700 1 -- 172.21.15.31:0/3849421613 learned_addr learned my addr 172.21.15.31:0/3849421613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409f7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 -- 172.21.15.31:0/3849421613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 msgr2=0x556409f6e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 -- 172.21.15.31:0/3849421613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 msgr2=0x556409f7bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409f7bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a0ff9700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 0x7fe18c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.326+0000 7fe1a37fe700 1 -- 172.21.15.31:0/3849421613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556409ce4d80 con 0x556409f6ed30 2023-10-06T21:36:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.327+0000 7fe1a0ff9700 1 -- 172.21.15.31:0/3801980559 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe1980cdfd0 con 0x7fe1a4104250 2023-10-06T21:36:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.327+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.327+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 0x7fe18c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.328+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f796a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe194009420 tx=0x7fe1940099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb61eacac57793ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.328+0000 7fe189ffb700 1 -- 172.21.15.31:0/3849421613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194003c00 con 0x556409f6ed30 2023-10-06T21:36:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.328+0000 7fe189ffb700 1 -- 172.21.15.31:0/3849421613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe19400f070 con 0x556409f6ed30 2023-10-06T21:36:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.328+0000 7fe189ffb700 1 -- 172.21.15.31:0/3849421613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194014630 con 0x556409f6ed30 2023-10-06T21:36:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3849421613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 msgr2=0x556409f796a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f796a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe194009420 tx=0x7fe1940099e0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 0x7fe18c0a3b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009ef0 tx=0x7fe19c009640 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3849421613 shutdown_connections 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409f7bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6e410 0x556409f6e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3849421613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f796a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.329+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3849421613 >> 172.21.15.31:0/3849421613 conn(0x556409edb200 msgr2=0x556409f61ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.330+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3849421613 shutdown_connections 2023-10-06T21:36:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.330+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3849421613 wait complete. 2023-10-06T21:36:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.330+0000 7fe1bd5e5b80 1 -- start start 2023-10-06T21:36:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 Processor -- start 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 0x556409f76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f76ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409eb9dd0 con 0x556409f6e410 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409e566b0 con 0x556409f6ed30 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409e56350 con 0x556409f79be0 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.331+0000 7fe1bd5e5b80 10 client.0 _ll_register_callbacks cb 0x556409cb37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54924/0 (socket says 172.21.15.31:54924) 2023-10-06T21:36:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a37fe700 1 -- 172.21.15.31:0/237397723 learned_addr learned my addr 172.21.15.31:0/237397723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 0x556409f76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f76ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/237397723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 msgr2=0x556409f76ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f76ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/237397723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 msgr2=0x556409f795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a2ffd700 1 -- 172.21.15.31:0/237397723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c009810 con 0x556409f6ed30 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.332+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f76ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 0x556409f76860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1980cb650 tx=0x7fe19809ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd050fe0bac7703 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe182ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19809b670 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe19809af70 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556409f59d50 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19809f0c0 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe1bd5e5b80 10 client.27158 fetch_fsmap learned FSMap version 45 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.333+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fe1940090d0 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556409f59d50 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe1980cb8d0 con 0x556409f6ed30 2023-10-06T21:36:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 10 client.27158 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 10 client.27158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 20 client.27158 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409f89320 0x556409f8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 4 client.27158 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556409eb2db0 con 0x556409f89320 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.334+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409f89320 0x556409f8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.335+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409f89320 0x556409f8b740 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62e1842dfc0a83 server_cookie=3dbd879cd52f4d03 in_seq=0 out_seq=0 2023-10-06T21:36:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.335+0000 7fe182ffd700 10 client.27158 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.336+0000 7fe182ffd700 1 -- 172.21.15.31:0/237397723 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556409eb2db0 con 0x556409f89320 2023-10-06T21:36:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.336+0000 7fe182ffd700 10 client.27158 handle_command_reply: tid=0 2023-10-06T21:36:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.336+0000 7fe1bd5e5b80 1 client.27158 shutdown 2023-10-06T21:36:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 msgr2=0x556409f76860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 0x556409f76860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1980cb650 tx=0x7fe19809ae30 comp rx=0 tx=0).stop 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 shutdown_connections 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f76ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409f89320 0x556409f8b740 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/237397723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6ed30 0x556409f76860 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.337+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 >> 172.21.15.31:0/237397723 conn(0x556409edb200 msgr2=0x556409f61b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.338+0000 7fe182ffd700 0 client.27158 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.338+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 shutdown_connections 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.338+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/237397723 wait complete. 2023-10-06T21:36:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.338+0000 7fe1bd5e5b80 20 client.27158 trim_cache size 0 max 16384 2023-10-06T21:36:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.360+0000 7fe1bd5e5b80 1 Processor -- start 2023-10-06T21:36:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.360+0000 7fe1bd5e5b80 1 -- start start 2023-10-06T21:36:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409d3a8c0 con 0x556409f6ed30 2023-10-06T21:36:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409f636a0 con 0x556409f6e410 2023-10-06T21:36:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409f60e60 con 0x556409f79be0 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43338/0 (socket says 172.21.15.31:43338) 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58614/0 (socket says 172.21.15.31:58614) 2023-10-06T21:36:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.361+0000 7fe1a3fff700 1 -- 172.21.15.31:0/3145143588 learned_addr learned my addr 172.21.15.31:0/3145143588 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 -- 172.21.15.31:0/3145143588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 msgr2=0x556409f79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 -- 172.21.15.31:0/3145143588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 msgr2=0x556409f71b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f71b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 -- 172.21.15.31:0/3145143588 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556409ccdc00 con 0x556409f79be0 2023-10-06T21:36:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a2ffd700 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.362+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe19c01e680 tx=0x7fe19c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8164aec9f029a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.363+0000 7fe18a7fc700 1 -- 172.21.15.31:0/3145143588 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01eee0 con 0x556409f79be0 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.363+0000 7fe18a7fc700 1 -- 172.21.15.31:0/3145143588 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe19c003680 con 0x556409f79be0 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.363+0000 7fe18a7fc700 1 -- 172.21.15.31:0/3145143588 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01f6f0 con 0x556409f79be0 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3145143588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 msgr2=0x556409f72080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe19c01e680 tx=0x7fe19c014000 comp rx=0 tx=0).stop 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3145143588 shutdown_connections 2023-10-06T21:36:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f79be0 0x556409f72080 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f6e410 0x556409f79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/3145143588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6ed30 0x556409f71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3145143588 >> 172.21.15.31:0/3145143588 conn(0x556409edb200 msgr2=0x556409f8b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3145143588 shutdown_connections 2023-10-06T21:36:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.364+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/3145143588 wait complete. 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 -- start start 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 Processor -- start 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f7e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6ed30 0x556409f7e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409f7ee90 con 0x556409f6e410 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409fdc690 con 0x556409f79be0 2023-10-06T21:36:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.365+0000 7fe1bd5e5b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556409fdc7d0 con 0x556409f6ed30 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.366+0000 7fe1bd5e5b80 10 client.0 _ll_register_callbacks cb 0x556409cb37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.366+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.366+0000 7fe1a3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58620/0 (socket says 172.21.15.31:58620) 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.366+0000 7fe1a3fff700 1 -- 172.21.15.31:0/1774472197 learned_addr learned my addr 172.21.15.31:0/1774472197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.366+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 -- 172.21.15.31:0/1774472197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6ed30 msgr2=0x556409f7e640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6ed30 0x556409f7e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 -- 172.21.15.31:0/1774472197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 msgr2=0x556409f7e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f7e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 -- 172.21.15.31:0/1774472197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556409ce4d80 con 0x556409f79be0 2023-10-06T21:36:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1a3fff700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe19c003df0 tx=0x7fe19c003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca0df39b8beb7fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe182ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:36:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c02e070 con 0x556409f79be0 2023-10-06T21:36:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe19400f070 con 0x556409f79be0 2023-10-06T21:36:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.367+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe19c0038f0 con 0x556409f79be0 2023-10-06T21:36:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.368+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19c003ab0 con 0x556409f79be0 2023-10-06T21:36:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.368+0000 7fe1bd5e5b80 10 client.27164 fetch_fsmap learned FSMap version 45 2023-10-06T21:36:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.368+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fe1940090d0 con 0x556409f79be0 2023-10-06T21:36:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.368+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe19400f070 con 0x556409f79be0 2023-10-06T21:36:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe19c028900 con 0x556409f79be0 2023-10-06T21:36:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 10 client.27164 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:36:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:36:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 20 client.27164 populate_metadata read hostname 'smithi031' 2023-10-06T21:36:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409fddea0 0x556409fe02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 4 client.27164 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:36:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.369+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556409eb2ad0 con 0x556409fddea0 2023-10-06T21:36:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.370+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409fddea0 0x556409fe02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.371+0000 7fe1a37fe700 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409fddea0 0x556409fe02c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177d0fd3c94234a server_cookie=4d5edaf9677a4e95 in_seq=0 out_seq=0 2023-10-06T21:36:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.371+0000 7fe182ffd700 10 client.27164 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.411+0000 7fe182ffd700 1 -- 172.21.15.31:0/1774472197 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556409eb2ad0 con 0x556409fddea0 2023-10-06T21:36:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.411+0000 7fe182ffd700 10 client.27164 handle_command_reply: tid=0 2023-10-06T21:36:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.411+0000 7fe1bd5e5b80 1 client.27164 shutdown 2023-10-06T21:36:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 msgr2=0x556409fdc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe19c003df0 tx=0x7fe19c003e20 comp rx=0 tx=0).stop 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 shutdown_connections 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556409f79be0 0x556409fdc150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556409fddea0 0x556409fe02c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556409f6e410 0x556409f7e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 --2- 172.21.15.31:0/1774472197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556409f6ed30 0x556409f7e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 >> 172.21.15.31:0/1774472197 conn(0x556409edb200 msgr2=0x556409f71b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.412+0000 7fe182ffd700 0 client.27164 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:36:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.413+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 shutdown_connections 2023-10-06T21:36:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.413+0000 7fe1bd5e5b80 1 -- 172.21.15.31:0/1774472197 wait complete. 2023-10-06T21:36:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.413+0000 7fe1bd5e5b80 20 client.27164 trim_cache size 0 max 16384 2023-10-06T21:36:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:36:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:36:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:36:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:36:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.414+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 msgr2=0x7fe18c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.414+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 0x7fe18c0a3b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009ef0 tx=0x7fe19c009640 comp rx=0 tx=0).stop 2023-10-06T21:36:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.414+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 msgr2=0x7fe1a4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.414+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe198009230 tx=0x7fe19800fa00 comp rx=0 tx=0).stop 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 shutdown_connections 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe18c0a16b0 0x7fe18c0a3b70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1a419dfe0 0x7fe1a419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a40ff320 0x7fe1a4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 --2- 172.21.15.31:0/3801980559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4104250 0x7fe1a4199970 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.415+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 >> 172.21.15.31:0/3801980559 conn(0x7fe1a4076e70 msgr2=0x7fe1a4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.417+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 shutdown_connections 2023-10-06T21:36:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:36:54.417+0000 7fe1a9bff700 1 -- 172.21.15.31:0/3801980559 wait complete. 2023-10-06T21:36:54.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:54 smithi031 ceph-mon[98539]: pgmap v502: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-10-06T21:36:54.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:54 smithi146 ceph-mon[102809]: pgmap v502: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-10-06T21:36:54.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:54 smithi142 ceph-mon[102951]: pgmap v502: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-10-06T21:36:54.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19742 6.09 MB/sec execute 43 sec latency 103.101 ms 2023-10-06T21:36:55.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 19870 6.14 MB/sec execute 44 sec latency 75.638 ms 2023-10-06T21:36:56.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:56 smithi031 ceph-mon[98539]: pgmap v503: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:56.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:56 smithi146 ceph-mon[102809]: pgmap v503: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:56 smithi142 ceph-mon[102951]: pgmap v503: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:56.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20039 6.27 MB/sec execute 45 sec latency 60.864 ms 2023-10-06T21:36:57.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20202 6.33 MB/sec execute 46 sec latency 68.815 ms 2023-10-06T21:36:58.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:36:58 smithi031 ceph-mon[98539]: pgmap v504: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:58.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:36:58 smithi146 ceph-mon[102809]: pgmap v504: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:36:58 smithi142 ceph-mon[102951]: pgmap v504: 97 pgs: 97 active+clean; 484 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2023-10-06T21:36:58.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20359 6.48 MB/sec execute 47 sec latency 80.947 ms 2023-10-06T21:36:59.761 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20463 6.41 MB/sec execute 48 sec latency 62.440 ms 2023-10-06T21:37:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:00 smithi031 ceph-mon[98539]: pgmap v505: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2023-10-06T21:37:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:00 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:00 smithi146 ceph-mon[102809]: pgmap v505: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2023-10-06T21:37:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:00 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:00 smithi142 ceph-mon[102951]: pgmap v505: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2023-10-06T21:37:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:00 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:00.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20583 6.49 MB/sec execute 49 sec latency 74.975 ms 2023-10-06T21:37:01.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20720 6.59 MB/sec execute 50 sec latency 105.057 ms 2023-10-06T21:37:02.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:02 smithi031 ceph-mon[98539]: pgmap v506: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-10-06T21:37:02.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:02 smithi146 ceph-mon[102809]: pgmap v506: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-10-06T21:37:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:02 smithi142 ceph-mon[102951]: pgmap v506: 97 pgs: 97 active+clean; 494 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-10-06T21:37:02.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20839 6.60 MB/sec execute 51 sec latency 77.446 ms 2023-10-06T21:37:03.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 20962 6.67 MB/sec execute 52 sec latency 73.123 ms 2023-10-06T21:37:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:04 smithi031 ceph-mon[98539]: pgmap v507: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2023-10-06T21:37:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:04 smithi146 ceph-mon[102809]: pgmap v507: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2023-10-06T21:37:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:04 smithi142 ceph-mon[102951]: pgmap v507: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2023-10-06T21:37:04.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21065 6.61 MB/sec execute 53 sec latency 102.456 ms 2023-10-06T21:37:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:05 smithi031 ceph-mon[98539]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:37:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:05 smithi146 ceph-mon[102809]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:37:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:05 smithi142 ceph-mon[102951]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:37:05.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21192 6.66 MB/sec execute 54 sec latency 64.463 ms 2023-10-06T21:37:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:06 smithi031 ceph-mon[98539]: pgmap v509: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:06.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:06 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:06 smithi146 ceph-mon[102809]: pgmap v509: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:06.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:06 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:06 smithi142 ceph-mon[102951]: pgmap v509: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:06.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:06 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:06.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21271 6.57 MB/sec execute 55 sec latency 152.974 ms 2023-10-06T21:37:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 -- 172.21.15.31:0/2030782813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073280 msgr2=0x7f6618073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 --2- 172.21.15.31:0/2030782813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073280 0x7f6618073660 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f660c009990 tx=0x7f660c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 -- 172.21.15.31:0/2030782813 shutdown_connections 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 --2- 172.21.15.31:0/2030782813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f661810d730 0x7f661810fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 --2- 172.21.15.31:0/2030782813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073280 0x7f6618073660 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 --2- 172.21.15.31:0/2030782813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6618073ba0 0x7f661810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 -- 172.21.15.31:0/2030782813 >> 172.21.15.31:0/2030782813 conn(0x7f66180fc9a0 msgr2=0x7f66180fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 -- 172.21.15.31:0/2030782813 shutdown_connections 2023-10-06T21:37:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.651+0000 7f661ebff700 1 -- 172.21.15.31:0/2030782813 wait complete. 2023-10-06T21:37:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.653+0000 7f661ebff700 1 Processor -- start 2023-10-06T21:37:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.653+0000 7f661ebff700 1 -- start start 2023-10-06T21:37:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073ba0 0x7f6618072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f661810d730 0x7f661806b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806bba0 con 0x7f6618073ba0 2023-10-06T21:37:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806bd10 con 0x7f6618073280 2023-10-06T21:37:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661ebff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806bed0 con 0x7f661810d730 2023-10-06T21:37:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661c99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661c99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41650/0 (socket says 172.21.15.31:41650) 2023-10-06T21:37:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661c99b700 1 -- 172.21.15.31:0/2558445402 learned_addr learned my addr 172.21.15.31:0/2558445402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.654+0000 7f661d19c700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f661810d730 0x7f661806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 -- 172.21.15.31:0/2558445402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f661810d730 msgr2=0x7f661806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f661810d730 0x7f661806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 -- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073ba0 msgr2=0x7f6618072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073ba0 0x7f6618072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 -- 172.21.15.31:0/2558445402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604009520 con 0x7f6618073280 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.655+0000 7f661c99b700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f660c005740 tx=0x7f660c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.656+0000 7f6615ffb700 1 -- 172.21.15.31:0/2558445402 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01d070 con 0x7f6618073280 2023-10-06T21:37:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.656+0000 7f6615ffb700 1 -- 172.21.15.31:0/2558445402 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f660c021410 con 0x7f6618073280 2023-10-06T21:37:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.656+0000 7f6615ffb700 1 -- 172.21.15.31:0/2558445402 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c017650 con 0x7f6618073280 2023-10-06T21:37:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.656+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f6618073280 2023-10-06T21:37:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.657+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618070220 con 0x7f6618073280 2023-10-06T21:37:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.662+0000 7f6615ffb700 1 -- 172.21.15.31:0/2558445402 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f660c00fb00 con 0x7f6618073280 2023-10-06T21:37:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.663+0000 7f6615ffb700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 0x7f66080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.663+0000 7f6615ffb700 1 -- 172.21.15.31:0/2558445402 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f660c0ce340 con 0x7f6618073280 2023-10-06T21:37:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.664+0000 7f663254cb80 1 Processor -- start 2023-10-06T21:37:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.664+0000 7f6617fff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 0x7f66080a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.664+0000 7f663254cb80 1 -- start start 2023-10-06T21:37:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5be87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 0x5609a5bf36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5b49400 con 0x5609a5bf3be0 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5b3abc0 con 0x5609a5be8410 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f663254cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5b384e0 con 0x5609a5be8d30 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f6617fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 0x5609a5bf36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f6617fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 0x5609a5bf36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:34468/0 (socket says 172.21.15.31:34468) 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f6617fff700 1 -- 172.21.15.31:0/2081484020 learned_addr learned my addr 172.21.15.31:0/2081484020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f6617fff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 0x7f66080a3b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d90 tx=0x7f6604005ce0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.665+0000 7f661c99b700 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5be87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 -- 172.21.15.31:0/2081484020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 msgr2=0x5609a5bf36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 0x5609a5bf36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 -- 172.21.15.31:0/2081484020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 msgr2=0x5609a5be87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5be87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.666+0000 7f661d19c700 1 -- 172.21.15.31:0/2081484020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a595ed80 con 0x5609a5bf3be0 2023-10-06T21:37:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.667+0000 7f661d19c700 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf5fd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6610011550 tx=0x7f6610009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ceeb15b21f4d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.667+0000 7f6602ffd700 1 -- 172.21.15.31:0/2081484020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661000e040 con 0x5609a5bf3be0 2023-10-06T21:37:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.667+0000 7f6602ffd700 1 -- 172.21.15.31:0/2081484020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f661000f040 con 0x5609a5bf3be0 2023-10-06T21:37:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.667+0000 7f6602ffd700 1 -- 172.21.15.31:0/2081484020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610004360 con 0x5609a5bf3be0 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.668+0000 7f663254cb80 1 -- 172.21.15.31:0/2081484020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 msgr2=0x5609a5bf5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.668+0000 7f663254cb80 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf5fd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6610011550 tx=0x7f6610009920 comp rx=0 tx=0).stop 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 -- 172.21.15.31:0/2081484020 shutdown_connections 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf5fd0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5be87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 --2- 172.21.15.31:0/2081484020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8d30 0x5609a5bf36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 -- 172.21.15.31:0/2081484020 >> 172.21.15.31:0/2081484020 conn(0x5609a5b55200 msgr2=0x5609a5bdbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 -- 172.21.15.31:0/2081484020 shutdown_connections 2023-10-06T21:37:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.669+0000 7f663254cb80 1 -- 172.21.15.31:0/2081484020 wait complete. 2023-10-06T21:37:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.670+0000 7f663254cb80 1 -- start start 2023-10-06T21:37:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.670+0000 7f663254cb80 1 Processor -- start 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 0x5609a5bf09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5bf0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5b33dd0 con 0x5609a5bf3be0 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5ad06b0 con 0x5609a5be8d30 2023-10-06T21:37:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5ad0350 con 0x5609a5be8410 2023-10-06T21:37:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.671+0000 7f663254cb80 10 client.0 _ll_register_callbacks cb 0x5609a592d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f661c99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 0x5609a5bf09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f661c99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 0x5609a5bf09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:34484/0 (socket says 172.21.15.31:34484) 2023-10-06T21:37:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f661c99b700 1 -- 172.21.15.31:0/2286430869 learned_addr learned my addr 172.21.15.31:0/2286430869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5bf0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 -- 172.21.15.31:0/2286430869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 msgr2=0x5609a5bf09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 0x5609a5bf09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 -- 172.21.15.31:0/2286430869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 msgr2=0x5609a5bf1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.672+0000 7f6617fff700 1 -- 172.21.15.31:0/2286430869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610011230 con 0x5609a5be8d30 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.673+0000 7f6617fff700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5bf0f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6604010710 tx=0x7f6604004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fe515ad5278a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.673+0000 7f65f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.673+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660401d640 con 0x5609a5be8d30 2023-10-06T21:37:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.673+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f660401d7a0 con 0x5609a5be8d30 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.673+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a5bd3d50 con 0x5609a5be8d30 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660401d960 con 0x5609a5be8d30 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 10 client.27176 fetch_fsmap learned FSMap version 47 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5609a5947c00 con 0x5609a5be8d30 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6604031020 con 0x5609a5be8d30 2023-10-06T21:37:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f660402b020 con 0x5609a5be8d30 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 10 client.27176 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 10 client.27176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 20 client.27176 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c03900 0x5609a5c05d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 4 client.27176 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.674+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a5b2cdb0 con 0x5609a5c03900 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.675+0000 7f661c99b700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c03900 0x5609a5c05d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.675+0000 7f65f7fff700 10 client.27176 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.675+0000 7f661c99b700 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c03900 0x5609a5c05d20 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b61036b8ecd6a7f server_cookie=a4c7437f192635ec in_seq=0 out_seq=0 2023-10-06T21:37:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.676+0000 7f65f7fff700 1 -- 172.21.15.31:0/2286430869 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609a5b2cdb0 con 0x5609a5c03900 2023-10-06T21:37:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.676+0000 7f65f7fff700 10 client.27176 handle_command_reply: tid=0 2023-10-06T21:37:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.676+0000 7f663254cb80 1 client.27176 shutdown 2023-10-06T21:37:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 msgr2=0x5609a5bf0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5bf0f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6604010710 tx=0x7f6604004750 comp rx=0 tx=0).stop 2023-10-06T21:37:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 shutdown_connections 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c03900 0x5609a5c05d20 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5bf3be0 0x5609a5bf1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5be8410 0x5609a5bf09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 --2- 172.21.15.31:0/2286430869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5bf0f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 >> 172.21.15.31:0/2286430869 conn(0x5609a5b55200 msgr2=0x5609a5bdbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.679+0000 7f65f7fff700 0 client.27176 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.680+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 shutdown_connections 2023-10-06T21:37:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.680+0000 7f663254cb80 1 -- 172.21.15.31:0/2286430869 wait complete. 2023-10-06T21:37:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.680+0000 7f663254cb80 20 client.27176 trim_cache size 0 max 16384 2023-10-06T21:37:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.709+0000 7f663254cb80 1 Processor -- start 2023-10-06T21:37:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.709+0000 7f663254cb80 1 -- start start 2023-10-06T21:37:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5be8410 0x5609a5bfa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5bf11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a59b48c0 con 0x5609a5be8410 2023-10-06T21:37:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5bdce20 con 0x5609a5be8d30 2023-10-06T21:37:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f663254cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5bfacf0 con 0x5609a5bf3be0 2023-10-06T21:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f6617fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f661c99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5be8410 0x5609a5bfa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f6617fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41662/0 (socket says 172.21.15.31:41662) 2023-10-06T21:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.710+0000 7f6617fff700 1 -- 172.21.15.31:0/2010483140 learned_addr learned my addr 172.21.15.31:0/2010483140 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f661d19c700 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5bf11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 -- 172.21.15.31:0/2010483140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 msgr2=0x5609a5bf11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5bf11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 -- 172.21.15.31:0/2010483140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5be8410 msgr2=0x5609a5bfa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5be8410 0x5609a5bfa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 -- 172.21.15.31:0/2010483140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a59851f0 con 0x5609a5be8d30 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f6617fff700 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6604026f90 tx=0x7f660402d400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205ce768e54957b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.711+0000 7f66037fe700 1 -- 172.21.15.31:0/2010483140 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604010db0 con 0x5609a5be8d30 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.712+0000 7f66037fe700 1 -- 172.21.15.31:0/2010483140 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6604031040 con 0x5609a5be8d30 2023-10-06T21:37:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.712+0000 7f66037fe700 1 -- 172.21.15.31:0/2010483140 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604005020 con 0x5609a5be8d30 2023-10-06T21:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.712+0000 7f663254cb80 1 -- 172.21.15.31:0/2010483140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 msgr2=0x5609a5beb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.712+0000 7f663254cb80 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6604026f90 tx=0x7f660402d400 comp rx=0 tx=0).stop 2023-10-06T21:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 -- 172.21.15.31:0/2010483140 shutdown_connections 2023-10-06T21:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5bf11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5be8410 0x5609a5bfa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 --2- 172.21.15.31:0/2010483140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8d30 0x5609a5beb2b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 -- 172.21.15.31:0/2010483140 >> 172.21.15.31:0/2010483140 conn(0x5609a5b55200 msgr2=0x5609a5c05910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 -- 172.21.15.31:0/2010483140 shutdown_connections 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.713+0000 7f663254cb80 1 -- 172.21.15.31:0/2010483140 wait complete. 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.714+0000 7f663254cb80 1 -- start start 2023-10-06T21:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.714+0000 7f663254cb80 1 Processor -- start 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5c4bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5c4c140 0x5609a5c56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5c56c80 con 0x5609a5c4c140 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5c56dc0 con 0x5609a5be8410 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a5c56f30 con 0x5609a5bf3be0 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f663254cb80 10 client.0 _ll_register_callbacks cb 0x7f660400eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f661c99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f661c99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41672/0 (socket says 172.21.15.31:41672) 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f661c99b700 1 -- 172.21.15.31:0/2649458305 learned_addr learned my addr 172.21.15.31:0/2649458305 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f6617fff700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5c4bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.715+0000 7f661d19c700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5c4c140 0x5609a5c56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 -- 172.21.15.31:0/2649458305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 msgr2=0x5609a5c4bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5c4bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 -- 172.21.15.31:0/2649458305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5c4c140 msgr2=0x5609a5c56740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5c4c140 0x5609a5c56740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 -- 172.21.15.31:0/2649458305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a5947c00 con 0x5609a5be8410 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f661c99b700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f660c0cbaa0 tx=0x7f660c021a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c4831c51e7ed37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f65f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c09b0e0 con 0x5609a5be8410 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f660c09ba10 con 0x5609a5be8410 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.716+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f661000f040 con 0x5609a5be8410 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c09b310 con 0x5609a5be8410 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f663254cb80 10 client.27182 fetch_fsmap learned FSMap version 47 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5609a595d250 con 0x5609a5be8410 2023-10-06T21:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f661000f040 con 0x5609a5be8410 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f660c09f120 con 0x5609a5be8410 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f663254cb80 10 client.27182 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f663254cb80 10 client.27182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.717+0000 7f663254cb80 20 client.27182 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.718+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c583d0 0x5609a5c5a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.718+0000 7f663254cb80 4 client.27182 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.718+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609a5b2cad0 con 0x5609a5c583d0 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.718+0000 7f6617fff700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c583d0 0x5609a5c5a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.719+0000 7f6617fff700 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c583d0 0x5609a5c5a7f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a18f33d2fafd93c server_cookie=6e4fc9e537624497 in_seq=0 out_seq=0 2023-10-06T21:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.719+0000 7f65f7fff700 10 client.27182 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:07.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21344 6.50 MB/sec execute 56 sec latency 331.834 ms 2023-10-06T21:37:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.768+0000 7f65f7fff700 1 -- 172.21.15.31:0/2649458305 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609a5b2cad0 con 0x5609a5c583d0 2023-10-06T21:37:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.768+0000 7f65f7fff700 10 client.27182 handle_command_reply: tid=0 2023-10-06T21:37:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.768+0000 7f663254cb80 1 client.27182 shutdown 2023-10-06T21:37:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 msgr2=0x5609a5c4b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f660c0cbaa0 tx=0x7f660c021a60 comp rx=0 tx=0).stop 2023-10-06T21:37:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 shutdown_connections 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5609a5c583d0 0x5609a5c5a7f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609a5c4c140 0x5609a5c56740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a5be8410 0x5609a5c4b6c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 --2- 172.21.15.31:0/2649458305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a5bf3be0 0x5609a5c4bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 >> 172.21.15.31:0/2649458305 conn(0x5609a5b55200 msgr2=0x5609a5bff980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.769+0000 7f65f7fff700 0 client.27182 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.770+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 shutdown_connections 2023-10-06T21:37:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.770+0000 7f663254cb80 1 -- 172.21.15.31:0/2649458305 wait complete. 2023-10-06T21:37:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.770+0000 7f663254cb80 20 client.27182 trim_cache size 0 max 16384 2023-10-06T21:37:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:37:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:37:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:37:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:37:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 msgr2=0x7f66080a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 0x7f66080a3b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d90 tx=0x7f6604005ce0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 msgr2=0x7f66180724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f660c005740 tx=0x7f660c00bfd0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 shutdown_connections 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f661810d730 0x7f661806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6618073280 0x7f66180724b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f66080a1670 0x7f66080a3b30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 --2- 172.21.15.31:0/2558445402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6618073ba0 0x7f6618072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.771+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 >> 172.21.15.31:0/2558445402 conn(0x7f66180fc9a0 msgr2=0x7f661810f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.772+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 shutdown_connections 2023-10-06T21:37:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:07.772+0000 7f661ebff700 1 -- 172.21.15.31:0/2558445402 wait complete. 2023-10-06T21:37:08.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:08 smithi031 ceph-mon[98539]: pgmap v510: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:08.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:08 smithi146 ceph-mon[102809]: pgmap v510: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:08 smithi142 ceph-mon[102951]: pgmap v510: 97 pgs: 97 active+clean; 506 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2023-10-06T21:37:08.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21432 6.40 MB/sec execute 57 sec latency 88.595 ms 2023-10-06T21:37:09.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21546 6.38 MB/sec execute 58 sec latency 81.163 ms 2023-10-06T21:37:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:10 smithi142 ceph-mon[102951]: pgmap v511: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:10 smithi031 ceph-mon[98539]: pgmap v511: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:10 smithi146 ceph-mon[102809]: pgmap v511: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:10.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21684 6.39 MB/sec execute 59 sec latency 88.052 ms 2023-10-06T21:37:11.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21821 6.39 MB/sec execute 60 sec latency 68.361 ms 2023-10-06T21:37:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:12 smithi031 ceph-mon[98539]: pgmap v512: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:12.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:12 smithi146 ceph-mon[102809]: pgmap v512: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:12.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:12 smithi142 ceph-mon[102951]: pgmap v512: 97 pgs: 97 active+clean; 519 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2023-10-06T21:37:12.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 21944 6.39 MB/sec execute 61 sec latency 76.481 ms 2023-10-06T21:37:13.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22064 6.39 MB/sec execute 62 sec latency 72.111 ms 2023-10-06T21:37:14.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:14 smithi031 ceph-mon[98539]: pgmap v513: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:14.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:14 smithi031 ceph-mon[98539]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:37:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:14 smithi146 ceph-mon[102809]: pgmap v513: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:14.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:14 smithi146 ceph-mon[102809]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:37:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:14 smithi142 ceph-mon[102951]: pgmap v513: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:14 smithi142 ceph-mon[102951]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:37:14.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22178 6.39 MB/sec execute 63 sec latency 79.204 ms 2023-10-06T21:37:15.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22303 6.42 MB/sec execute 64 sec latency 67.103 ms 2023-10-06T21:37:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:16 smithi031 ceph-mon[98539]: pgmap v515: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:16 smithi146 ceph-mon[102809]: pgmap v515: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:16 smithi142 ceph-mon[102951]: pgmap v515: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:16.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22427 6.39 MB/sec execute 65 sec latency 72.428 ms 2023-10-06T21:37:17.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22557 6.37 MB/sec execute 66 sec latency 74.640 ms 2023-10-06T21:37:18.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:18 smithi031 ceph-mon[98539]: pgmap v516: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:18 smithi146 ceph-mon[102809]: pgmap v516: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:18 smithi142 ceph-mon[102951]: pgmap v516: 97 pgs: 97 active+clean; 528 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2023-10-06T21:37:18.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22685 6.36 MB/sec execute 67 sec latency 73.588 ms 2023-10-06T21:37:19.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22805 6.30 MB/sec execute 68 sec latency 82.426 ms 2023-10-06T21:37:20.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:20 smithi142 ceph-mon[102951]: pgmap v517: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:20 smithi031 ceph-mon[98539]: pgmap v517: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:20.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:20 smithi146 ceph-mon[102809]: pgmap v517: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:20.762 INFO:tasks.workunit.client.0.smithi031.stdout: 10 22900 6.27 MB/sec execute 69 sec latency 401.433 ms 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 -- 172.21.15.31:0/2644561450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994105e20 msgr2=0x7f99940682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2644561450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994105e20 0x7f99940682d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9990009990 tx=0x7f9990009ca0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 -- 172.21.15.31:0/2644561450 shutdown_connections 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2644561450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99941038b0 0x7f9994103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2644561450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994105e20 0x7f99940682d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2644561450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9994068810 0x7f9994068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 -- 172.21.15.31:0/2644561450 >> 172.21.15.31:0/2644561450 conn(0x7f9994076b70 msgr2=0x7f9994076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 -- 172.21.15.31:0/2644561450 shutdown_connections 2023-10-06T21:37:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.006+0000 7f999c3ff700 1 -- 172.21.15.31:0/2644561450 wait complete. 2023-10-06T21:37:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.008+0000 7f999c3ff700 1 Processor -- start 2023-10-06T21:37:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.008+0000 7f999c3ff700 1 -- start start 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99941038b0 0x7f999419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9994105e20 0x7f99941a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941a2620 con 0x7f9994105e20 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941a2790 con 0x7f99941038b0 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999c3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941a2950 con 0x7f9994068810 2023-10-06T21:37:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999a19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999a19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46258/0 (socket says 172.21.15.31:46258) 2023-10-06T21:37:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999a19b700 1 -- 172.21.15.31:0/2486215748 learned_addr learned my addr 172.21.15.31:0/2486215748 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.009+0000 7f999a99c700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9994105e20 0x7f99941a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 -- 172.21.15.31:0/2486215748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99941038b0 msgr2=0x7f999419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99941038b0 0x7f999419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 -- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9994105e20 msgr2=0x7f99941a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9994105e20 0x7f99941a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 -- 172.21.15.31:0/2486215748 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99900095f0 con 0x7f9994068810 2023-10-06T21:37:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.010+0000 7f999a19b700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f9990004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.011+0000 7f998b7fe700 1 -- 172.21.15.31:0/2486215748 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999001d070 con 0x7f9994068810 2023-10-06T21:37:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.011+0000 7f998b7fe700 1 -- 172.21.15.31:0/2486215748 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9990022470 con 0x7f9994068810 2023-10-06T21:37:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.011+0000 7f998b7fe700 1 -- 172.21.15.31:0/2486215748 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990017650 con 0x7f9994068810 2023-10-06T21:37:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.011+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99941a2c30 con 0x7f9994068810 2023-10-06T21:37:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.012+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99941aa470 con 0x7f9994068810 2023-10-06T21:37:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.016+0000 7f998b7fe700 1 -- 172.21.15.31:0/2486215748 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f99900225e0 con 0x7f9994068810 2023-10-06T21:37:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.018+0000 7f99afda8b80 1 Processor -- start 2023-10-06T21:37:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.019+0000 7f99afda8b80 1 -- start start 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.019+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b57cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.019+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b58a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f99afda8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b4e6400 con 0x55650b57d520 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f99afda8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b4d7bc0 con 0x55650b5883d0 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f99afda8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b4d54e0 con 0x55650b57cc00 2023-10-06T21:37:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f998b7fe700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 0x7f99800a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f998b7fe700 1 -- 172.21.15.31:0/2486215748 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f99900ce630 con 0x7f9994068810 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f999a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f999a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50798/0 (socket says 172.21.15.31:50798) 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f999a19b700 1 -- 172.21.15.31:0/3019755565 learned_addr learned my addr 172.21.15.31:0/3019755565 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.020+0000 7f999a99c700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b57cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999999a700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b58a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a19b700 1 -- 172.21.15.31:0/3019755565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 msgr2=0x55650b57cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a19b700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b57cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a19b700 1 -- 172.21.15.31:0/3019755565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 msgr2=0x55650b58a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a19b700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b58a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a19b700 1 -- 172.21.15.31:0/3019755565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650b2fbd80 con 0x55650b57d520 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999999a700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b58a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a99c700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b57cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.021+0000 7f999a99c700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 0x7f99800a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.022+0000 7f999a19b700 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f999000f740 tx=0x7f9990097910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e11224d75a8ee0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.022+0000 7f9988bfe700 1 -- 172.21.15.31:0/3019755565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999009af30 con 0x55650b57d520 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.022+0000 7f9988bfe700 1 -- 172.21.15.31:0/3019755565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99900228c0 con 0x55650b57d520 2023-10-06T21:37:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.022+0000 7f9988bfe700 1 -- 172.21.15.31:0/3019755565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990075e60 con 0x55650b57d520 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 -- 172.21.15.31:0/3019755565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 msgr2=0x55650b587e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f999a99c700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 0x7f99800a3c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f998c00bb40 tx=0x7f998c0095a0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f999000f740 tx=0x7f9990097910 comp rx=0 tx=0).stop 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 -- 172.21.15.31:0/3019755565 shutdown_connections 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b58a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b57cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 --2- 172.21.15.31:0/3019755565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b587e90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.023+0000 7f99afda8b80 1 -- 172.21.15.31:0/3019755565 >> 172.21.15.31:0/3019755565 conn(0x55650b4f2200 msgr2=0x55650b579080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.024+0000 7f99afda8b80 1 -- 172.21.15.31:0/3019755565 shutdown_connections 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.024+0000 7f99afda8b80 1 -- 172.21.15.31:0/3019755565 wait complete. 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.024+0000 7f99afda8b80 1 -- start start 2023-10-06T21:37:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.025+0000 7f99afda8b80 1 Processor -- start 2023-10-06T21:37:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b5887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b589230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b4d0dd0 con 0x55650b5883d0 2023-10-06T21:37:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b46d6b0 con 0x55650b57d520 2023-10-06T21:37:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b46d350 con 0x55650b57cc00 2023-10-06T21:37:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f99afda8b80 10 client.0 _ll_register_callbacks cb 0x55650b2ca7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f999999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f999999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58446/0 (socket says 172.21.15.31:58446) 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.026+0000 7f999999a700 1 -- 172.21.15.31:0/805851058 learned_addr learned my addr 172.21.15.31:0/805851058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999a19b700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b5887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 -- 172.21.15.31:0/805851058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 msgr2=0x55650b5887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b5887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 -- 172.21.15.31:0/805851058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 msgr2=0x55650b589230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b589230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 -- 172.21.15.31:0/805851058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c009cf0 con 0x55650b57d520 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f999999a700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99840117b0 tx=0x7f9984011b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979173c13e49e173 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f997dffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.027+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400f7f0 con 0x55650b57d520 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55650b570d50 con 0x55650b57d520 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f998400f950 con 0x55650b57d520 2023-10-06T21:37:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f998400fb10 con 0x55650b57d520 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 10 client.27191 fetch_fsmap learned FSMap version 47 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f9984009880 con 0x55650b57d520 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55650b570d50 con 0x55650b57d520 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f99840183b0 con 0x55650b57d520 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 10 client.27191 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 10 client.27191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.028+0000 7f99afda8b80 20 client.27191 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b597ce0 0x55650b59a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f99afda8b80 4 client.27191 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:37:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55650b4c9db0 con 0x55650b597ce0 2023-10-06T21:37:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f999a19b700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b597ce0 0x55650b59a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f999a19b700 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b597ce0 0x55650b59a100 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9293cb39ffe7327 server_cookie=3906cc01f65cbc5f in_seq=0 out_seq=0 2023-10-06T21:37:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.029+0000 7f997dffb700 10 client.27191 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.030+0000 7f997dffb700 1 -- 172.21.15.31:0/805851058 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55650b4c9db0 con 0x55650b597ce0 2023-10-06T21:37:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.030+0000 7f997dffb700 10 client.27191 handle_command_reply: tid=0 2023-10-06T21:37:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.030+0000 7f99afda8b80 1 client.27191 shutdown 2023-10-06T21:37:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 msgr2=0x55650b588cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99840117b0 tx=0x7f9984011b70 comp rx=0 tx=0).stop 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 shutdown_connections 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b597ce0 0x55650b59a100 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b589230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b5887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 --2- 172.21.15.31:0/805851058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b588cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 >> 172.21.15.31:0/805851058 conn(0x55650b4f2200 msgr2=0x55650b578c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f997dffb700 0 client.27191 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 shutdown_connections 2023-10-06T21:37:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.031+0000 7f99afda8b80 1 -- 172.21.15.31:0/805851058 wait complete. 2023-10-06T21:37:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.032+0000 7f99afda8b80 20 client.27191 trim_cache size 0 max 16384 2023-10-06T21:37:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 Processor -- start 2023-10-06T21:37:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 -- start start 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b581ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b3518c0 con 0x55650b5883d0 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b58cd50 con 0x55650b57d520 2023-10-06T21:37:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.058+0000 7f99afda8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b585a10 con 0x55650b57cc00 2023-10-06T21:37:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999a99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50822/0 (socket says 172.21.15.31:50822) 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b581ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999a99c700 1 -- 172.21.15.31:0/1307090358 learned_addr learned my addr 172.21.15.31:0/1307090358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999a19b700 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.059+0000 7f999999a700 1 -- 172.21.15.31:0/1307090358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 msgr2=0x55650b58c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999999a700 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999999a700 1 -- 172.21.15.31:0/1307090358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 msgr2=0x55650b593ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999999a700 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999999a700 1 -- 172.21.15.31:0/1307090358 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99941aa470 con 0x55650b57d520 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999a99c700 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f999999a700 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b581ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9984018710 tx=0x7f9984005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676e57545c96ec9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f99893ff700 1 -- 172.21.15.31:0/1307090358 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984018980 con 0x55650b57d520 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f99893ff700 1 -- 172.21.15.31:0/1307090358 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9984004030 con 0x55650b57d520 2023-10-06T21:37:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.060+0000 7f99893ff700 1 -- 172.21.15.31:0/1307090358 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400fa50 con 0x55650b57d520 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.061+0000 7f99afda8b80 1 -- 172.21.15.31:0/1307090358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 msgr2=0x55650b581ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.061+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b581ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9984018710 tx=0x7f9984005d60 comp rx=0 tx=0).stop 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.061+0000 7f99afda8b80 1 -- 172.21.15.31:0/1307090358 shutdown_connections 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.061+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b5883d0 0x55650b593ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1307090358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b57d520 0x55650b581ca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 -- 172.21.15.31:0/1307090358 >> 172.21.15.31:0/1307090358 conn(0x55650b4f2200 msgr2=0x55650b5777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 -- 172.21.15.31:0/1307090358 shutdown_connections 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 -- 172.21.15.31:0/1307090358 wait complete. 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.062+0000 7f99afda8b80 1 -- start start 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 Processor -- start 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b58aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b58b690 con 0x55650b57d520 2023-10-06T21:37:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b5eb080 con 0x55650b5883d0 2023-10-06T21:37:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650b5eb240 con 0x55650b57cc00 2023-10-06T21:37:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.063+0000 7f99afda8b80 10 client.0 _ll_register_callbacks cb 0x55650b2ca7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a99c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b58aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a99c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58478/0 (socket says 172.21.15.31:58478) 2023-10-06T21:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a99c700 1 -- 172.21.15.31:0/1156544737 learned_addr learned my addr 172.21.15.31:0/1156544737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a19b700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a99c700 1 -- 172.21.15.31:0/1156544737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 msgr2=0x55650b58a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.064+0000 7f999a99c700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f999a99c700 1 -- 172.21.15.31:0/1156544737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 msgr2=0x55650b58aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f999a99c700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b58aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f999a99c700 1 -- 172.21.15.31:0/1156544737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c013040 con 0x55650b5883d0 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f999a99c700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f998c018cf0 tx=0x7f998c004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ae93473bd47dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f997dffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c022040 con 0x55650b5883d0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f998c006e40 con 0x55650b5883d0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.065+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99900228c0 con 0x55650b5883d0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f998c01ba40 con 0x55650b5883d0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 10 client.27197 fetch_fsmap learned FSMap version 47 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f9984009880 con 0x55650b5883d0 2023-10-06T21:37:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99900228c0 con 0x55650b5883d0 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f998c026c50 con 0x55650b5883d0 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 10 client.27197 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 10 client.27197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 20 client.27197 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b5eca30 0x55650b5eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 4 client.27197 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.066+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55650b4c9ad0 con 0x55650b5eca30 2023-10-06T21:37:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.067+0000 7f999a19b700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b5eca30 0x55650b5eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.067+0000 7f999a19b700 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b5eca30 0x55650b5eee50 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac70f6d5a724aee server_cookie=c83e813d5e36f085 in_seq=0 out_seq=0 2023-10-06T21:37:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.067+0000 7f997dffb700 10 client.27197 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.121+0000 7f997dffb700 1 -- 172.21.15.31:0/1156544737 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55650b4c9ad0 con 0x55650b5eca30 2023-10-06T21:37:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.121+0000 7f997dffb700 10 client.27197 handle_command_reply: tid=0 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 client.27197 shutdown 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 msgr2=0x55650b5eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f998c018cf0 tx=0x7f998c004be0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 shutdown_connections 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55650b5883d0 0x55650b5eab10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55650b57cc00 0x55650b58a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55650b5eca30 0x55650b5eee50 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 --2- 172.21.15.31:0/1156544737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650b57d520 0x55650b58aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f997dffb700 0 client.27197 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 >> 172.21.15.31:0/1156544737 conn(0x55650b4f2200 msgr2=0x55650b581ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 shutdown_connections 2023-10-06T21:37:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.122+0000 7f99afda8b80 1 -- 172.21.15.31:0/1156544737 wait complete. 2023-10-06T21:37:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.123+0000 7f99afda8b80 20 client.27197 trim_cache size 0 max 16384 2023-10-06T21:37:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:37:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:37:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:37:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:37:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 msgr2=0x7f99800a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 0x7f99800a3c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f998c00bb40 tx=0x7f998c0095a0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 msgr2=0x7f999419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f9990004730 comp rx=0 tx=0).stop 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 shutdown_connections 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f99800a17c0 0x7f99800a3c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9994105e20 0x7f99941a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9994068810 0x7f999419d430 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f9990004730 comp rx=0 tx=0).stop 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 --2- 172.21.15.31:0/2486215748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99941038b0 0x7f999419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.126+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 >> 172.21.15.31:0/2486215748 conn(0x7f9994076b70 msgr2=0x7f99940fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.128+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 shutdown_connections 2023-10-06T21:37:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:21.128+0000 7f999c3ff700 1 -- 172.21.15.31:0/2486215748 wait complete. 2023-10-06T21:37:21.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23005 6.23 MB/sec execute 70 sec latency 88.380 ms 2023-10-06T21:37:22.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:22 smithi031 ceph-mon[98539]: pgmap v518: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:22.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:22 smithi146 ceph-mon[102809]: pgmap v518: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:22.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:22 smithi142 ceph-mon[102951]: pgmap v518: 97 pgs: 97 active+clean; 532 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-10-06T21:37:22.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23120 6.23 MB/sec execute 71 sec latency 70.390 ms 2023-10-06T21:37:23.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23250 6.24 MB/sec execute 72 sec latency 68.130 ms 2023-10-06T21:37:24.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:24 smithi031 ceph-mon[98539]: pgmap v519: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2023-10-06T21:37:24.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:24 smithi146 ceph-mon[102809]: pgmap v519: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2023-10-06T21:37:24.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:24 smithi142 ceph-mon[102951]: pgmap v519: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2023-10-06T21:37:24.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23402 6.33 MB/sec execute 73 sec latency 63.130 ms 2023-10-06T21:37:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:25 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:25 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:25 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:25.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23581 6.44 MB/sec execute 74 sec latency 71.657 ms 2023-10-06T21:37:26.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:26 smithi031 ceph-mon[98539]: pgmap v520: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-10-06T21:37:26.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:26 smithi146 ceph-mon[102809]: pgmap v520: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-10-06T21:37:26.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:26 smithi142 ceph-mon[102951]: pgmap v520: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-10-06T21:37:26.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23724 6.48 MB/sec execute 75 sec latency 196.389 ms 2023-10-06T21:37:27.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23861 6.50 MB/sec execute 76 sec latency 301.635 ms 2023-10-06T21:37:28.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:28 smithi031 ceph-mon[98539]: pgmap v521: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-10-06T21:37:28.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:28 smithi146 ceph-mon[102809]: pgmap v521: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-10-06T21:37:28.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:28 smithi142 ceph-mon[102951]: pgmap v521: 97 pgs: 97 active+clean; 535 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-10-06T21:37:28.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 23972 6.48 MB/sec execute 77 sec latency 75.203 ms 2023-10-06T21:37:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:29 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:29 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:29 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:37:29.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24091 6.55 MB/sec execute 78 sec latency 62.246 ms 2023-10-06T21:37:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:30 smithi031 ceph-mon[98539]: pgmap v522: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 270 op/s 2023-10-06T21:37:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:30 smithi146 ceph-mon[102809]: pgmap v522: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 270 op/s 2023-10-06T21:37:30.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24202 6.57 MB/sec execute 79 sec latency 74.389 ms 2023-10-06T21:37:31.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:30 smithi142 ceph-mon[102951]: pgmap v522: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 270 op/s 2023-10-06T21:37:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:31 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:37:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:31 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:37:31.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:31 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:37:31.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24322 6.59 MB/sec execute 80 sec latency 88.316 ms 2023-10-06T21:37:32.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:32 smithi031 ceph-mon[98539]: pgmap v523: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 3.4 MiB/s wr, 172 op/s 2023-10-06T21:37:32.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:32 smithi146 ceph-mon[102809]: pgmap v523: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 3.4 MiB/s wr, 172 op/s 2023-10-06T21:37:32.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:32 smithi142 ceph-mon[102951]: pgmap v523: 97 pgs: 97 active+clean; 551 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 3.4 MiB/s wr, 172 op/s 2023-10-06T21:37:32.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24433 6.58 MB/sec execute 81 sec latency 70.627 ms 2023-10-06T21:37:33.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24544 6.57 MB/sec execute 82 sec latency 64.744 ms 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.354+0000 7f8f301ff700 1 -- 172.21.15.31:0/2571434920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 msgr2=0x7f8f280686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.354+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2571434920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280686b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098f0 tx=0x7f8f24009c00 comp rx=0 tx=0).stop 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 -- 172.21.15.31:0/2571434920 shutdown_connections 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2571434920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 0x7f8f2810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2571434920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280686b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2571434920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f2810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 -- 172.21.15.31:0/2571434920 >> 172.21.15.31:0/2571434920 conn(0x7f8f28076b70 msgr2=0x7f8f28076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 -- 172.21.15.31:0/2571434920 shutdown_connections 2023-10-06T21:37:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.355+0000 7f8f301ff700 1 -- 172.21.15.31:0/2571434920 wait complete. 2023-10-06T21:37:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.356+0000 7f8f301ff700 1 Processor -- start 2023-10-06T21:37:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.356+0000 7f8f301ff700 1 -- start start 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f28072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 0x7f8f2806b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806bba0 con 0x7f8f280682d0 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806bd10 con 0x7f8f2810db20 2023-10-06T21:37:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f301ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806bed0 con 0x7f8f28068bf0 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f28072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53610/0 (socket says 172.21.15.31:53610) 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f28072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44966/0 (socket says 172.21.15.31:44966) 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2df9b700 1 -- 172.21.15.31:0/2020615120 learned_addr learned my addr 172.21.15.31:0/2020615120 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.357+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 0x7f8f2806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 -- 172.21.15.31:0/2020615120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 msgr2=0x7f8f28072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f28072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 -- 172.21.15.31:0/2020615120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 msgr2=0x7f8f2806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 0x7f8f2806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 -- 172.21.15.31:0/2020615120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24009520 con 0x7f8f280682d0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.358+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009810 tx=0x7f8f2400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.359+0000 7f8f1effd700 1 -- 172.21.15.31:0/2020615120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400ba10 con 0x7f8f280682d0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.359+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2806c0f0 con 0x7f8f280682d0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.359+0000 7f8f1effd700 1 -- 172.21.15.31:0/2020615120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2400bb70 con 0x7f8f280682d0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.359+0000 7f8f1effd700 1 -- 172.21.15.31:0/2020615120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24004c90 con 0x7f8f280682d0 2023-10-06T21:37:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.359+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f280703b0 con 0x7f8f280682d0 2023-10-06T21:37:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.364+0000 7f8f1effd700 1 -- 172.21.15.31:0/2020615120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8f24005040 con 0x7f8f280682d0 2023-10-06T21:37:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.365+0000 7f8f43b0db80 1 Processor -- start 2023-10-06T21:37:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.365+0000 7f8f43b0db80 1 -- start start 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d01286fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d01291e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d011f0400 con 0x557d012923d0 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d011e1bc0 con 0x557d01287520 2023-10-06T21:37:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d011df4e0 con 0x557d01286c00 2023-10-06T21:37:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f2e79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d01286fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f2e79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d01286fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44978/0 (socket says 172.21.15.31:44978) 2023-10-06T21:37:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f2e79c700 1 -- 172.21.15.31:0/4279608834 learned_addr learned my addr 172.21.15.31:0/4279608834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f1effd700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 0x7f8f140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.366+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d01291e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f1effd700 1 -- 172.21.15.31:0/2020615120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8f240ce040 con 0x7f8f280682d0 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 -- 172.21.15.31:0/4279608834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 msgr2=0x557d01286fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d01286fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 -- 172.21.15.31:0/4279608834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 msgr2=0x557d012947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 -- 172.21.15.31:0/4279608834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d01005d80 con 0x557d01287520 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 0x7f8f140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d01291e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cb6c0 tx=0x7f8f24009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87c969f35173151 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.367+0000 7f8f13fff700 1 -- 172.21.15.31:0/4279608834 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2409b6e0 con 0x557d01287520 2023-10-06T21:37:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.368+0000 7f8f13fff700 1 -- 172.21.15.31:0/4279608834 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2409f130 con 0x557d01287520 2023-10-06T21:37:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.368+0000 7f8f13fff700 1 -- 172.21.15.31:0/4279608834 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240a10c0 con 0x557d01287520 2023-10-06T21:37:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.368+0000 7f8f43b0db80 1 -- 172.21.15.31:0/4279608834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 msgr2=0x557d01291e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.368+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d01291e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cb6c0 tx=0x7f8f24009260 comp rx=0 tx=0).stop 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.368+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 0x7f8f140a3b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f20009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 -- 172.21.15.31:0/4279608834 shutdown_connections 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d01286fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/4279608834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d01291e90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 -- 172.21.15.31:0/4279608834 >> 172.21.15.31:0/4279608834 conn(0x557d011fc200 msgr2=0x557d01283080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 -- 172.21.15.31:0/4279608834 shutdown_connections 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 -- 172.21.15.31:0/4279608834 wait complete. 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.369+0000 7f8f43b0db80 1 -- start start 2023-10-06T21:37:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 Processor -- start 2023-10-06T21:37:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01287520 0x557d01292cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d01293230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d011dadd0 con 0x557d01286c00 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d011776b0 con 0x557d012923d0 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.370+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d01177350 con 0x557d01287520 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f43b0db80 10 client.0 _ll_register_callbacks cb 0x557d00fd47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01287520 0x557d01292cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53622/0 (socket says 172.21.15.31:53622) 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 -- 172.21.15.31:0/1848681020 learned_addr learned my addr 172.21.15.31:0/1848681020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d01293230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 -- 172.21.15.31:0/1848681020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01287520 msgr2=0x557d01292cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01287520 0x557d01292cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 -- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 msgr2=0x557d01293230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d01293230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.371+0000 7f8f2df9b700 1 -- 172.21.15.31:0/1848681020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2409fe70 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8f2409ace0 tx=0x7f8f24097320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753234e41195b65d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f117fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240a3360 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8f240052f0 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d0127ad50 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f240b9170 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f43b0db80 10 client.17487 fetch_fsmap learned FSMap version 49 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.372+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557d00feec00 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f24064080 con 0x557d01286c00 2023-10-06T21:37:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f240adbe0 con 0x557d01286c00 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 10 client.17487 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 20 client.17487 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012a1650 0x557d012a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 4 client.17487 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.373+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d011d3db0 con 0x557d012a1650 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.374+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012a1650 0x557d012a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.375+0000 7f8f117fa700 10 client.17487 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.375+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012a1650 0x557d012a3a40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6891497a92d287d server_cookie=7c254ba3f87e90fe in_seq=0 out_seq=0 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.376+0000 7f8f117fa700 1 -- 172.21.15.31:0/1848681020 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d011d3db0 con 0x557d012a1650 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.376+0000 7f8f117fa700 10 client.17487 handle_command_reply: tid=0 2023-10-06T21:37:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.376+0000 7f8f43b0db80 1 client.17487 shutdown 2023-10-06T21:37:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.376+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 msgr2=0x557d012927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.376+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8f2409ace0 tx=0x7f8f24097320 comp rx=0 tx=0).stop 2023-10-06T21:37:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 shutdown_connections 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d01293230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012a1650 0x557d012a3a40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01286c00 0x557d012927b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/1848681020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01287520 0x557d01292cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 >> 172.21.15.31:0/1848681020 conn(0x557d011fc200 msgr2=0x557d01282c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f117fa700 0 client.17487 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.377+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 shutdown_connections 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.378+0000 7f8f43b0db80 1 -- 172.21.15.31:0/1848681020 wait complete. 2023-10-06T21:37:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.378+0000 7f8f43b0db80 20 client.17487 trim_cache size 0 max 16384 2023-10-06T21:37:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.404+0000 7f8f43b0db80 1 Processor -- start 2023-10-06T21:37:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.404+0000 7f8f43b0db80 1 -- start start 2023-10-06T21:37:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d0128cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0105b8c0 con 0x557d01287520 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d01296d20 con 0x557d012923d0 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d01287b90 con 0x557d01286c00 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53632/0 (socket says 172.21.15.31:53632) 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f2d79a700 1 -- 172.21.15.31:0/470966224 learned_addr learned my addr 172.21.15.31:0/470966224 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.405+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d0128cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 -- 172.21.15.31:0/470966224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 msgr2=0x557d012967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 -- 172.21.15.31:0/470966224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 msgr2=0x557d0128cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d0128cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 -- 172.21.15.31:0/470966224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d012920a0 con 0x557d01287520 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806ccb0 tx=0x7f8f18004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4d4647234d574a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.406+0000 7f8f1cff9700 1 -- 172.21.15.31:0/470966224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18017040 con 0x557d01287520 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.407+0000 7f8f1cff9700 1 -- 172.21.15.31:0/470966224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f1801b330 con 0x557d01287520 2023-10-06T21:37:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.407+0000 7f8f1cff9700 1 -- 172.21.15.31:0/470966224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801b730 con 0x557d01287520 2023-10-06T21:37:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.407+0000 7f8f43b0db80 1 -- 172.21.15.31:0/470966224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 msgr2=0x557d0128c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.407+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806ccb0 tx=0x7f8f18004320 comp rx=0 tx=0).stop 2023-10-06T21:37:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 -- 172.21.15.31:0/470966224 shutdown_connections 2023-10-06T21:37:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d012923d0 0x557d0128cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/470966224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d01287520 0x557d0128c880 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 -- 172.21.15.31:0/470966224 >> 172.21.15.31:0/470966224 conn(0x557d011fc200 msgr2=0x557d0128bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 -- 172.21.15.31:0/470966224 shutdown_connections 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 -- 172.21.15.31:0/470966224 wait complete. 2023-10-06T21:37:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.408+0000 7f8f43b0db80 1 -- start start 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.409+0000 7f8f43b0db80 1 Processor -- start 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.409+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012ed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d012edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d012e9970 con 0x557d012923d0 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d012e9ae0 con 0x557d01287520 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d012e9ca0 con 0x557d01286c00 2023-10-06T21:37:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f43b0db80 10 client.0 _ll_register_callbacks cb 0x7f8f20009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f2d79a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d012edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012ed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f2df9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012ed7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:45018/0 (socket says 172.21.15.31:45018) 2023-10-06T21:37:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.410+0000 7f8f2df9b700 1 -- 172.21.15.31:0/3448062036 learned_addr learned my addr 172.21.15.31:0/3448062036 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2e79c700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 -- 172.21.15.31:0/3448062036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 msgr2=0x557d012ed7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012ed7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 -- 172.21.15.31:0/3448062036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 msgr2=0x557d012e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 -- 172.21.15.31:0/3448062036 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f18009d10 con 0x557d01287520 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f2d79a700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d012edd10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f18003a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d38ca748237494 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.411+0000 7f8f117fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18017040 con 0x557d01287520 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f2409f130 con 0x557d01287520 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8f180049c0 con 0x557d01287520 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f18004b80 con 0x557d01287520 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f43b0db80 10 client.27215 fetch_fsmap learned FSMap version 49 2023-10-06T21:37:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557d0102c1f0 con 0x557d01287520 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.412+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2409f130 con 0x557d01287520 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f1802d070 con 0x557d01287520 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 10 client.27215 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 10 client.27215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 20 client.27215 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012f66f0 0x557d012f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 4 client.27215 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:37:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.413+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d011d3ad0 con 0x557d012f66f0 2023-10-06T21:37:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.414+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012f66f0 0x557d012f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.414+0000 7f8f2df9b700 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012f66f0 0x557d012f8b10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6d0914a32cdb7b server_cookie=2c449bd56793fcec in_seq=0 out_seq=0 2023-10-06T21:37:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.414+0000 7f8f117fa700 10 client.27215 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f117fa700 1 -- 172.21.15.31:0/3448062036 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d011d3ad0 con 0x557d012f66f0 2023-10-06T21:37:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f117fa700 10 client.27215 handle_command_reply: tid=0 2023-10-06T21:37:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 client.27215 shutdown 2023-10-06T21:37:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 msgr2=0x557d012edd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d012edd10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f18003a60 comp rx=0 tx=0).stop 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 shutdown_connections 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557d012f66f0 0x557d012f8b10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d012923d0 0x557d012e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557d01286c00 0x557d012ed7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f117fa700 0 client.27215 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 --2- 172.21.15.31:0/3448062036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d01287520 0x557d012edd10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 >> 172.21.15.31:0/3448062036 conn(0x557d011fc200 msgr2=0x557d0128aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 shutdown_connections 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.471+0000 7f8f43b0db80 1 -- 172.21.15.31:0/3448062036 wait complete. 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.472+0000 7f8f43b0db80 20 client.27215 trim_cache size 0 max 16384 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:37:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 msgr2=0x7f8f140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 0x7f8f140a3b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f20009040 comp rx=0 tx=0).stop 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 msgr2=0x7f8f280724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009810 tx=0x7f8f2400b560 comp rx=0 tx=0).stop 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 shutdown_connections 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2810db20 0x7f8f2806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f140a1670 0x7f8f140a3b30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f280682d0 0x7f8f280724b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 --2- 172.21.15.31:0/2020615120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f28068bf0 0x7f8f28072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 >> 172.21.15.31:0/2020615120 conn(0x7f8f28076b70 msgr2=0x7f8f2810f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 shutdown_connections 2023-10-06T21:37:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:34.473+0000 7f8f301ff700 1 -- 172.21.15.31:0/2020615120 wait complete. 2023-10-06T21:37:34.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:34 smithi031 ceph-mon[98539]: pgmap v524: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-10-06T21:37:34.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:34 smithi146 ceph-mon[102809]: pgmap v524: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-10-06T21:37:34.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24656 6.57 MB/sec execute 83 sec latency 88.553 ms 2023-10-06T21:37:35.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:34 smithi142 ceph-mon[102951]: pgmap v524: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-10-06T21:37:35.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24744 6.51 MB/sec execute 84 sec latency 288.167 ms 2023-10-06T21:37:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:36 smithi146 ceph-mon[102809]: pgmap v525: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:37:36.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:36 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:37:36.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:36 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:37:36.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:36 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:37:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:36 smithi142 ceph-mon[102951]: pgmap v525: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:37:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:36 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:37:36.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:36 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:37:36.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:36 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:37:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:36 smithi031 ceph-mon[98539]: pgmap v525: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:37:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:36 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:37:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:36 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:37:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:36 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:37:36.763 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24856 6.49 MB/sec execute 85 sec latency 65.680 ms 2023-10-06T21:37:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:37 smithi146 ceph-mon[102809]: pgmap v526: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:37 smithi146 ceph-mon[102809]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2023-10-06T21:37:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:37 smithi142 ceph-mon[102951]: pgmap v526: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:37.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:37 smithi142 ceph-mon[102951]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2023-10-06T21:37:37.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:37 smithi031 ceph-mon[98539]: pgmap v526: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:37.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:37 smithi031 ceph-mon[98539]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2023-10-06T21:37:37.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 24957 6.44 MB/sec execute 86 sec latency 70.736 ms 2023-10-06T21:37:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:38 smithi146 ceph-mon[102809]: pgmap v527: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:38 smithi142 ceph-mon[102951]: pgmap v527: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:38.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:38 smithi031 ceph-mon[98539]: pgmap v527: 97 pgs: 97 active+clean; 560 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 192 op/s 2023-10-06T21:37:38.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25070 6.40 MB/sec execute 87 sec latency 63.134 ms 2023-10-06T21:37:39.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25201 6.42 MB/sec execute 88 sec latency 69.699 ms 2023-10-06T21:37:40.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25341 6.46 MB/sec execute 89 sec latency 76.030 ms 2023-10-06T21:37:41.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:41 smithi142 ceph-mon[102951]: pgmap v528: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:41 smithi031 ceph-mon[98539]: pgmap v528: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:41 smithi146 ceph-mon[102809]: pgmap v528: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:41.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25461 6.43 MB/sec execute 90 sec latency 72.457 ms 2023-10-06T21:37:42.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25619 6.51 MB/sec execute 91 sec latency 75.410 ms 2023-10-06T21:37:43.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:43 smithi142 ceph-mon[102951]: pgmap v529: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:43 smithi031 ceph-mon[98539]: pgmap v529: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:43 smithi146 ceph-mon[102809]: pgmap v529: 97 pgs: 97 active+clean; 566 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-06T21:37:43.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25730 6.48 MB/sec execute 92 sec latency 69.489 ms 2023-10-06T21:37:44.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25845 6.46 MB/sec execute 93 sec latency 70.514 ms 2023-10-06T21:37:45.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:45 smithi142 ceph-mon[102951]: pgmap v530: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:45.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:45 smithi031 ceph-mon[98539]: pgmap v530: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:45 smithi146 ceph-mon[102809]: pgmap v530: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:45.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 25965 6.40 MB/sec execute 94 sec latency 68.529 ms 2023-10-06T21:37:46.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26099 6.39 MB/sec execute 95 sec latency 69.470 ms 2023-10-06T21:37:47.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:47 smithi142 ceph-mon[102951]: pgmap v531: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:47.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:47 smithi031 ceph-mon[98539]: pgmap v531: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:47.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:47 smithi146 ceph-mon[102809]: pgmap v531: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.694+0000 7fce3affd700 1 -- 172.21.15.31:0/721444946 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c003a80 con 0x7fce3c105ef0 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 -- 172.21.15.31:0/721444946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c105ef0 msgr2=0x7fce3c1062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 --2- 172.21.15.31:0/721444946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c105ef0 0x7fce3c1062d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0099c0 tx=0x7fce2c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 -- 172.21.15.31:0/721444946 shutdown_connections 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 --2- 172.21.15.31:0/721444946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c100930 0x7fce3c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 --2- 172.21.15.31:0/721444946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c105ef0 0x7fce3c1062d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 --2- 172.21.15.31:0/721444946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 0x7fce3c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 -- 172.21.15.31:0/721444946 >> 172.21.15.31:0/721444946 conn(0x7fce3c076b70 msgr2=0x7fce3c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 -- 172.21.15.31:0/721444946 shutdown_connections 2023-10-06T21:37:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.695+0000 7fce423ff700 1 -- 172.21.15.31:0/721444946 wait complete. 2023-10-06T21:37:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 Processor -- start 2023-10-06T21:37:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 -- start start 2023-10-06T21:37:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 0x7fce3c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c105ef0 0x7fce3c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19e2e0 con 0x7fce3c0fff70 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19e450 con 0x7fce3c100930 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.697+0000 7fce423ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19e610 con 0x7fce3c105ef0 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53530/0 (socket says 172.21.15.31:53530) 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 -- 172.21.15.31:0/193878556 learned_addr learned my addr 172.21.15.31:0/193878556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce4099c700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c105ef0 0x7fce3c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3bfff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 0x7fce3c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 -- 172.21.15.31:0/193878556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c105ef0 msgr2=0x7fce3c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c105ef0 0x7fce3c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 -- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 msgr2=0x7fce3c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 0x7fce3c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.698+0000 7fce3b7fe700 1 -- 172.21.15.31:0/193878556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0095f0 con 0x7fce3c100930 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.699+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e9a0 tx=0x7fce3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.699+0000 7fce397fa700 1 -- 172.21.15.31:0/193878556 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000ca10 con 0x7fce3c100930 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.699+0000 7fce397fa700 1 -- 172.21.15.31:0/193878556 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce30010430 con 0x7fce3c100930 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.699+0000 7fce397fa700 1 -- 172.21.15.31:0/193878556 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30018680 con 0x7fce3c100930 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.700+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c19e8f0 con 0x7fce3c100930 2023-10-06T21:37:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.700+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c1a6130 con 0x7fce3c100930 2023-10-06T21:37:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.707+0000 7fce55cf8b80 1 Processor -- start 2023-10-06T21:37:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.709+0000 7fce397fa700 1 -- 172.21.15.31:0/193878556 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fce300105a0 con 0x7fce3c100930 2023-10-06T21:37:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 -- start start 2023-10-06T21:37:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247410 0x55a7c72477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7252be0 0x55a7c7254fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c71a8400 con 0x55a7c7252be0 2023-10-06T21:37:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c7199bc0 con 0x55a7c7247410 2023-10-06T21:37:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.710+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c71974e0 con 0x55a7c7247d30 2023-10-06T21:37:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47316/0 (socket says 172.21.15.31:47316) 2023-10-06T21:37:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 -- 172.21.15.31:0/3837444655 learned_addr learned my addr 172.21.15.31:0/3837444655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce4099c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247410 0x55a7c72477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 -- 172.21.15.31:0/3837444655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247410 msgr2=0x55a7c72477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247410 0x55a7c72477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 -- 172.21.15.31:0/3837444655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7252be0 msgr2=0x55a7c7254fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7252be0 0x55a7c7254fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.712+0000 7fce3bfff700 1 -- 172.21.15.31:0/3837444655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c6fbdd80 con 0x55a7c7247d30 2023-10-06T21:37:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.713+0000 7fce397fa700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 0x7fce240a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.713+0000 7fce3bfff700 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592017d1af78bab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.713+0000 7fce397fa700 1 -- 172.21.15.31:0/193878556 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fce30014070 con 0x7fce3c100930 2023-10-06T21:37:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.714+0000 7fce227fc700 1 -- 172.21.15.31:0/3837444655 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c009e60 con 0x55a7c7247d30 2023-10-06T21:37:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.714+0000 7fce4099c700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 0x7fce240a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.714+0000 7fce227fc700 1 -- 172.21.15.31:0/3837444655 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce2c003800 con 0x55a7c7247d30 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.714+0000 7fce227fc700 1 -- 172.21.15.31:0/3837444655 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c015450 con 0x55a7c7247d30 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 -- 172.21.15.31:0/3837444655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 msgr2=0x55a7c72526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c005c00 comp rx=0 tx=0).stop 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce4099c700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 0x7fce240a3c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce34009fd0 tx=0x7fce3400b040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 -- 172.21.15.31:0/3837444655 shutdown_connections 2023-10-06T21:37:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7252be0 0x55a7c7254fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247410 0x55a7c72477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/3837444655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c72526a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.715+0000 7fce55cf8b80 1 -- 172.21.15.31:0/3837444655 >> 172.21.15.31:0/3837444655 conn(0x55a7c71b4200 msgr2=0x55a7c723aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.716+0000 7fce55cf8b80 1 -- 172.21.15.31:0/3837444655 shutdown_connections 2023-10-06T21:37:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.716+0000 7fce55cf8b80 1 -- 172.21.15.31:0/3837444655 wait complete. 2023-10-06T21:37:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.716+0000 7fce55cf8b80 1 -- start start 2023-10-06T21:37:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.716+0000 7fce55cf8b80 1 Processor -- start 2023-10-06T21:37:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c724f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c724fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c7192dd0 con 0x55a7c7247410 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c712f6b0 con 0x55a7c7252be0 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c712f350 con 0x55a7c7247d30 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce55cf8b80 10 client.0 _ll_register_callbacks cb 0x55a7c6f8c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.717+0000 7fce4099c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c724fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce4099c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c724fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53548/0 (socket says 172.21.15.31:53548) 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce4099c700 1 -- 172.21.15.31:0/4132750330 learned_addr learned my addr 172.21.15.31:0/4132750330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c724f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 -- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 msgr2=0x55a7c724fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c724fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 -- 172.21.15.31:0/4132750330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 msgr2=0x55a7c72525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.718+0000 7fce3b7fe700 1 -- 172.21.15.31:0/4132750330 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c10d560 con 0x55a7c7247d30 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce3b7fe700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c724f860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce300ca960 tx=0x7fce30099b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2b2a8d85421b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3009e160 con 0x55a7c7247d30 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fce3009a8f0 con 0x55a7c7247d30 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7c7232d50 con 0x55a7c7247d30 2023-10-06T21:37:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce3009a1f0 con 0x55a7c7247d30 2023-10-06T21:37:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce55cf8b80 10 client.27169 fetch_fsmap learned FSMap version 49 2023-10-06T21:37:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a7c6fa6c00 con 0x55a7c7247d30 2023-10-06T21:37:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.719+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce300b6020 con 0x55a7c7247d30 2023-10-06T21:37:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fce30095fe0 con 0x55a7c7247d30 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 10 client.27169 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 10 client.27169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 20 client.27169 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c7273590 0x55a7c72759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 4 client.27169 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7c718bdb0 con 0x55a7c7273590 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.720+0000 7fce3bfff700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c7273590 0x55a7c72759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.721+0000 7fce1b7fe700 10 client.27169 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.721+0000 7fce3bfff700 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c7273590 0x55a7c72759f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81a98cab3e283a8 server_cookie=d6a4b18ad278dfd7 in_seq=0 out_seq=0 2023-10-06T21:37:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce1b7fe700 1 -- 172.21.15.31:0/4132750330 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7c718bdb0 con 0x55a7c7273590 2023-10-06T21:37:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce1b7fe700 10 client.27169 handle_command_reply: tid=0 2023-10-06T21:37:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 client.27169 shutdown 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 msgr2=0x55a7c724f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c724f860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce300ca960 tx=0x7fce30099b50 comp rx=0 tx=0).stop 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 shutdown_connections 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c724fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c7273590 0x55a7c72759f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/4132750330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c724f860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 >> 172.21.15.31:0/4132750330 conn(0x55a7c71b4200 msgr2=0x55a7c723ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.722+0000 7fce1b7fe700 0 client.27169 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.723+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 shutdown_connections 2023-10-06T21:37:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.723+0000 7fce55cf8b80 1 -- 172.21.15.31:0/4132750330 wait complete. 2023-10-06T21:37:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.723+0000 7fce55cf8b80 20 client.27169 trim_cache size 0 max 16384 2023-10-06T21:37:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.748+0000 7fce55cf8b80 1 Processor -- start 2023-10-06T21:37:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.748+0000 7fce55cf8b80 1 -- start start 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c724a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 0x55a7c724ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c70138c0 con 0x55a7c7247410 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c723c580 con 0x55a7c7247d30 2023-10-06T21:37:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c72507f0 con 0x55a7c7252be0 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce4099c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 0x55a7c724ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 0x55a7c724ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53550/0 (socket says 172.21.15.31:53550) 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c724a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce4099c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47332/0 (socket says 172.21.15.31:47332) 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.749+0000 7fce3b7fe700 1 -- 172.21.15.31:0/2924964299 learned_addr learned my addr 172.21.15.31:0/2924964299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 -- 172.21.15.31:0/2924964299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 msgr2=0x55a7c724ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 0x55a7c724ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 -- 172.21.15.31:0/2924964299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 msgr2=0x55a7c724a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c724a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 -- 172.21.15.31:0/2924964299 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c6fe41f0 con 0x55a7c7252be0 2023-10-06T21:37:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.750+0000 7fce4099c700 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce34003ec0 tx=0x7fce34017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac4461d8ca0bd81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.751+0000 7fce22ffd700 1 -- 172.21.15.31:0/2924964299 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400e040 con 0x55a7c7252be0 2023-10-06T21:37:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.751+0000 7fce22ffd700 1 -- 172.21.15.31:0/2924964299 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce3400f040 con 0x55a7c7252be0 2023-10-06T21:37:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.751+0000 7fce22ffd700 1 -- 172.21.15.31:0/2924964299 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3402c420 con 0x55a7c7252be0 2023-10-06T21:37:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 -- 172.21.15.31:0/2924964299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 msgr2=0x55a7c724b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce34003ec0 tx=0x7fce34017000 comp rx=0 tx=0).stop 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 -- 172.21.15.31:0/2924964299 shutdown_connections 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7252be0 0x55a7c724b140 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c724a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/2924964299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7247d30 0x55a7c724ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 -- 172.21.15.31:0/2924964299 >> 172.21.15.31:0/2924964299 conn(0x55a7c71b4200 msgr2=0x55a7c725de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.752+0000 7fce55cf8b80 1 -- 172.21.15.31:0/2924964299 shutdown_connections 2023-10-06T21:37:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.753+0000 7fce55cf8b80 1 -- 172.21.15.31:0/2924964299 wait complete. 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.753+0000 7fce55cf8b80 1 -- start start 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.753+0000 7fce55cf8b80 1 Processor -- start 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c7259600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c7257cb0 con 0x55a7c7247410 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c7257e20 con 0x55a7c7252be0 2023-10-06T21:37:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c7257fe0 con 0x55a7c7247d30 2023-10-06T21:37:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce55cf8b80 10 client.0 _ll_register_callbacks cb 0x7fce30099fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:37:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.754+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c7259600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:53554/0 (socket says 172.21.15.31:53554) 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 -- 172.21.15.31:0/45596605 learned_addr learned my addr 172.21.15.31:0/45596605 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 -- 172.21.15.31:0/45596605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 msgr2=0x55a7c7259600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c7259600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 -- 172.21.15.31:0/45596605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 msgr2=0x55a7c72590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.755+0000 7fce4099c700 1 -- 172.21.15.31:0/45596605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34003ba0 con 0x55a7c7252be0 2023-10-06T21:37:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce4099c700 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce34013010 tx=0x7fce340246c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4341edddadc8661d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:37:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:37:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400e070 con 0x55a7c7252be0 2023-10-06T21:37:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fce34004cd0 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce2c003800 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.756+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce34030040 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 10 client.27233 fetch_fsmap learned FSMap version 49 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a7c6fbc250 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce2c003800 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fce3402c4b0 con 0x55a7c7252be0 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 10 client.27233 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 10 client.27233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 20 client.27233 populate_metadata read hostname 'smithi031' 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c72c8650 0x55a7c72caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 4 client.27233 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:37:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.757+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7c718bad0 con 0x55a7c72c8650 2023-10-06T21:37:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.758+0000 7fce3bfff700 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c72c8650 0x55a7c72caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:37:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.759+0000 7fce1b7fe700 10 client.27233 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.759+0000 7fce3bfff700 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c72c8650 0x55a7c72caa70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8d0c523a14d8c5 server_cookie=e906c9855519109b in_seq=0 out_seq=0 2023-10-06T21:37:47.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26223 6.38 MB/sec execute 96 sec latency 82.518 ms 2023-10-06T21:37:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.810+0000 7fce1b7fe700 1 -- 172.21.15.31:0/45596605 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7c718bad0 con 0x55a7c72c8650 2023-10-06T21:37:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.810+0000 7fce1b7fe700 10 client.27233 handle_command_reply: tid=0 2023-10-06T21:37:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.810+0000 7fce55cf8b80 1 client.27233 shutdown 2023-10-06T21:37:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 msgr2=0x55a7c72576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce34013010 tx=0x7fce340246c0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 shutdown_connections 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7c7252be0 0x55a7c72576b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a7c72c8650 0x55a7c72caa70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7c7247410 0x55a7c72590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 --2- 172.21.15.31:0/45596605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a7c7247d30 0x55a7c7259600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce1b7fe700 0 client.27233 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 >> 172.21.15.31:0/45596605 conn(0x55a7c71b4200 msgr2=0x55a7c724ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 shutdown_connections 2023-10-06T21:37:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.811+0000 7fce55cf8b80 1 -- 172.21.15.31:0/45596605 wait complete. 2023-10-06T21:37:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.812+0000 7fce55cf8b80 20 client.27233 trim_cache size 0 max 16384 2023-10-06T21:37:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:37:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:37:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:37:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:37:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 msgr2=0x7fce240a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 0x7fce240a3c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce34009fd0 tx=0x7fce3400b040 comp rx=0 tx=0).stop 2023-10-06T21:37:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 msgr2=0x7fce3c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:37:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e9a0 tx=0x7fce3000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 shutdown_connections 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce3c105ef0 0x7fce3c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fce240a1740 0x7fce240a3c00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce3c0fff70 0x7fce3c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 --2- 172.21.15.31:0/193878556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce3c100930 0x7fce3c199630 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 >> 172.21.15.31:0/193878556 conn(0x7fce3c076b70 msgr2=0x7fce3c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 shutdown_connections 2023-10-06T21:37:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:37:47.813+0000 7fce423ff700 1 -- 172.21.15.31:0/193878556 wait complete. 2023-10-06T21:37:48.764 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26337 6.34 MB/sec execute 97 sec latency 88.614 ms 2023-10-06T21:37:49.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:49 smithi142 ceph-mon[102951]: pgmap v532: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2023-10-06T21:37:49.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:49 smithi031 ceph-mon[98539]: pgmap v532: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2023-10-06T21:37:49.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:49 smithi146 ceph-mon[102809]: pgmap v532: 97 pgs: 97 active+clean; 558 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2023-10-06T21:37:49.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26465 6.34 MB/sec execute 98 sec latency 68.752 ms 2023-10-06T21:37:50.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26589 6.36 MB/sec execute 99 sec latency 134.138 ms 2023-10-06T21:37:51.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:51 smithi142 ceph-mon[102951]: pgmap v533: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2023-10-06T21:37:51.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:51 smithi031 ceph-mon[98539]: pgmap v533: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2023-10-06T21:37:51.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:51 smithi146 ceph-mon[102809]: pgmap v533: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2023-10-06T21:37:51.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26727 6.39 MB/sec execute 100 sec latency 62.212 ms 2023-10-06T21:37:52.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 26891 6.42 MB/sec execute 101 sec latency 69.941 ms 2023-10-06T21:37:53.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:53 smithi142 ceph-mon[102951]: pgmap v534: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2023-10-06T21:37:53.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:53 smithi031 ceph-mon[98539]: pgmap v534: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2023-10-06T21:37:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:53 smithi146 ceph-mon[102809]: pgmap v534: 97 pgs: 97 active+clean; 559 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2023-10-06T21:37:53.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27035 6.49 MB/sec execute 102 sec latency 67.288 ms 2023-10-06T21:37:54.765 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27207 6.52 MB/sec execute 103 sec latency 70.343 ms 2023-10-06T21:37:55.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:55 smithi142 ceph-mon[102951]: pgmap v535: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2023-10-06T21:37:55.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:55 smithi031 ceph-mon[98539]: pgmap v535: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2023-10-06T21:37:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:55 smithi146 ceph-mon[102809]: pgmap v535: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2023-10-06T21:37:55.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27296 6.51 MB/sec execute 104 sec latency 436.209 ms 2023-10-06T21:37:56.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27395 6.53 MB/sec execute 105 sec latency 459.342 ms 2023-10-06T21:37:57.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:57 smithi142 ceph-mon[102951]: pgmap v536: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:57 smithi031 ceph-mon[98539]: pgmap v536: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:57.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:57 smithi146 ceph-mon[102809]: pgmap v536: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:57.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27485 6.51 MB/sec execute 106 sec latency 424.815 ms 2023-10-06T21:37:58.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27598 6.52 MB/sec execute 107 sec latency 63.759 ms 2023-10-06T21:37:59.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:37:59 smithi142 ceph-mon[102951]: pgmap v537: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:59.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:37:59 smithi031 ceph-mon[98539]: pgmap v537: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:59.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:37:59 smithi146 ceph-mon[102809]: pgmap v537: 97 pgs: 97 active+clean; 572 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2023-10-06T21:37:59.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27741 6.61 MB/sec execute 108 sec latency 71.128 ms 2023-10-06T21:38:00.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27830 6.58 MB/sec execute 109 sec latency 361.517 ms 2023-10-06T21:38:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 -- 172.21.15.31:0/3815119077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d410fa90 msgr2=0x7f63d41083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 --2- 172.21.15.31:0/3815119077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d410fa90 0x7f63d41083f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b0a0 tx=0x7f63d000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 -- 172.21.15.31:0/3815119077 shutdown_connections 2023-10-06T21:38:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 --2- 172.21.15.31:0/3815119077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d411a690 0x7f63d411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 --2- 172.21.15.31:0/3815119077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410f620 0x7f63d4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 --2- 172.21.15.31:0/3815119077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d410fa90 0x7f63d41083f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 -- 172.21.15.31:0/3815119077 >> 172.21.15.31:0/3815119077 conn(0x7f63d406cfa0 msgr2=0x7f63d406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 -- 172.21.15.31:0/3815119077 shutdown_connections 2023-10-06T21:38:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.040+0000 7f63dbdff700 1 -- 172.21.15.31:0/3815119077 wait complete. 2023-10-06T21:38:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.041+0000 7f63dbdff700 1 Processor -- start 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 -- start start 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d410f620 0x7f63d41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d411a690 0x7f63d41aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41aafa0 con 0x7f63d410f620 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41ab110 con 0x7f63d411a690 2023-10-06T21:38:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63dbdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41ab2d0 con 0x7f63d410fa90 2023-10-06T21:38:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63d939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63d939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42110/0 (socket says 172.21.15.31:42110) 2023-10-06T21:38:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63d939a700 1 -- 172.21.15.31:0/2838412717 learned_addr learned my addr 172.21.15.31:0/2838412717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.042+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d410f620 0x7f63d41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63da39c700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d411a690 0x7f63d41aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 -- 172.21.15.31:0/2838412717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d411a690 msgr2=0x7f63d41aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d411a690 0x7f63d41aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 -- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d410f620 msgr2=0x7f63d41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d410f620 0x7f63d41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 -- 172.21.15.31:0/2838412717 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009d00 con 0x7f63d410fa90 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.043+0000 7f63d939a700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009bd0 tx=0x7f63d0009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.044+0000 7f63caffd700 1 -- 172.21.15.31:0/2838412717 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e070 con 0x7f63d410fa90 2023-10-06T21:38:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.044+0000 7f63caffd700 1 -- 172.21.15.31:0/2838412717 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63d0005950 con 0x7f63d410fa90 2023-10-06T21:38:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.044+0000 7f63caffd700 1 -- 172.21.15.31:0/2838412717 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001be60 con 0x7f63d410fa90 2023-10-06T21:38:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.044+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d41ab550 con 0x7f63d410fa90 2023-10-06T21:38:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.044+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41b2df0 con 0x7f63d410fa90 2023-10-06T21:38:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.049+0000 7f63caffd700 1 -- 172.21.15.31:0/2838412717 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63d001b5c0 con 0x7f63d410fa90 2023-10-06T21:38:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.051+0000 7f63ef820b80 1 Processor -- start 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 -- start start 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa7849400 con 0x555aa78dfc00 2023-10-06T21:38:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa783abc0 con 0x555aa78e0520 2023-10-06T21:38:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.052+0000 7f63ef820b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa78384e0 con 0x555aa78eb3d0 2023-10-06T21:38:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.053+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.053+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54890/0 (socket says 172.21.15.31:54890) 2023-10-06T21:38:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.053+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2047481493 learned_addr learned my addr 172.21.15.31:0/2047481493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.053+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2047481493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 msgr2=0x555aa78ed7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.054+0000 7f63d939a700 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.054+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78ed7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.054+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2047481493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 msgr2=0x555aa78dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.054+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.054+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2047481493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa765ed80 con 0x555aa78e0520 2023-10-06T21:38:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.055+0000 7f63d939a700 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:38:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.055+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007810 tx=0x7f63cc007b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d309e1300f1a10d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.055+0000 7f63caffd700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 0x7f63bc0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.055+0000 7f63caffd700 1 -- 172.21.15.31:0/2838412717 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f63d00cec30 con 0x7f63d410fa90 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.055+0000 7f63bbfff700 1 -- 172.21.15.31:0/2047481493 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0125f0 con 0x555aa78e0520 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.056+0000 7f63bbfff700 1 -- 172.21.15.31:0/2047481493 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63cc012be0 con 0x555aa78e0520 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.056+0000 7f63bbfff700 1 -- 172.21.15.31:0/2047481493 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc013dd0 con 0x555aa78e0520 2023-10-06T21:38:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.056+0000 7f63da39c700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 0x7f63bc0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.056+0000 7f63ef820b80 1 -- 172.21.15.31:0/2047481493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 msgr2=0x555aa78eae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.056+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007810 tx=0x7f63cc007b20 comp rx=0 tx=0).stop 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63da39c700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 0x7f63bc0a3bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63d406a250 tx=0x7f63c40095c0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 -- 172.21.15.31:0/2047481493 shutdown_connections 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2047481493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78eae90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 -- 172.21.15.31:0/2047481493 >> 172.21.15.31:0/2047481493 conn(0x555aa7855200 msgr2=0x555aa78dc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 -- 172.21.15.31:0/2047481493 shutdown_connections 2023-10-06T21:38:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.057+0000 7f63ef820b80 1 -- 172.21.15.31:0/2047481493 wait complete. 2023-10-06T21:38:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.058+0000 7f63ef820b80 1 -- start start 2023-10-06T21:38:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.058+0000 7f63ef820b80 1 Processor -- start 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa7833dd0 con 0x555aa78dfc00 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa77d06b0 con 0x555aa78e0520 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa77d0350 con 0x555aa78eb3d0 2023-10-06T21:38:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63ef820b80 10 client.0 _ll_register_callbacks cb 0x555aa762d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59250/0 (socket says 172.21.15.31:59250) 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63d9b9b700 1 -- 172.21.15.31:0/1397108726 learned_addr learned my addr 172.21.15.31:0/1397108726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63d939a700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.059+0000 7f63da39c700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 -- 172.21.15.31:0/1397108726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 msgr2=0x555aa78e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 -- 172.21.15.31:0/1397108726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 msgr2=0x555aa78e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 -- 172.21.15.31:0/1397108726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc00e040 con 0x555aa78e0520 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63d939a700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78e86f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cc2b0 tx=0x7f63d009b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0615ddb2f2ee401 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:38:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.060+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001ae90 con 0x555aa78e0520 2023-10-06T21:38:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.061+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f63d009c2d0 con 0x555aa78e0520 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.061+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aa78d3d50 con 0x555aa78e0520 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.061+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d009bbd0 con 0x555aa78e0520 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.061+0000 7f63ef820b80 10 client.27245 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.061+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555aa7647c00 con 0x555aa78e0520 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555aa78d3d50 con 0x555aa78e0520 2023-10-06T21:38:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63d009b0c0 con 0x555aa78e0520 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 10 client.27245 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 10 client.27245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 20 client.27245 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa78fab90 0x555aa78fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 4 client.27245 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aa782cdb0 con 0x555aa78fab90 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.062+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa78fab90 0x555aa78fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.063+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa78fab90 0x555aa78fcfb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af289d4c6a5eefe server_cookie=c83edde98dbdffe3 in_seq=0 out_seq=0 2023-10-06T21:38:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.063+0000 7f63b97fa700 10 client.27245 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.064+0000 7f63b97fa700 1 -- 172.21.15.31:0/1397108726 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555aa782cdb0 con 0x555aa78fab90 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.064+0000 7f63b97fa700 10 client.27245 handle_command_reply: tid=0 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.064+0000 7f63ef820b80 1 client.27245 shutdown 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 msgr2=0x555aa78e86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78e86f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cc2b0 tx=0x7f63d009b710 comp rx=0 tx=0).stop 2023-10-06T21:38:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 shutdown_connections 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78eb3d0 0x555aa78e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa78fab90 0x555aa78fcfb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1397108726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78e0520 0x555aa78e86f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 >> 172.21.15.31:0/1397108726 conn(0x555aa7855200 msgr2=0x555aa78dbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63b97fa700 0 client.27245 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 shutdown_connections 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.065+0000 7f63ef820b80 1 -- 172.21.15.31:0/1397108726 wait complete. 2023-10-06T21:38:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.066+0000 7f63ef820b80 20 client.27245 trim_cache size 0 max 16384 2023-10-06T21:38:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 Processor -- start 2023-10-06T21:38:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 -- start start 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 0x555aa78f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78eb3d0 0x555aa78e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa76b48c0 con 0x555aa78eb3d0 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa782cdb0 con 0x555aa78dfc00 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.092+0000 7f63ef820b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa78e0e70 con 0x555aa78e0520 2023-10-06T21:38:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63d939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63d939a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42140/0 (socket says 172.21.15.31:42140) 2023-10-06T21:38:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63d939a700 1 -- 172.21.15.31:0/1568716241 learned_addr learned my addr 172.21.15.31:0/1568716241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 0x555aa78f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63da39c700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78eb3d0 0x555aa78e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.093+0000 7f63d9b9b700 1 -- 172.21.15.31:0/1568716241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 msgr2=0x555aa78e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d9b9b700 1 -- 172.21.15.31:0/1568716241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78eb3d0 msgr2=0x555aa78e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78eb3d0 0x555aa78e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d9b9b700 1 -- 172.21.15.31:0/1568716241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa76851f0 con 0x555aa78dfc00 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d939a700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 0x555aa78f1f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63cc004790 tx=0x7f63cc013930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6008f1ae5643910c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63c8bff700 1 -- 172.21.15.31:0/1568716241 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc013480 con 0x555aa78dfc00 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63c8bff700 1 -- 172.21.15.31:0/1568716241 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63cc006ff0 con 0x555aa78dfc00 2023-10-06T21:38:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.094+0000 7f63c8bff700 1 -- 172.21.15.31:0/1568716241 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc007380 con 0x555aa78dfc00 2023-10-06T21:38:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 -- 172.21.15.31:0/1568716241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 msgr2=0x555aa78f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 0x555aa78f1f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63cc004790 tx=0x7f63cc013930 comp rx=0 tx=0).stop 2023-10-06T21:38:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 -- 172.21.15.31:0/1568716241 shutdown_connections 2023-10-06T21:38:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78eb3d0 0x555aa78e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78dfc00 0x555aa78f1f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 --2- 172.21.15.31:0/1568716241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.095+0000 7f63ef820b80 1 -- 172.21.15.31:0/1568716241 >> 172.21.15.31:0/1568716241 conn(0x555aa7855200 msgr2=0x555aa78e3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.096+0000 7f63ef820b80 1 -- 172.21.15.31:0/1568716241 shutdown_connections 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.096+0000 7f63ef820b80 1 -- 172.21.15.31:0/1568716241 wait complete. 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.096+0000 7f63ef820b80 1 -- start start 2023-10-06T21:38:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.096+0000 7f63ef820b80 1 Processor -- start 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78eb3d0 0x555aa7947990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa7947ed0 con 0x555aa78dfc00 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa7948040 con 0x555aa78eb3d0 2023-10-06T21:38:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa79481b0 con 0x555aa78e0520 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63ef820b80 10 client.0 _ll_register_callbacks cb 0x7f63d0098000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63d9b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59266/0 (socket says 172.21.15.31:59266) 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2533503421 learned_addr learned my addr 172.21.15.31:0/2533503421 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.097+0000 7f63d939a700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2533503421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 msgr2=0x555aa78f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78eb3d0 msgr2=0x555aa7947990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78eb3d0 0x555aa7947990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 -- 172.21.15.31:0/2533503421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555aa7647c00 con 0x555aa78dfc00 2023-10-06T21:38:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63d9b9b700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007760 tx=0x7f63cc006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f73851f348f441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.098+0000 7f63b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:38:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc012e10 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f63cc023050 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63cc012be0 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc026530 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63ef820b80 10 client.17517 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.099+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555aa765ed80 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63cc012be0 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63cc02e020 con 0x555aa78dfc00 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 10 client.17517 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 20 client.17517 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa794f500 0x555aa7951920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 4 client.17517 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:38:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.100+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555aa782cad0 con 0x555aa794f500 2023-10-06T21:38:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.101+0000 7f63d939a700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa794f500 0x555aa7951920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.102+0000 7f63b97fa700 10 client.17517 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.102+0000 7f63d939a700 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa794f500 0x555aa7951920 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b56a17b1baef906 server_cookie=27325be699962fa2 in_seq=0 out_seq=0 2023-10-06T21:38:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.157+0000 7f63b97fa700 1 -- 172.21.15.31:0/2533503421 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555aa782cad0 con 0x555aa794f500 2023-10-06T21:38:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.157+0000 7f63b97fa700 10 client.17517 handle_command_reply: tid=0 2023-10-06T21:38:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.158+0000 7f63ef820b80 1 client.17517 shutdown 2023-10-06T21:38:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.159+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 msgr2=0x555aa78f0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.159+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007760 tx=0x7f63cc006cc0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 shutdown_connections 2023-10-06T21:38:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555aa78eb3d0 0x555aa7947990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x555aa794f500 0x555aa7951920 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555aa78dfc00 0x555aa78f0110 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 --2- 172.21.15.31:0/2533503421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555aa78e0520 0x555aa78f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 >> 172.21.15.31:0/2533503421 conn(0x555aa7855200 msgr2=0x555aa78e8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63b97fa700 0 client.17517 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 shutdown_connections 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.160+0000 7f63ef820b80 1 -- 172.21.15.31:0/2533503421 wait complete. 2023-10-06T21:38:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.161+0000 7f63ef820b80 20 client.17517 trim_cache size 0 max 16384 2023-10-06T21:38:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:38:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:38:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:38:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 msgr2=0x7f63bc0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 0x7f63bc0a3bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63d406a250 tx=0x7f63c40095c0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 msgr2=0x7f63d41a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009bd0 tx=0x7f63d0009570 comp rx=0 tx=0).stop 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 shutdown_connections 2023-10-06T21:38:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d411a690 0x7f63d41aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63bc0a1730 0x7f63bc0a3bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63d410f620 0x7f63d41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 --2- 172.21.15.31:0/2838412717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d410fa90 0x7f63d41a62f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 >> 172.21.15.31:0/2838412717 conn(0x7f63d406cfa0 msgr2=0x7f63d4118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 shutdown_connections 2023-10-06T21:38:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:01.162+0000 7f63dbdff700 1 -- 172.21.15.31:0/2838412717 wait complete. 2023-10-06T21:38:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:01 smithi031 ceph-mon[98539]: pgmap v538: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2023-10-06T21:38:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:01 smithi146 ceph-mon[102809]: pgmap v538: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2023-10-06T21:38:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:01 smithi142 ceph-mon[102951]: pgmap v538: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2023-10-06T21:38:01.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 27923 6.59 MB/sec execute 110 sec latency 137.321 ms 2023-10-06T21:38:02.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28008 6.54 MB/sec execute 111 sec latency 75.585 ms 2023-10-06T21:38:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:03 smithi031 ceph-mon[98539]: pgmap v539: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-10-06T21:38:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:03 smithi146 ceph-mon[102809]: pgmap v539: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-10-06T21:38:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:03 smithi142 ceph-mon[102951]: pgmap v539: 97 pgs: 97 active+clean; 581 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-10-06T21:38:03.766 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28121 6.53 MB/sec execute 112 sec latency 67.716 ms 2023-10-06T21:38:04.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28220 6.51 MB/sec execute 113 sec latency 96.692 ms 2023-10-06T21:38:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:05 smithi031 ceph-mon[98539]: pgmap v540: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 257 op/s 2023-10-06T21:38:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:05 smithi031 ceph-mon[98539]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:38:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:05 smithi146 ceph-mon[102809]: pgmap v540: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 257 op/s 2023-10-06T21:38:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:05 smithi146 ceph-mon[102809]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:38:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:05 smithi142 ceph-mon[102951]: pgmap v540: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 257 op/s 2023-10-06T21:38:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:05 smithi142 ceph-mon[102951]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:38:05.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28345 6.49 MB/sec execute 114 sec latency 58.366 ms 2023-10-06T21:38:06.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28450 6.46 MB/sec execute 115 sec latency 69.380 ms 2023-10-06T21:38:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:07 smithi031 ceph-mon[98539]: pgmap v542: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:07.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:07 smithi146 ceph-mon[102809]: pgmap v542: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:07 smithi142 ceph-mon[102951]: pgmap v542: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:07.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28566 6.43 MB/sec execute 116 sec latency 68.504 ms 2023-10-06T21:38:08.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28671 6.42 MB/sec execute 117 sec latency 82.024 ms 2023-10-06T21:38:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:09 smithi031 ceph-mon[98539]: pgmap v543: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:09 smithi146 ceph-mon[102809]: pgmap v543: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:09 smithi142 ceph-mon[102951]: pgmap v543: 97 pgs: 97 active+clean; 595 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-06T21:38:09.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28794 6.44 MB/sec execute 118 sec latency 86.380 ms 2023-10-06T21:38:10.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 28919 6.42 MB/sec execute 119 sec latency 65.679 ms 2023-10-06T21:38:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:11 smithi031 ceph-mon[98539]: pgmap v544: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:11 smithi146 ceph-mon[102809]: pgmap v544: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:11.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:11 smithi142 ceph-mon[102951]: pgmap v544: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:11.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29079 6.49 MB/sec execute 120 sec latency 78.112 ms 2023-10-06T21:38:12.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29194 6.48 MB/sec execute 121 sec latency 70.567 ms 2023-10-06T21:38:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:13 smithi031 ceph-mon[98539]: pgmap v545: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:13 smithi146 ceph-mon[102809]: pgmap v545: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:13 smithi142 ceph-mon[102951]: pgmap v545: 97 pgs: 97 active+clean; 606 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-06T21:38:13.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29329 6.48 MB/sec execute 122 sec latency 72.469 ms 2023-10-06T21:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 -- 172.21.15.31:0/1911273251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c073ba0 msgr2=0x7f2e9c10d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/1911273251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c10d1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098f0 tx=0x7f2e8c009c00 comp rx=0 tx=0).stop 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 -- 172.21.15.31:0/1911273251 shutdown_connections 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/1911273251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c10d730 0x7f2e9c10fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/1911273251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/1911273251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c10d1f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 -- 172.21.15.31:0/1911273251 >> 172.21.15.31:0/1911273251 conn(0x7f2e9c0fc9a0 msgr2=0x7f2e9c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 -- 172.21.15.31:0/1911273251 shutdown_connections 2023-10-06T21:38:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.395+0000 7f2ea47ff700 1 -- 172.21.15.31:0/1911273251 wait complete. 2023-10-06T21:38:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 Processor -- start 2023-10-06T21:38:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 -- start start 2023-10-06T21:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c10d730 0x7f2e9c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c19e350 con 0x7f2e9c073ba0 2023-10-06T21:38:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c19e4c0 con 0x7f2e9c10d730 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.397+0000 7f2ea47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c19e680 con 0x7f2e9c073280 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49808/0 (socket says 172.21.15.31:49808) 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 -- 172.21.15.31:0/2867677421 learned_addr learned my addr 172.21.15.31:0/2867677421 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c10d730 0x7f2e9c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 -- 172.21.15.31:0/2867677421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c10d730 msgr2=0x7f2e9c19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c10d730 0x7f2e9c19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 -- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c073ba0 msgr2=0x7f2e9c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.398+0000 7f2ea259b700 1 -- 172.21.15.31:0/2867677421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c009520 con 0x7f2e9c073280 2023-10-06T21:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2ea259b700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9a0 tx=0x7f2e9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2e937fe700 1 -- 172.21.15.31:0/2867677421 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98009790 con 0x7f2e9c073280 2023-10-06T21:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2e937fe700 1 -- 172.21.15.31:0/2867677421 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e9800cbd0 con 0x7f2e9c073280 2023-10-06T21:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c19e960 con 0x7f2e9c073280 2023-10-06T21:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2e937fe700 1 -- 172.21.15.31:0/2867677421 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98018680 con 0x7f2e9c073280 2023-10-06T21:38:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.399+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c1a61a0 con 0x7f2e9c073280 2023-10-06T21:38:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.404+0000 7f2e937fe700 1 -- 172.21.15.31:0/2867677421 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2e9800cd40 con 0x7f2e9c073280 2023-10-06T21:38:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.405+0000 7f2eb80e8b80 1 Processor -- start 2023-10-06T21:38:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.405+0000 7f2e937fe700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 0x7f2e880a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2e937fe700 1 -- 172.21.15.31:0/2867677421 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2e98014070 con 0x7f2e9c073280 2023-10-06T21:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 -- start start 2023-10-06T21:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 0x7f2e880a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 0x564f43ba27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43afe400 con 0x564f43b94c00 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43aefbc0 con 0x564f43ba03d0 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.406+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43aed4e0 con 0x564f43b95520 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea2d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 0x564f43ba27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49820/0 (socket says 172.21.15.31:49820) 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/3516877839 learned_addr learned my addr 172.21.15.31:0/3516877839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 0x7f2e880a3c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c068f90 tx=0x7f2e940099e0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/3516877839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 msgr2=0x564f43ba27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 0x564f43ba27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/3516877839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 msgr2=0x564f43b94fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b94fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/3516877839 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f43913d80 con 0x564f43b95520 2023-10-06T21:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.407+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 0x564f43ba27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.408+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098f0 tx=0x7f2e8c005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf9dfcc27c3db35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.408+0000 7f2e90ff9700 1 -- 172.21.15.31:0/3516877839 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c01c040 con 0x564f43b95520 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.408+0000 7f2e90ff9700 1 -- 172.21.15.31:0/3516877839 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e8c00f8f0 con 0x564f43b95520 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.408+0000 7f2e90ff9700 1 -- 172.21.15.31:0/3516877839 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c005040 con 0x564f43b95520 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.409+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3516877839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 msgr2=0x564f43b9fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.409+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098f0 tx=0x7f2e8c005960 comp rx=0 tx=0).stop 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.410+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3516877839 shutdown_connections 2023-10-06T21:38:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.410+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43ba03d0 0x564f43ba27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.410+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b94fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.410+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3516877839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9fe90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.410+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3516877839 >> 172.21.15.31:0/3516877839 conn(0x564f43b0a200 msgr2=0x564f43b91080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.411+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3516877839 shutdown_connections 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.411+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3516877839 wait complete. 2023-10-06T21:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.411+0000 7f2eb80e8b80 1 -- start start 2023-10-06T21:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.412+0000 7f2eb80e8b80 1 Processor -- start 2023-10-06T21:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.412+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b9fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b95520 0x564f43b9cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43ae8dd0 con 0x564f43b94c00 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43a856b0 con 0x564f43b95520 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43a85350 con 0x564f43ba03d0 2023-10-06T21:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2eb80e8b80 10 client.0 _ll_register_callbacks cb 0x564f438e27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2ea2d9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2ea2d9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49824/0 (socket says 172.21.15.31:49824) 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2ea2d9c700 1 -- 172.21.15.31:0/3368289533 learned_addr learned my addr 172.21.15.31:0/3368289533 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2ea259b700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b9fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.413+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b95520 0x564f43b9cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 -- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b95520 msgr2=0x564f43b9cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b95520 0x564f43b9cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 -- 172.21.15.31:0/3368289533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 msgr2=0x564f43b9fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b9fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 -- 172.21.15.31:0/3368289533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e94019460 con 0x564f43ba03d0 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e94017470 tx=0x7f2e94004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d114e6ae25c955 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2e85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:38:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.414+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400e040 con 0x564f43ba03d0 2023-10-06T21:38:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2e94017c00 con 0x564f43ba03d0 2023-10-06T21:38:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f43ae1db0 con 0x564f43ba03d0 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e94017dc0 con 0x564f43ba03d0 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2eb80e8b80 10 client.27199 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f2e9c110870 con 0x564f43ba03d0 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.415+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f43ae1db0 con 0x564f43ba03d0 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e9402b410 con 0x564f43ba03d0 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 10 client.27199 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 20 client.27199 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43baff80 0x564f43bb23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 4 client.27199 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f43ae1ad0 con 0x564f43baff80 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.416+0000 7f2ea259b700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43baff80 0x564f43bb23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.417+0000 7f2e85ffb700 10 client.27199 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.417+0000 7f2ea259b700 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43baff80 0x564f43bb23a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e4a425a21965fa server_cookie=af89f511564b8ccf in_seq=0 out_seq=0 2023-10-06T21:38:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2e85ffb700 1 -- 172.21.15.31:0/3368289533 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f43ae1ad0 con 0x564f43baff80 2023-10-06T21:38:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2e85ffb700 10 client.27199 handle_command_reply: tid=0 2023-10-06T21:38:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 client.27199 shutdown 2023-10-06T21:38:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 msgr2=0x564f43b9d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e94017470 tx=0x7f2e94004b40 comp rx=0 tx=0).stop 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 shutdown_connections 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b9d620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43baff80 0x564f43bb23a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b94c00 0x564f43b9fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/3368289533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b95520 0x564f43b9cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 >> 172.21.15.31:0/3368289533 conn(0x564f43b0a200 msgr2=0x564f43b90c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.418+0000 7f2e85ffb700 0 client.27199 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.419+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 shutdown_connections 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.419+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/3368289533 wait complete. 2023-10-06T21:38:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.419+0000 7f2eb80e8b80 20 client.27199 trim_cache size 0 max 16384 2023-10-06T21:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.446+0000 7f2eb80e8b80 1 Processor -- start 2023-10-06T21:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 -- start start 2023-10-06T21:38:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43ba6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43ba03d0 0x564f43b9f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f439698c0 con 0x564f43ba03d0 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43b98e60 con 0x564f43b94c00 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43b92430 con 0x564f43b95520 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.447+0000 7f2ea1d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49840/0 (socket says 172.21.15.31:49840) 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/1158322080 learned_addr learned my addr 172.21.15.31:0/1158322080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea259b700 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43ba6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43ba03d0 0x564f43b9f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/1158322080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 msgr2=0x564f43ba6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43ba6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/1158322080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43ba03d0 msgr2=0x564f43b9f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43ba03d0 0x564f43b9f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.448+0000 7f2ea1d9a700 1 -- 172.21.15.31:0/1158322080 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f438fcc00 con 0x564f43b95520 2023-10-06T21:38:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.449+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0059d0 tx=0x7f2e8c005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bc7920f277eeeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.449+0000 7f2e917fa700 1 -- 172.21.15.31:0/1158322080 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c01c070 con 0x564f43b95520 2023-10-06T21:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.450+0000 7f2e917fa700 1 -- 172.21.15.31:0/1158322080 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e8c00f8f0 con 0x564f43b95520 2023-10-06T21:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.450+0000 7f2e917fa700 1 -- 172.21.15.31:0/1158322080 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c005290 con 0x564f43b95520 2023-10-06T21:38:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/1158322080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 msgr2=0x564f43b9c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0059d0 tx=0x7f2e8c005b20 comp rx=0 tx=0).stop 2023-10-06T21:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/1158322080 shutdown_connections 2023-10-06T21:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43ba03d0 0x564f43b9f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43ba6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/1158322080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43b95520 0x564f43b9c7b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/1158322080 >> 172.21.15.31:0/1158322080 conn(0x564f43b0a200 msgr2=0x564f43bab670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/1158322080 shutdown_connections 2023-10-06T21:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.451+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/1158322080 wait complete. 2023-10-06T21:38:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.452+0000 7f2eb80e8b80 1 -- start start 2023-10-06T21:38:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.453+0000 7f2eb80e8b80 1 Processor -- start 2023-10-06T21:38:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b95520 0x564f43b99270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b997b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43b99cf0 con 0x564f43b95520 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43b99e60 con 0x564f43b94c00 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43b9a020 con 0x564f43ba03d0 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2eb80e8b80 10 client.0 _ll_register_callbacks cb 0x7f2e94019990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2ea259b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2ea259b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:37570/0 (socket says 172.21.15.31:37570) 2023-10-06T21:38:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.454+0000 7f2ea259b700 1 -- 172.21.15.31:0/4145479084 learned_addr learned my addr 172.21.15.31:0/4145479084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b95520 0x564f43b99270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea2d9c700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 -- 172.21.15.31:0/4145479084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 msgr2=0x564f43b997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 -- 172.21.15.31:0/4145479084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b95520 msgr2=0x564f43b99270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b95520 0x564f43b99270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 -- 172.21.15.31:0/4145479084 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f43913d80 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2ea259b700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e98096d20 tx=0x7f2e980965f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a21dda562382bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.455+0000 7f2e85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980caea0 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2e9809e400 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f43934f10 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e980a2640 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2eb80e8b80 10 client.27266 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.456+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564f4393a1f0 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f43934f10 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e980a21b0 con 0x564f43b94c00 2023-10-06T21:38:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 10 client.27266 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 10 client.27266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 20 client.27266 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43c05230 0x564f43c07650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 4 client.27266 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.457+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f43969f00 con 0x564f43c05230 2023-10-06T21:38:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.458+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43c05230 0x564f43c07650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.458+0000 7f2ea1d9a700 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43c05230 0x564f43c07650 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137b0b52185c9156 server_cookie=439d0d4fe936d36e in_seq=0 out_seq=0 2023-10-06T21:38:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.458+0000 7f2e85ffb700 10 client.27266 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2e85ffb700 1 -- 172.21.15.31:0/4145479084 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f43969f00 con 0x564f43c05230 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2e85ffb700 10 client.27266 handle_command_reply: tid=0 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 client.27266 shutdown 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 msgr2=0x564f43b9aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e98096d20 tx=0x7f2e980965f0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 shutdown_connections 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f43ba03d0 0x564f43b997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f43b94c00 0x564f43b9aa30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.516+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f43c05230 0x564f43c07650 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2eb80e8b80 1 --2- 172.21.15.31:0/4145479084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f43b95520 0x564f43b99270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2e85ffb700 0 client.27266 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 >> 172.21.15.31:0/4145479084 conn(0x564f43b0a200 msgr2=0x564f43b982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 shutdown_connections 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2eb80e8b80 1 -- 172.21.15.31:0/4145479084 wait complete. 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.517+0000 7f2eb80e8b80 20 client.27266 trim_cache size 0 max 16384 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:38:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 msgr2=0x7f2e880a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 0x7f2e880a3c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c068f90 tx=0x7f2e940099e0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 msgr2=0x7f2e9c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9a0 tx=0x7f2e9800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 shutdown_connections 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9c10d730 0x7f2e9c19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e9c073280 0x7f2e9c199160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2e880a1740 0x7f2e880a3c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 --2- 172.21.15.31:0/2867677421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e9c073ba0 0x7f2e9c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.518+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 >> 172.21.15.31:0/2867677421 conn(0x7f2e9c0fc9a0 msgr2=0x7f2e9c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.519+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 shutdown_connections 2023-10-06T21:38:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:14.519+0000 7f2ea47ff700 1 -- 172.21.15.31:0/2867677421 wait complete. 2023-10-06T21:38:14.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29434 6.44 MB/sec execute 123 sec latency 117.881 ms 2023-10-06T21:38:15.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:15 smithi142 ceph-mon[102951]: pgmap v546: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:15.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:15 smithi142 ceph-mon[102951]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:38:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:15 smithi031 ceph-mon[98539]: pgmap v546: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:15 smithi031 ceph-mon[98539]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:38:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:15 smithi146 ceph-mon[102809]: pgmap v546: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:15 smithi146 ceph-mon[102809]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:38:15.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29510 6.39 MB/sec execute 124 sec latency 389.671 ms 2023-10-06T21:38:16.767 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29633 6.38 MB/sec execute 125 sec latency 69.348 ms 2023-10-06T21:38:17.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:17 smithi142 ceph-mon[102951]: pgmap v548: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:17 smithi031 ceph-mon[98539]: pgmap v548: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:17 smithi146 ceph-mon[102809]: pgmap v548: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:17.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29733 6.36 MB/sec execute 126 sec latency 427.749 ms 2023-10-06T21:38:18.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29861 6.34 MB/sec execute 127 sec latency 73.178 ms 2023-10-06T21:38:19.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:19 smithi142 ceph-mon[102951]: pgmap v549: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:19 smithi031 ceph-mon[98539]: pgmap v549: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:19 smithi146 ceph-mon[102809]: pgmap v549: 97 pgs: 97 active+clean; 617 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2023-10-06T21:38:19.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 29990 6.35 MB/sec execute 128 sec latency 73.809 ms 2023-10-06T21:38:20.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30096 6.35 MB/sec execute 129 sec latency 228.990 ms 2023-10-06T21:38:21.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:21 smithi142 ceph-mon[102951]: pgmap v550: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:21 smithi031 ceph-mon[98539]: pgmap v550: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:21 smithi146 ceph-mon[102809]: pgmap v550: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:21.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30225 6.34 MB/sec execute 130 sec latency 64.409 ms 2023-10-06T21:38:22.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30370 6.40 MB/sec execute 131 sec latency 65.280 ms 2023-10-06T21:38:23.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:23 smithi142 ceph-mon[102951]: pgmap v551: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:23 smithi031 ceph-mon[98539]: pgmap v551: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:23 smithi146 ceph-mon[102809]: pgmap v551: 97 pgs: 97 active+clean; 618 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 243 op/s 2023-10-06T21:38:23.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30514 6.43 MB/sec execute 132 sec latency 70.764 ms 2023-10-06T21:38:24.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30693 6.47 MB/sec execute 133 sec latency 71.079 ms 2023-10-06T21:38:25.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:25 smithi142 ceph-mon[102951]: pgmap v552: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2023-10-06T21:38:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:25 smithi031 ceph-mon[98539]: pgmap v552: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2023-10-06T21:38:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:25 smithi146 ceph-mon[102809]: pgmap v552: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2023-10-06T21:38:25.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30823 6.50 MB/sec execute 134 sec latency 77.931 ms 2023-10-06T21:38:26.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 30947 6.49 MB/sec execute 135 sec latency 74.425 ms 2023-10-06T21:38:27.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:27 smithi142 ceph-mon[102951]: pgmap v553: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-10-06T21:38:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:27 smithi031 ceph-mon[98539]: pgmap v553: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-10-06T21:38:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:27 smithi146 ceph-mon[102809]: pgmap v553: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-10-06T21:38:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 -- 172.21.15.31:0/748853482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101ab0 msgr2=0x7fe710105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 --2- 172.21.15.31:0/748853482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101ab0 0x7fe710105af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe700009990 tx=0x7fe700009ca0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 -- 172.21.15.31:0/748853482 shutdown_connections 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 --2- 172.21.15.31:0/748853482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 0x7fe71010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 --2- 172.21.15.31:0/748853482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710101100 0x7fe7101014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 --2- 172.21.15.31:0/748853482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101ab0 0x7fe710105af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 -- 172.21.15.31:0/748853482 >> 172.21.15.31:0/748853482 conn(0x7fe7100fc9a0 msgr2=0x7fe7100fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 -- 172.21.15.31:0/748853482 shutdown_connections 2023-10-06T21:38:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.754+0000 7fe7175ff700 1 -- 172.21.15.31:0/748853482 wait complete. 2023-10-06T21:38:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.756+0000 7fe7175ff700 1 Processor -- start 2023-10-06T21:38:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.756+0000 7fe7175ff700 1 -- start start 2023-10-06T21:38:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.756+0000 7fe7175ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 0x7fe710074360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe7175ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 0x7fe7100748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe7175ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe7175ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710108550 con 0x7fe710074de0 2023-10-06T21:38:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe7175ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101086c0 con 0x7fe710106220 2023-10-06T21:38:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe7175ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710108880 con 0x7fe710101100 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57146/0 (socket says 172.21.15.31:57146) 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 -- 172.21.15.31:0/2168637613 learned_addr learned my addr 172.21.15.31:0/2168637613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 0x7fe7100748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe71539b700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 0x7fe710074360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 -- 172.21.15.31:0/2168637613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 msgr2=0x7fe710074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 0x7fe710074360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 -- 172.21.15.31:0/2168637613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 msgr2=0x7fe7100748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.757+0000 7fe715b9c700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 0x7fe7100748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe715b9c700 1 -- 172.21.15.31:0/2168637613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000095f0 con 0x7fe710074de0 2023-10-06T21:38:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe71539b700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 0x7fe710074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:38:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe715b9c700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e940 tx=0x7fe70800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe7067fc700 1 -- 172.21.15.31:0/2168637613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7080097b0 con 0x7fe710074de0 2023-10-06T21:38:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe7067fc700 1 -- 172.21.15.31:0/2168637613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe70800ee90 con 0x7fe710074de0 2023-10-06T21:38:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe7067fc700 1 -- 172.21.15.31:0/2168637613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708005450 con 0x7fe710074de0 2023-10-06T21:38:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.758+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe710108b60 con 0x7fe710074de0 2023-10-06T21:38:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.759+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7101b2fd0 con 0x7fe710074de0 2023-10-06T21:38:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.764+0000 7fe7067fc700 1 -- 172.21.15.31:0/2168637613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe708005630 con 0x7fe710074de0 2023-10-06T21:38:27.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31051 6.52 MB/sec execute 136 sec latency 67.294 ms 2023-10-06T21:38:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.765+0000 7fe72ae43b80 1 Processor -- start 2023-10-06T21:38:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 -- start start 2023-10-06T21:38:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe7067fc700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 0x7fe6fc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de1520 0x5635e1debe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1dec3d0 0x5635e1dee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1d4a400 con 0x5635e1de1520 2023-10-06T21:38:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1d3bbc0 con 0x5635e1de0c00 2023-10-06T21:38:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1d394e0 con 0x5635e1dec3d0 2023-10-06T21:38:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.766+0000 7fe7067fc700 1 -- 172.21.15.31:0/2168637613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe708014070 con 0x7fe710074de0 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45406/0 (socket says 172.21.15.31:45406) 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 -- 172.21.15.31:0/1778292685 learned_addr learned my addr 172.21.15.31:0/1778292685 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 -- 172.21.15.31:0/1778292685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1dec3d0 msgr2=0x5635e1dee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1dec3d0 0x5635e1dee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 -- 172.21.15.31:0/1778292685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de1520 msgr2=0x5635e1debe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de1520 0x5635e1debe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.767+0000 7fe71539b700 1 -- 172.21.15.31:0/1778292685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e1b5fd80 con 0x5635e1de0c00 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.768+0000 7fe71539b700 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d520 tx=0x7fe70c00d830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8523476389f4c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.768+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 0x7fe6fc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.768+0000 7fe6fb7fe700 1 -- 172.21.15.31:0/1778292685 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c012800 con 0x5635e1de0c00 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.768+0000 7fe6fb7fe700 1 -- 172.21.15.31:0/1778292685 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe70c00de40 con 0x5635e1de0c00 2023-10-06T21:38:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.768+0000 7fe6fb7fe700 1 -- 172.21.15.31:0/1778292685 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c011bf0 con 0x5635e1de0c00 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 0x7fe6fc0a3ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe700006010 tx=0x7fe700005400 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1778292685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 msgr2=0x5635e1de0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d520 tx=0x7fe70c00d830 comp rx=0 tx=0).stop 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1778292685 shutdown_connections 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1dec3d0 0x5635e1dee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1de0fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1778292685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de1520 0x5635e1debe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1778292685 >> 172.21.15.31:0/1778292685 conn(0x5635e1d56200 msgr2=0x5635e1ddd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1778292685 shutdown_connections 2023-10-06T21:38:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.769+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1778292685 wait complete. 2023-10-06T21:38:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.770+0000 7fe72ae43b80 1 -- start start 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.770+0000 7fe72ae43b80 1 Processor -- start 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1de9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1de97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1d34dd0 con 0x5635e1dec3d0 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1cd16b0 con 0x5635e1de0c00 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1cd1350 con 0x5635e1de1520 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe72ae43b80 10 client.0 _ll_register_callbacks cb 0x5635e1b2e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.771+0000 7fe71539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45420/0 (socket says 172.21.15.31:45420) 2023-10-06T21:38:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 -- 172.21.15.31:0/2017946930 learned_addr learned my addr 172.21.15.31:0/2017946930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe715b9c700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1de97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1de9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 -- 172.21.15.31:0/2017946930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 msgr2=0x5635e1de9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1de9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 -- 172.21.15.31:0/2017946930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 msgr2=0x5635e1de97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1de97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 -- 172.21.15.31:0/2017946930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c00d150 con 0x5635e1de0c00 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1de9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe71539b700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe70c011790 tx=0x7fe70c004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ad97e566ea14aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.772+0000 7fe6f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.773+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c011c00 con 0x5635e1de0c00 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.773+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe70c011d60 con 0x5635e1de0c00 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.773+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e1dd4d50 con 0x5635e1de0c00 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.773+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70c012600 con 0x5635e1de0c00 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.773+0000 7fe72ae43b80 10 client.27278 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.774+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5635e1b48c00 con 0x5635e1de0c00 2023-10-06T21:38:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.774+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5635e1dd4d50 con 0x5635e1de0c00 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe70c01b7b0 con 0x5635e1de0c00 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 10 client.27278 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 10 client.27278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 20 client.27278 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1dfbc90 0x5635e1dfe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 4 client.27278 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.775+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e1d2ddb0 con 0x5635e1dfbc90 2023-10-06T21:38:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.776+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1dfbc90 0x5635e1dfe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.777+0000 7fe714b9a700 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1dfbc90 0x5635e1dfe0b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3930d50a86225e server_cookie=58ac3df7c581333e in_seq=0 out_seq=0 2023-10-06T21:38:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.777+0000 7fe6f8ff9700 10 client.27278 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.778+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/2017946930 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635e1d2ddb0 con 0x5635e1dfbc90 2023-10-06T21:38:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.778+0000 7fe6f8ff9700 10 client.27278 handle_command_reply: tid=0 2023-10-06T21:38:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.778+0000 7fe72ae43b80 1 client.27278 shutdown 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.778+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 msgr2=0x5635e1debe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.778+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe70c011790 tx=0x7fe70c004f70 comp rx=0 tx=0).stop 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 shutdown_connections 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1dfbc90 0x5635e1dfe0b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1de97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de0c00 0x5635e1debe10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/2017946930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1de9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe6f8ff9700 0 client.27278 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 >> 172.21.15.31:0/2017946930 conn(0x5635e1d56200 msgr2=0x5635e1ddcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 shutdown_connections 2023-10-06T21:38:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 1 -- 172.21.15.31:0/2017946930 wait complete. 2023-10-06T21:38:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.779+0000 7fe72ae43b80 20 client.27278 trim_cache size 0 max 16384 2023-10-06T21:38:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 Processor -- start 2023-10-06T21:38:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 -- start start 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1deb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1dec3d0 0x5635e1de6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1bb58c0 con 0x5635e1de0c00 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1dd98d0 con 0x5635e1dec3d0 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1dea6c0 con 0x5635e1de1520 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.807+0000 7fe71539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57164/0 (socket says 172.21.15.31:57164) 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 -- 172.21.15.31:0/1511651368 learned_addr learned my addr 172.21.15.31:0/1511651368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe715b9c700 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1dec3d0 0x5635e1de6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 -- 172.21.15.31:0/1511651368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 msgr2=0x5635e1deb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1deb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 -- 172.21.15.31:0/1511651368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1dec3d0 msgr2=0x5635e1de6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1dec3d0 0x5635e1de6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 -- 172.21.15.31:0/1511651368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e1b861f0 con 0x5635e1de0c00 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.808+0000 7fe71539b700 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007a30 tx=0x7fe70c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a219d7dc1976d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.809+0000 7fe6fbfff700 1 -- 172.21.15.31:0/1511651368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00dce0 con 0x5635e1de0c00 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.809+0000 7fe6fbfff700 1 -- 172.21.15.31:0/1511651368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe70c01bce0 con 0x5635e1de0c00 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.809+0000 7fe6fbfff700 1 -- 172.21.15.31:0/1511651368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c012860 con 0x5635e1de0c00 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1511651368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 msgr2=0x5635e1de8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007a30 tx=0x7fe70c01bf80 comp rx=0 tx=0).stop 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1511651368 shutdown_connections 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1dec3d0 0x5635e1de6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1de0c00 0x5635e1de8890 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/1511651368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de1520 0x5635e1deb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1511651368 >> 172.21.15.31:0/1511651368 conn(0x5635e1d56200 msgr2=0x5635e1de4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1511651368 shutdown_connections 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- 172.21.15.31:0/1511651368 wait complete. 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.810+0000 7fe72ae43b80 1 -- start start 2023-10-06T21:38:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.811+0000 7fe72ae43b80 1 Processor -- start 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.811+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.811+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de1520 0x5635e1df1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.811+0000 7fe72ae43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1e4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1df1d50 con 0x5635e1dec3d0 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1e4f000 con 0x5635e1de1520 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe72ae43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e1e4f140 con 0x5635e1de0c00 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe72ae43b80 10 client.0 _ll_register_callbacks cb 0x7fe70c00d370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:44216/0 (socket says 172.21.15.31:44216) 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 -- 172.21.15.31:0/876494426 learned_addr learned my addr 172.21.15.31:0/876494426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe714b9a700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de1520 0x5635e1df1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe715b9c700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1e4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 -- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de1520 msgr2=0x5635e1df1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de1520 0x5635e1df1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 -- 172.21.15.31:0/876494426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 msgr2=0x5635e1e4eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1e4eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe71539b700 1 -- 172.21.15.31:0/876494426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e1b48c00 con 0x5635e1de0c00 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.812+0000 7fe715b9c700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1e4eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe71539b700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007710 tx=0x7fe70c007740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f8d38fda33a404 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe6f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00dce0 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e1d2dad0 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe70c01b450 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70c01b610 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe72ae43b80 10 client.27211 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.813+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5635e1b5e250 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5635e1d2dad0 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe70c005240 con 0x5635e1de0c00 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 10 client.27211 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 10 client.27211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 20 client.27211 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1e50470 0x5635e1e52890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 4 client.27211 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635e1b80f10 con 0x5635e1e50470 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.814+0000 7fe714b9a700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1e50470 0x5635e1e52890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.815+0000 7fe6f8ff9700 10 client.27211 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.815+0000 7fe714b9a700 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1e50470 0x5635e1e52890 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67420d6125f23333 server_cookie=71c2e3839cc3cb72 in_seq=0 out_seq=0 2023-10-06T21:38:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe6f8ff9700 1 -- 172.21.15.31:0/876494426 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635e1b80f10 con 0x5635e1e50470 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe6f8ff9700 10 client.27211 handle_command_reply: tid=0 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 client.27211 shutdown 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 msgr2=0x5635e1df0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007710 tx=0x7fe70c007740 comp rx=0 tx=0).stop 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 shutdown_connections 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5635e1e50470 0x5635e1e52890 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e1dec3d0 0x5635e1e4eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635e1de0c00 0x5635e1df0f00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 --2- 172.21.15.31:0/876494426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e1de1520 0x5635e1df1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 >> 172.21.15.31:0/876494426 conn(0x5635e1d56200 msgr2=0x5635e1ddc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.859+0000 7fe6f8ff9700 0 client.27211 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.860+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 shutdown_connections 2023-10-06T21:38:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.860+0000 7fe72ae43b80 1 -- 172.21.15.31:0/876494426 wait complete. 2023-10-06T21:38:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.860+0000 7fe72ae43b80 20 client.27211 trim_cache size 0 max 16384 2023-10-06T21:38:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 msgr2=0x7fe6fc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 0x7fe6fc0a3ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe700006010 tx=0x7fe700005400 comp rx=0 tx=0).stop 2023-10-06T21:38:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 msgr2=0x7fe710107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e940 tx=0x7fe70800ec50 comp rx=0 tx=0).stop 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 shutdown_connections 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe6fc0a15f0 0x7fe6fc0a3ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710074de0 0x7fe710107e20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe710101100 0x7fe710074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 --2- 172.21.15.31:0/2168637613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe710106220 0x7fe7100748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 >> 172.21.15.31:0/2168637613 conn(0x7fe7100fc9a0 msgr2=0x7fe7100fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 shutdown_connections 2023-10-06T21:38:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:27.861+0000 7fe7175ff700 1 -- 172.21.15.31:0/2168637613 wait complete. 2023-10-06T21:38:28.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31180 6.54 MB/sec execute 137 sec latency 83.117 ms 2023-10-06T21:38:29.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:29 smithi142 ceph-mon[102951]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 187 op/s 2023-10-06T21:38:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:29 smithi031 ceph-mon[98539]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 187 op/s 2023-10-06T21:38:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:29 smithi146 ceph-mon[102809]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 187 op/s 2023-10-06T21:38:29.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31296 6.53 MB/sec execute 138 sec latency 71.085 ms 2023-10-06T21:38:30.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31423 6.57 MB/sec execute 139 sec latency 67.104 ms 2023-10-06T21:38:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:31 smithi031 ceph-mon[98539]: pgmap v555: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2023-10-06T21:38:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:31 smithi146 ceph-mon[102809]: pgmap v555: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2023-10-06T21:38:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:31 smithi142 ceph-mon[102951]: pgmap v555: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2023-10-06T21:38:31.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31515 6.54 MB/sec execute 140 sec latency 68.471 ms 2023-10-06T21:38:32.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31609 6.52 MB/sec execute 141 sec latency 69.853 ms 2023-10-06T21:38:33.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:33 smithi031 ceph-mon[98539]: pgmap v556: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-06T21:38:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:33 smithi146 ceph-mon[102809]: pgmap v556: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-06T21:38:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:33 smithi142 ceph-mon[102951]: pgmap v556: 97 pgs: 97 active+clean; 643 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-06T21:38:33.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31713 6.52 MB/sec execute 142 sec latency 73.164 ms 2023-10-06T21:38:34.768 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31828 6.50 MB/sec execute 143 sec latency 68.716 ms 2023-10-06T21:38:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:35 smithi031 ceph-mon[98539]: pgmap v557: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 264 op/s 2023-10-06T21:38:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:35 smithi146 ceph-mon[102809]: pgmap v557: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 264 op/s 2023-10-06T21:38:35.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:35 smithi142 ceph-mon[102951]: pgmap v557: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 264 op/s 2023-10-06T21:38:35.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31886 6.46 MB/sec execute 144 sec latency 384.689 ms 2023-10-06T21:38:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:36 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:38:36.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:36 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:38:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:36 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:38:36.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 31993 6.43 MB/sec execute 145 sec latency 444.647 ms 2023-10-06T21:38:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:37 smithi031 ceph-mon[98539]: pgmap v558: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-10-06T21:38:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:37 smithi146 ceph-mon[102809]: pgmap v558: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-10-06T21:38:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:37 smithi142 ceph-mon[102951]: pgmap v558: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-10-06T21:38:37.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32080 6.43 MB/sec execute 146 sec latency 360.851 ms 2023-10-06T21:38:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:38 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:38:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:38 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:38:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:38 smithi031 ceph-mon[98539]: pgmap v559: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 181 op/s 2023-10-06T21:38:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:38 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:38:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:38 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:38:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:38 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:38:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:38 smithi146 ceph-mon[102809]: pgmap v559: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 181 op/s 2023-10-06T21:38:38.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:38 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:38:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:38 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:38:38.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:38 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:38:38.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:38 smithi142 ceph-mon[102951]: pgmap v559: 97 pgs: 97 active+clean; 652 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 181 op/s 2023-10-06T21:38:38.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:38 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:38:38.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32180 6.41 MB/sec execute 147 sec latency 75.949 ms 2023-10-06T21:38:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:39 smithi031 ceph-mon[98539]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2023-10-06T21:38:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:39 smithi146 ceph-mon[102809]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2023-10-06T21:38:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:39 smithi142 ceph-mon[102951]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2023-10-06T21:38:39.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32284 6.39 MB/sec execute 148 sec latency 80.285 ms 2023-10-06T21:38:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:40 smithi031 ceph-mon[98539]: pgmap v560: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2023-10-06T21:38:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:40 smithi146 ceph-mon[102809]: pgmap v560: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2023-10-06T21:38:40.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:40 smithi142 ceph-mon[102951]: pgmap v560: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2023-10-06T21:38:40.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32382 6.37 MB/sec execute 149 sec latency 226.788 ms 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 -- 172.21.15.31:0/856428272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 msgr2=0x7fcc601014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 --2- 172.21.15.31:0/856428272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc601014b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009990 tx=0x7fcc5c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 -- 172.21.15.31:0/856428272 shutdown_connections 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 --2- 172.21.15.31:0/856428272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc6010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 --2- 172.21.15.31:0/856428272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc601014b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 --2- 172.21.15.31:0/856428272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60101a80 0x7fcc60105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 -- 172.21.15.31:0/856428272 >> 172.21.15.31:0/856428272 conn(0x7fcc600fc950 msgr2=0x7fcc600fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 -- 172.21.15.31:0/856428272 shutdown_connections 2023-10-06T21:38:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.080+0000 7fcc67bff700 1 -- 172.21.15.31:0/856428272 wait complete. 2023-10-06T21:38:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 Processor -- start 2023-10-06T21:38:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 -- start start 2023-10-06T21:38:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc60107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60107770 0x7fcc601a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a9d80 con 0x7fcc601010d0 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a9ef0 con 0x7fcc601061f0 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.082+0000 7fcc67bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601aa0b0 con 0x7fcc60107770 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47014/0 (socket says 172.21.15.31:47014) 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 -- 172.21.15.31:0/3963057245 learned_addr learned my addr 172.21.15.31:0/3963057245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6519a700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc60107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6619c700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60107770 0x7fcc601a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 -- 172.21.15.31:0/3963057245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60107770 msgr2=0x7fcc601a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60107770 0x7fcc601a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 -- 172.21.15.31:0/3963057245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 msgr2=0x7fcc60107230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc60107230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 -- 172.21.15.31:0/3963057245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0095f0 con 0x7fcc601010d0 2023-10-06T21:38:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6519a700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc60107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:38:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.083+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0036c0 tx=0x7fcc5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.084+0000 7fcc56ffd700 1 -- 172.21.15.31:0/3963057245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c01d070 con 0x7fcc601010d0 2023-10-06T21:38:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.084+0000 7fcc56ffd700 1 -- 172.21.15.31:0/3963057245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc5c003f40 con 0x7fcc601010d0 2023-10-06T21:38:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.084+0000 7fcc56ffd700 1 -- 172.21.15.31:0/3963057245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c022c40 con 0x7fcc601010d0 2023-10-06T21:38:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.084+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc601aa330 con 0x7fcc601010d0 2023-10-06T21:38:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.084+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc601aa700 con 0x7fcc601010d0 2023-10-06T21:38:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.088+0000 7fcc56ffd700 1 -- 172.21.15.31:0/3963057245 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcc5c022470 con 0x7fcc601010d0 2023-10-06T21:38:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.089+0000 7fcc7b49db80 1 Processor -- start 2023-10-06T21:38:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 -- start start 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c956d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c94c9400 con 0x55d7c9560520 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c94babc0 con 0x55d7c955fc00 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c94b84e0 con 0x55d7c956b3d0 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc6619c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc6619c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49666/0 (socket says 172.21.15.31:49666) 2023-10-06T21:38:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc6619c700 1 -- 172.21.15.31:0/3313030415 learned_addr learned my addr 172.21.15.31:0/3313030415 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 0x7fcc4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.090+0000 7fcc6519a700 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c956d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc56ffd700 1 -- 172.21.15.31:0/3963057245 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fcc5c0ce750 con 0x7fcc601010d0 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6619c700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 0x7fcc4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 -- 172.21.15.31:0/3313030415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 msgr2=0x55d7c956d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c956d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 -- 172.21.15.31:0/3313030415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 msgr2=0x55d7c955ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 -- 172.21.15.31:0/3313030415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7c92ded80 con 0x55d7c9560520 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6619c700 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6599b700 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956ae90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0cbdd0 tx=0x7fcc5c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5670bf3c2e749c41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc4bfff700 1 -- 172.21.15.31:0/3313030415 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c09b050 con 0x55d7c9560520 2023-10-06T21:38:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc6619c700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 0x7fcc4c0a3b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc58012b60 tx=0x7fcc58004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc4bfff700 1 -- 172.21.15.31:0/3313030415 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc5c0040b0 con 0x55d7c9560520 2023-10-06T21:38:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.091+0000 7fcc4bfff700 1 -- 172.21.15.31:0/3313030415 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0a3a80 con 0x55d7c9560520 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 -- 172.21.15.31:0/3313030415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 msgr2=0x55d7c956ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956ae90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0cbdd0 tx=0x7fcc5c009310 comp rx=0 tx=0).stop 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 -- 172.21.15.31:0/3313030415 shutdown_connections 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c956d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c955ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/3313030415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956ae90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 -- 172.21.15.31:0/3313030415 >> 172.21.15.31:0/3313030415 conn(0x55d7c94d5200 msgr2=0x55d7c955c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 -- 172.21.15.31:0/3313030415 shutdown_connections 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.092+0000 7fcc7b49db80 1 -- 172.21.15.31:0/3313030415 wait complete. 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.093+0000 7fcc7b49db80 1 -- start start 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.093+0000 7fcc7b49db80 1 Processor -- start 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c9560520 0x55d7c956bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c956c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c94b3dd0 con 0x55d7c955fc00 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c94506b0 con 0x55d7c956b3d0 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c9450350 con 0x55d7c9560520 2023-10-06T21:38:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc7b49db80 10 client.0 _ll_register_callbacks cb 0x55d7c92ad7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47028/0 (socket says 172.21.15.31:47028) 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.094+0000 7fcc6599b700 1 -- 172.21.15.31:0/1839021467 learned_addr learned my addr 172.21.15.31:0/1839021467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6619c700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c956c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c9560520 0x55d7c956bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 -- 172.21.15.31:0/1839021467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c9560520 msgr2=0x55d7c956bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c9560520 0x55d7c956bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 -- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 msgr2=0x55d7c956c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c956c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 -- 172.21.15.31:0/1839021467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc58012610 con 0x55d7c955fc00 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc6599b700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c098160 tx=0x7fcc5c097430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653c14f28217d5af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc497fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c09bb50 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fcc5c09b6f0 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.095+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7c9553d50 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5c09bdf0 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 10 client.17559 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fcc601aa700 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d7c9553d50 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc5c0a3680 con 0x55d7c955fc00 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 10 client.17559 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 10 client.17559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 20 client.17559 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c957a5e0 0x55d7c957c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 4 client.17559 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:38:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.096+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7c94acdb0 con 0x55d7c957a5e0 2023-10-06T21:38:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.097+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c957a5e0 0x55d7c957c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.097+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c957a5e0 0x55d7c957c9d0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ded70d62760c1db server_cookie=99d04d768ddc7112 in_seq=0 out_seq=0 2023-10-06T21:38:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.097+0000 7fcc497fa700 10 client.17559 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc497fa700 1 -- 172.21.15.31:0/1839021467 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7c94acdb0 con 0x55d7c957a5e0 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc497fa700 10 client.17559 handle_command_reply: tid=0 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 client.17559 shutdown 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 msgr2=0x55d7c956b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c098160 tx=0x7fcc5c097430 comp rx=0 tx=0).stop 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 shutdown_connections 2023-10-06T21:38:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c956c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c957a5e0 0x55d7c957c9d0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c955fc00 0x55d7c956b7b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1839021467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c9560520 0x55d7c956bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 >> 172.21.15.31:0/1839021467 conn(0x55d7c94d5200 msgr2=0x55d7c955bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc497fa700 0 client.17559 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 shutdown_connections 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.098+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1839021467 wait complete. 2023-10-06T21:38:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.099+0000 7fcc7b49db80 20 client.17559 trim_cache size 0 max 16384 2023-10-06T21:38:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 Processor -- start 2023-10-06T21:38:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 -- start start 2023-10-06T21:38:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c95702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c955b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c93348c0 con 0x55d7c9560520 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c9570870 con 0x55d7c955fc00 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.114+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c95663a0 con 0x55d7c956b3d0 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47042/0 (socket says 172.21.15.31:47042) 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 -- 172.21.15.31:0/1159779567 learned_addr learned my addr 172.21.15.31:0/1159779567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6599b700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c95702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6619c700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c955b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 -- 172.21.15.31:0/1159779567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 msgr2=0x55d7c955b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c955b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 -- 172.21.15.31:0/1159779567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 msgr2=0x55d7c95702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c95702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 -- 172.21.15.31:0/1159779567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7c92c7c00 con 0x55d7c9560520 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6599b700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c95702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc6519a700 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000c00 tx=0x7fcc500043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce364ca96501780 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc54bff700 1 -- 172.21.15.31:0/1159779567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50017040 con 0x55d7c9560520 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.115+0000 7fcc54bff700 1 -- 172.21.15.31:0/1159779567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc5001b530 con 0x55d7c9560520 2023-10-06T21:38:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc54bff700 1 -- 172.21.15.31:0/1159779567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5001b730 con 0x55d7c9560520 2023-10-06T21:38:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1159779567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 msgr2=0x55d7c9564ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000c00 tx=0x7fcc500043f0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1159779567 shutdown_connections 2023-10-06T21:38:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c956b3d0 0x55d7c955b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c955fc00 0x55d7c95702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/1159779567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c9564ca0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1159779567 >> 172.21.15.31:0/1159779567 conn(0x55d7c94d5200 msgr2=0x55d7c955a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1159779567 shutdown_connections 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.116+0000 7fcc7b49db80 1 -- 172.21.15.31:0/1159779567 wait complete. 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 -- start start 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 Processor -- start 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c95cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c95cd910 con 0x55d7c9560520 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c95cda50 con 0x55d7c956b3d0 2023-10-06T21:38:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c95cdc10 con 0x55d7c955fc00 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.117+0000 7fcc7b49db80 10 client.0 _ll_register_callbacks cb 0x7fcc580079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:37752/0 (socket says 172.21.15.31:37752) 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 -- 172.21.15.31:0/4004239227 learned_addr learned my addr 172.21.15.31:0/4004239227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6619c700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c95cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 -- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 msgr2=0x55d7c95cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c95cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 -- 172.21.15.31:0/4004239227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 msgr2=0x55d7c956e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 -- 172.21.15.31:0/4004239227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5000bb80 con 0x55d7c955fc00 2023-10-06T21:38:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6619c700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c95cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc6599b700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c004240 tx=0x7fcc5c09ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b191d050f54ebd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc497fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.118+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c09b050 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fcc5c09bdf0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc5c0040b0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5c09b6f0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 10 client.27229 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d7c93051f0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc5c0040b0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc5c0974e0 con 0x55d7c955fc00 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 10 client.27229 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 20 client.27229 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c95cee50 0x55d7c95d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 4 client.27229 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.119+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7c94acad0 con 0x55d7c95cee50 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.120+0000 7fcc6519a700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c95cee50 0x55d7c95d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.120+0000 7fcc497fa700 10 client.27229 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.120+0000 7fcc6519a700 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c95cee50 0x55d7c95d1240 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4a0ff2ff9251dc server_cookie=d175f55868a89c6d in_seq=0 out_seq=0 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.485+0000 7fcc497fa700 1 -- 172.21.15.31:0/4004239227 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7c94acad0 con 0x55d7c95cee50 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.485+0000 7fcc497fa700 10 client.27229 handle_command_reply: tid=0 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.485+0000 7fcc7b49db80 1 client.27229 shutdown 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.485+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 msgr2=0x55d7c956dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.485+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c004240 tx=0x7fcc5c09ae90 comp rx=0 tx=0).stop 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 shutdown_connections 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7c956b3d0 0x55d7c95cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c955fc00 0x55d7c956dfd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d7c95cee50 0x55d7c95d1240 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 --2- 172.21.15.31:0/4004239227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7c9560520 0x55d7c956e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 >> 172.21.15.31:0/4004239227 conn(0x55d7c94d5200 msgr2=0x55d7c9565810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc497fa700 0 client.27229 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 shutdown_connections 2023-10-06T21:38:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.486+0000 7fcc7b49db80 1 -- 172.21.15.31:0/4004239227 wait complete. 2023-10-06T21:38:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.487+0000 7fcc7b49db80 20 client.27229 trim_cache size 0 max 16384 2023-10-06T21:38:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:38:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:38:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:38:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:38:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.491+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 msgr2=0x7fcc4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.491+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 0x7fcc4c0a3b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc58012b60 tx=0x7fcc58004b40 comp rx=0 tx=0).stop 2023-10-06T21:38:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.491+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 msgr2=0x7fcc60106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.491+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0036c0 tx=0x7fcc5c0057f0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 shutdown_connections 2023-10-06T21:38:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc60107770 0x7fcc601a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fcc4c0a1670 0x7fcc4c0a3b30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc601010d0 0x7fcc60106cf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 --2- 172.21.15.31:0/3963057245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc601061f0 0x7fcc60107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 >> 172.21.15.31:0/3963057245 conn(0x7fcc600fc950 msgr2=0x7fcc6010d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 shutdown_connections 2023-10-06T21:38:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:41.492+0000 7fcc67bff700 1 -- 172.21.15.31:0/3963057245 wait complete. 2023-10-06T21:38:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:41 smithi142 ceph-mon[102951]: pgmap v561: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2023-10-06T21:38:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:41 smithi031 ceph-mon[98539]: pgmap v561: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2023-10-06T21:38:41.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32461 6.38 MB/sec execute 150 sec latency 502.911 ms 2023-10-06T21:38:41.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:41 smithi146 ceph-mon[102809]: pgmap v561: 97 pgs: 97 active+clean; 651 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2023-10-06T21:38:42.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32585 6.40 MB/sec execute 151 sec latency 104.779 ms 2023-10-06T21:38:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:43 smithi142 ceph-mon[102951]: pgmap v562: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-10-06T21:38:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:43 smithi031 ceph-mon[98539]: pgmap v562: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-10-06T21:38:43.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32701 6.38 MB/sec execute 152 sec latency 64.363 ms 2023-10-06T21:38:43.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:43 smithi146 ceph-mon[102809]: pgmap v562: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-10-06T21:38:44.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32835 6.39 MB/sec execute 153 sec latency 68.672 ms 2023-10-06T21:38:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:45 smithi142 ceph-mon[102951]: pgmap v563: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:45 smithi031 ceph-mon[98539]: pgmap v563: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:45.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 32969 6.37 MB/sec execute 154 sec latency 66.180 ms 2023-10-06T21:38:45.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:45 smithi146 ceph-mon[102809]: pgmap v563: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:46.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33084 6.36 MB/sec execute 155 sec latency 71.965 ms 2023-10-06T21:38:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:47 smithi142 ceph-mon[102951]: pgmap v564: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:47 smithi031 ceph-mon[98539]: pgmap v564: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:47.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33200 6.35 MB/sec execute 156 sec latency 72.383 ms 2023-10-06T21:38:47.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:47 smithi146 ceph-mon[102809]: pgmap v564: 97 pgs: 97 active+clean; 645 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-10-06T21:38:48.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33315 6.32 MB/sec execute 157 sec latency 70.317 ms 2023-10-06T21:38:49.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33464 6.35 MB/sec execute 158 sec latency 218.868 ms 2023-10-06T21:38:50.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:50 smithi142 ceph-mon[102951]: pgmap v565: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-10-06T21:38:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:50 smithi031 ceph-mon[98539]: pgmap v565: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-10-06T21:38:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:50 smithi146 ceph-mon[102809]: pgmap v565: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-10-06T21:38:50.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33592 6.34 MB/sec execute 159 sec latency 68.543 ms 2023-10-06T21:38:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:51 smithi142 ceph-mon[102951]: pgmap v566: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-10-06T21:38:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:51 smithi031 ceph-mon[98539]: pgmap v566: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-10-06T21:38:51.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33716 6.34 MB/sec execute 160 sec latency 63.377 ms 2023-10-06T21:38:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:51 smithi146 ceph-mon[102809]: pgmap v566: 97 pgs: 97 active+clean; 653 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-10-06T21:38:52.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 33866 6.37 MB/sec execute 161 sec latency 64.220 ms 2023-10-06T21:38:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:53 smithi142 ceph-mon[102951]: pgmap v567: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 278 op/s 2023-10-06T21:38:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:53 smithi031 ceph-mon[98539]: pgmap v567: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 278 op/s 2023-10-06T21:38:53.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34023 6.41 MB/sec execute 162 sec latency 72.207 ms 2023-10-06T21:38:53.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:53 smithi146 ceph-mon[102809]: pgmap v567: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 278 op/s 2023-10-06T21:38:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.727+0000 7f65d01ff700 1 -- 172.21.15.31:0/3299670352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c8115f40 msgr2=0x7f65c8118330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.727+0000 7f65d01ff700 1 --2- 172.21.15.31:0/3299670352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c8115f40 0x7f65c8118330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f65c000b0a0 tx=0x7f65c000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 -- 172.21.15.31:0/3299670352 shutdown_connections 2023-10-06T21:38:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 --2- 172.21.15.31:0/3299670352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c8115f40 0x7f65c8118330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 --2- 172.21.15.31:0/3299670352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 --2- 172.21.15.31:0/3299670352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c81083f0 0x7f65c8108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 -- 172.21.15.31:0/3299670352 >> 172.21.15.31:0/3299670352 conn(0x7f65c806cfa0 msgr2=0x7f65c806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 -- 172.21.15.31:0/3299670352 shutdown_connections 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.728+0000 7f65d01ff700 1 -- 172.21.15.31:0/3299670352 wait complete. 2023-10-06T21:38:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.729+0000 7f65d01ff700 1 Processor -- start 2023-10-06T21:38:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 -- start start 2023-10-06T21:38:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c81083f0 0x7f65c810bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 0x7f65c81b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c810c8d0 con 0x7f65c810b5e0 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81b24b0 con 0x7f65c81083f0 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.730+0000 7f65d01ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81b25f0 con 0x7f65c8115f40 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c81083f0 0x7f65c810bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49104/0 (socket says 172.21.15.31:49104) 2023-10-06T21:38:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 -- 172.21.15.31:0/892314086 learned_addr learned my addr 172.21.15.31:0/892314086 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65ce79c700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 0x7f65c81b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 -- 172.21.15.31:0/892314086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 msgr2=0x7f65c81b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 0x7f65c81b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 -- 172.21.15.31:0/892314086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c81083f0 msgr2=0x7f65c810bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c81083f0 0x7f65c810bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 -- 172.21.15.31:0/892314086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4009520 con 0x7f65c810b5e0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65ce79c700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 0x7f65c81b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.731+0000 7f65cd79a700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f65b800e9a0 tx=0x7f65b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.732+0000 7f65beffd700 1 -- 172.21.15.31:0/892314086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009790 con 0x7f65c810b5e0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.732+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0009d00 con 0x7f65c810b5e0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.732+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c81b2c20 con 0x7f65c810b5e0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.732+0000 7f65beffd700 1 -- 172.21.15.31:0/892314086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65b8010430 con 0x7f65c810b5e0 2023-10-06T21:38:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.732+0000 7f65beffd700 1 -- 172.21.15.31:0/892314086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800cea0 con 0x7f65c810b5e0 2023-10-06T21:38:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.736+0000 7f65beffd700 1 -- 172.21.15.31:0/892314086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f65b800ca10 con 0x7f65c810b5e0 2023-10-06T21:38:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.738+0000 7f65e3c00b80 1 Processor -- start 2023-10-06T21:38:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.738+0000 7f65e3c00b80 1 -- start start 2023-10-06T21:38:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65beffd700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 0x7f65b40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56418462c400 con 0x5641846c2c00 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56418461dbc0 con 0x5641846c3520 2023-10-06T21:38:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56418461b4e0 con 0x5641846ce3d0 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65beffd700 1 -- 172.21.15.31:0/892314086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f65b8014070 con 0x7f65c810b5e0 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65ce79c700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 0x7f65b40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49106/0 (socket says 172.21.15.31:49106) 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65cdf9b700 1 -- 172.21.15.31:0/487727192 learned_addr learned my addr 172.21.15.31:0/487727192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.739+0000 7f65ce79c700 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 -- 172.21.15.31:0/487727192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 msgr2=0x5641846cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 -- 172.21.15.31:0/487727192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 msgr2=0x5641846c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 -- 172.21.15.31:0/487727192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564184441d80 con 0x5641846ce3d0 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.740+0000 7f65ce79c700 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d07c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009fd0 tx=0x7f65c0008c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d061ac563771c9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.741+0000 7f65b3fff700 1 -- 172.21.15.31:0/487727192 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000e040 con 0x5641846ce3d0 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.741+0000 7f65ce79c700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 0x7f65b40a3b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65c001c000 tx=0x7f65c0026000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.741+0000 7f65b3fff700 1 -- 172.21.15.31:0/487727192 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65c00092e0 con 0x5641846ce3d0 2023-10-06T21:38:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.741+0000 7f65b3fff700 1 -- 172.21.15.31:0/487727192 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0029440 con 0x5641846ce3d0 2023-10-06T21:38:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 -- 172.21.15.31:0/487727192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 msgr2=0x5641846d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d07c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009fd0 tx=0x7f65c0008c20 comp rx=0 tx=0).stop 2023-10-06T21:38:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 -- 172.21.15.31:0/487727192 shutdown_connections 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d07c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/487727192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 -- 172.21.15.31:0/487727192 >> 172.21.15.31:0/487727192 conn(0x564184638200 msgr2=0x5641846bf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 -- 172.21.15.31:0/487727192 shutdown_connections 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.742+0000 7f65e3c00b80 1 -- 172.21.15.31:0/487727192 wait complete. 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.743+0000 7f65e3c00b80 1 -- start start 2023-10-06T21:38:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.743+0000 7f65e3c00b80 1 Processor -- start 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c3520 0x5641846cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564184616dd0 con 0x5641846c3520 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641845b36b0 con 0x5641846ce3d0 2023-10-06T21:38:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641845b3350 con 0x5641846c2c00 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65e3c00b80 10 client.0 _ll_register_callbacks cb 0x5641844107a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48334/0 (socket says 172.21.15.31:48334) 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65cdf9b700 1 -- 172.21.15.31:0/3494171013 learned_addr learned my addr 172.21.15.31:0/3494171013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.744+0000 7f65ce79c700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c3520 0x5641846cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 -- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 msgr2=0x5641846cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 -- 172.21.15.31:0/3494171013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c3520 msgr2=0x5641846cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c3520 0x5641846cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 -- 172.21.15.31:0/3494171013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c4009af0 con 0x5641846c2c00 2023-10-06T21:38:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65ce79c700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65c4011d10 tx=0x7f65c400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d1b11e92418e63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.745+0000 7f65b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4016070 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f65c400fe40 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641846b6d50 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c401a3f0 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65e3c00b80 10 client.27241 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.746+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f65c8119010 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641846b6d50 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f65c40195f0 con 0x5641846c2c00 2023-10-06T21:38:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 10 client.27241 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 10 client.27241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 20 client.27241 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5641846de060 0x5641846e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 4 client.27241 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.747+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56418460fdb0 con 0x5641846de060 2023-10-06T21:38:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.748+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5641846de060 0x5641846e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.748+0000 7f65b17fa700 10 client.27241 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.748+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5641846de060 0x5641846e0480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa709440806baf2b server_cookie=ecc9f5c1cb26d374 in_seq=0 out_seq=0 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.749+0000 7f65b17fa700 1 -- 172.21.15.31:0/3494171013 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56418460fdb0 con 0x5641846de060 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.749+0000 7f65b17fa700 10 client.27241 handle_command_reply: tid=0 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.749+0000 7f65e3c00b80 1 client.27241 shutdown 2023-10-06T21:38:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 msgr2=0x5641846cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65c4011d10 tx=0x7f65c400f6a0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 shutdown_connections 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c2c00 0x5641846cb170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5641846de060 0x5641846e0480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3494171013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c3520 0x5641846cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 >> 172.21.15.31:0/3494171013 conn(0x564184638200 msgr2=0x5641846bec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65b17fa700 0 client.27241 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 shutdown_connections 2023-10-06T21:38:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3494171013 wait complete. 2023-10-06T21:38:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.750+0000 7f65e3c00b80 20 client.27241 trim_cache size 0 max 16384 2023-10-06T21:38:54.769 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34188 6.45 MB/sec execute 163 sec latency 69.246 ms 2023-10-06T21:38:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.776+0000 7f65e3c00b80 1 Processor -- start 2023-10-06T21:38:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.776+0000 7f65e3c00b80 1 -- start start 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 0x5641846d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641844978c0 con 0x5641846c2c00 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641846c6fe0 con 0x5641846ce3d0 2023-10-06T21:38:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641846b6890 con 0x5641846c3520 2023-10-06T21:38:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65cd79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 0x5641846d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65ce79c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49114/0 (socket says 172.21.15.31:49114) 2023-10-06T21:38:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.777+0000 7f65cdf9b700 1 -- 172.21.15.31:0/3754511811 learned_addr learned my addr 172.21.15.31:0/3754511811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cd79a700 1 -- 172.21.15.31:0/3754511811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 msgr2=0x5641846c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cd79a700 1 -- 172.21.15.31:0/3754511811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 msgr2=0x5641846ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cd79a700 1 -- 172.21.15.31:0/3754511811 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56418442ac00 con 0x5641846c3520 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.778+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.779+0000 7f65cd79a700 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 0x5641846d9670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65b80ca8b0 tx=0x7f65b80092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=418b58084054d424 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.779+0000 7f65bcff9700 1 -- 172.21.15.31:0/3754511811 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b809a140 con 0x5641846c3520 2023-10-06T21:38:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.779+0000 7f65bcff9700 1 -- 172.21.15.31:0/3754511811 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65b809e290 con 0x5641846c3520 2023-10-06T21:38:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.779+0000 7f65bcff9700 1 -- 172.21.15.31:0/3754511811 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8099630 con 0x5641846c3520 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.780+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3754511811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 msgr2=0x5641846d9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.780+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 0x5641846d9670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65b80ca8b0 tx=0x7f65b80092d0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3754511811 shutdown_connections 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846ce3d0 0x5641846c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/3754511811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846c3520 0x5641846d9670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3754511811 >> 172.21.15.31:0/3754511811 conn(0x564184638200 msgr2=0x5641846cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3754511811 shutdown_connections 2023-10-06T21:38:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 -- 172.21.15.31:0/3754511811 wait complete. 2023-10-06T21:38:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.781+0000 7f65e3c00b80 1 -- start start 2023-10-06T21:38:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 Processor -- start 2023-10-06T21:38:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564184730fc0 con 0x5641846c2c00 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564184731130 con 0x5641846c3520 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641847312f0 con 0x5641846ce3d0 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.782+0000 7f65e3c00b80 10 client.0 _ll_register_callbacks cb 0x5641844107a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cd79a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cd79a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:34742/0 (socket says 172.21.15.31:34742) 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cd79a700 1 -- 172.21.15.31:0/1317033273 learned_addr learned my addr 172.21.15.31:0/1317033273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65ce79c700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 -- 172.21.15.31:0/1317033273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 msgr2=0x5641846d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 -- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 msgr2=0x5641846d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65cdf9b700 1 -- 172.21.15.31:0/1317033273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c8119010 con 0x5641846c2c00 2023-10-06T21:38:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.783+0000 7f65ce79c700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:38:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.784+0000 7f65cdf9b700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846d2f00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005a80 tx=0x7f65c4019dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a143ed96605e0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.784+0000 7f65b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.784+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4016040 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.784+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f65c40046d0 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.784+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65c00092e0 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.785+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c400fec0 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.785+0000 7f65e3c00b80 10 client.17583 fetch_fsmap learned FSMap version 49 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.785+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f65c81b2c20 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.785+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65c00092e0 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f65c4005430 con 0x5641846c2c00 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 10 client.17583 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:38:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 20 client.17583 populate_metadata read hostname 'smithi031' 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564184732a20 0x564184734e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 4 client.17583 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56418460fad0 con 0x564184732a20 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.786+0000 7f65cd79a700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564184732a20 0x564184734e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.787+0000 7f65cd79a700 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564184732a20 0x564184734e40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146b49a14db143c8 server_cookie=ec7f4a7c7fed0236 in_seq=0 out_seq=0 2023-10-06T21:38:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.787+0000 7f65b17fa700 10 client.17583 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.834+0000 7f65b17fa700 1 -- 172.21.15.31:0/1317033273 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56418460fad0 con 0x564184732a20 2023-10-06T21:38:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.834+0000 7f65b17fa700 10 client.17583 handle_command_reply: tid=0 2023-10-06T21:38:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.834+0000 7f65e3c00b80 1 client.17583 shutdown 2023-10-06T21:38:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.834+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 msgr2=0x5641846d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.834+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846d2f00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005a80 tx=0x7f65c4019dc0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 shutdown_connections 2023-10-06T21:38:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641846ce3d0 0x5641846d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564184732a20 0x564184734e40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641846c2c00 0x5641846d2f00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 --2- 172.21.15.31:0/1317033273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641846c3520 0x5641846d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65b17fa700 0 client.17583 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 >> 172.21.15.31:0/1317033273 conn(0x564184638200 msgr2=0x5641846c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 shutdown_connections 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.835+0000 7f65e3c00b80 1 -- 172.21.15.31:0/1317033273 wait complete. 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.836+0000 7f65e3c00b80 20 client.17583 trim_cache size 0 max 16384 2023-10-06T21:38:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:38:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:38:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:38:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:38:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 msgr2=0x7f65b40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 0x7f65b40a3b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65c001c000 tx=0x7f65c0026000 comp rx=0 tx=0).stop 2023-10-06T21:38:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 msgr2=0x7f65c810c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:38:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f65b800e9a0 tx=0x7f65b800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 shutdown_connections 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c8115f40 0x7f65c81b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c81083f0 0x7f65c810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f65b40a1670 0x7f65b40a3b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 --2- 172.21.15.31:0/892314086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c810b5e0 0x7f65c810c070 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.838+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 >> 172.21.15.31:0/892314086 conn(0x7f65c806cfa0 msgr2=0x7f65c80718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.839+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 shutdown_connections 2023-10-06T21:38:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:38:54.839+0000 7f65d01ff700 1 -- 172.21.15.31:0/892314086 wait complete. 2023-10-06T21:38:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:55 smithi142 ceph-mon[102951]: pgmap v568: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:55 smithi031 ceph-mon[98539]: pgmap v568: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:55.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34282 6.44 MB/sec execute 164 sec latency 174.416 ms 2023-10-06T21:38:55.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:55 smithi146 ceph-mon[102809]: pgmap v568: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:56.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34370 6.42 MB/sec execute 165 sec latency 73.543 ms 2023-10-06T21:38:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:38:57 smithi142 ceph-mon[102951]: pgmap v569: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:38:57 smithi031 ceph-mon[98539]: pgmap v569: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:57.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34489 6.43 MB/sec execute 166 sec latency 66.965 ms 2023-10-06T21:38:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:38:57 smithi146 ceph-mon[102809]: pgmap v569: 97 pgs: 97 active+clean; 664 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 175 op/s 2023-10-06T21:38:58.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34638 6.47 MB/sec execute 167 sec latency 78.698 ms 2023-10-06T21:38:59.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34753 6.48 MB/sec execute 168 sec latency 75.309 ms 2023-10-06T21:39:00.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:00 smithi142 ceph-mon[102951]: pgmap v570: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 256 op/s 2023-10-06T21:39:00.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:00 smithi031 ceph-mon[98539]: pgmap v570: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 256 op/s 2023-10-06T21:39:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:00 smithi146 ceph-mon[102809]: pgmap v570: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 256 op/s 2023-10-06T21:39:00.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34872 6.48 MB/sec execute 169 sec latency 134.963 ms 2023-10-06T21:39:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:01 smithi142 ceph-mon[102951]: pgmap v571: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-10-06T21:39:01.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:01 smithi031 ceph-mon[98539]: pgmap v571: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-10-06T21:39:01.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 34974 6.47 MB/sec execute 170 sec latency 229.622 ms 2023-10-06T21:39:01.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:01 smithi146 ceph-mon[102809]: pgmap v571: 97 pgs: 97 active+clean; 673 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-10-06T21:39:02.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35067 6.46 MB/sec execute 171 sec latency 67.185 ms 2023-10-06T21:39:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:03 smithi142 ceph-mon[102951]: pgmap v572: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:39:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:03 smithi142 ceph-mon[102951]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:39:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:03 smithi031 ceph-mon[98539]: pgmap v572: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:39:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:03 smithi031 ceph-mon[98539]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:39:03.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35143 6.43 MB/sec execute 172 sec latency 114.342 ms 2023-10-06T21:39:03.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:03 smithi146 ceph-mon[102809]: pgmap v572: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:39:03.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:03 smithi146 ceph-mon[102809]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:39:04.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35262 6.44 MB/sec execute 173 sec latency 87.742 ms 2023-10-06T21:39:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:05 smithi142 ceph-mon[102951]: pgmap v574: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:05 smithi031 ceph-mon[98539]: pgmap v574: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:05.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35400 6.44 MB/sec execute 174 sec latency 72.434 ms 2023-10-06T21:39:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:05 smithi146 ceph-mon[102809]: pgmap v574: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:06.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35527 6.43 MB/sec execute 175 sec latency 71.174 ms 2023-10-06T21:39:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:07 smithi142 ceph-mon[102951]: pgmap v575: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:07.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:07 smithi031 ceph-mon[98539]: pgmap v575: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:07.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35649 6.43 MB/sec execute 176 sec latency 137.581 ms 2023-10-06T21:39:07.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:07 smithi146 ceph-mon[102809]: pgmap v575: 97 pgs: 97 active+clean; 685 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 209 op/s 2023-10-06T21:39:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 -- 172.21.15.31:0/3287993785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c81074e0 msgr2=0x7fd6c8107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 --2- 172.21.15.31:0/3287993785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c81074e0 0x7fd6c8107960 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40099c0 tx=0x7fd6b4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 -- 172.21.15.31:0/3287993785 shutdown_connections 2023-10-06T21:39:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 --2- 172.21.15.31:0/3287993785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 0x7fd6c8114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 --2- 172.21.15.31:0/3287993785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c810d5a0 0x7fd6c810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 --2- 172.21.15.31:0/3287993785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c81074e0 0x7fd6c8107960 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 -- 172.21.15.31:0/3287993785 >> 172.21.15.31:0/3287993785 conn(0x7fd6c8076cd0 msgr2=0x7fd6c8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.069+0000 7fd6cedff700 1 -- 172.21.15.31:0/3287993785 shutdown_connections 2023-10-06T21:39:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.070+0000 7fd6cedff700 1 -- 172.21.15.31:0/3287993785 wait complete. 2023-10-06T21:39:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.071+0000 7fd6cedff700 1 Processor -- start 2023-10-06T21:39:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.071+0000 7fd6cedff700 1 -- start start 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 0x7fd6c819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 0x7fd6c81a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c81a1590 con 0x7fd6c810d5a0 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c81a1700 con 0x7fd6c8107ea0 2023-10-06T21:39:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cedff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c81a18c0 con 0x7fd6c81074e0 2023-10-06T21:39:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6ccb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6ccb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:53024/0 (socket says 172.21.15.31:53024) 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1576550620 learned_addr learned my addr 172.21.15.31:0/1576550620 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.072+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 0x7fd6c81a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 0x7fd6c819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1576550620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 msgr2=0x7fd6c819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 0x7fd6c819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 msgr2=0x7fd6c81a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 0x7fd6c81a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1576550620 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40095f0 con 0x7fd6c81074e0 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 0x7fd6c81a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00e9a0 tx=0x7fd6bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.073+0000 7fd6c5ffb700 1 -- 172.21.15.31:0/1576550620 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc009790 con 0x7fd6c81074e0 2023-10-06T21:39:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.074+0000 7fd6c5ffb700 1 -- 172.21.15.31:0/1576550620 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6bc00cbd0 con 0x7fd6c81074e0 2023-10-06T21:39:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.074+0000 7fd6c5ffb700 1 -- 172.21.15.31:0/1576550620 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc018680 con 0x7fd6c81074e0 2023-10-06T21:39:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.074+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c81a1ba0 con 0x7fd6c81074e0 2023-10-06T21:39:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.074+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c81aa500 con 0x7fd6c81074e0 2023-10-06T21:39:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.080+0000 7fd6c5ffb700 1 -- 172.21.15.31:0/1576550620 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd6bc00cd40 con 0x7fd6c81074e0 2023-10-06T21:39:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.081+0000 7fd6e2716b80 1 Processor -- start 2023-10-06T21:39:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.082+0000 7fd6e2716b80 1 -- start start 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc6944400 con 0x55abc69db520 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc6935bc0 con 0x55abc69dac00 2023-10-06T21:39:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.083+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69334e0 con 0x55abc69e63d0 2023-10-06T21:39:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:53026/0 (socket says 172.21.15.31:53026) 2023-10-06T21:39:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6c7fff700 1 -- 172.21.15.31:0/3759066952 learned_addr learned my addr 172.21.15.31:0/3759066952 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6c5ffb700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 0x7fd6b80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.084+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 -- 172.21.15.31:0/3759066952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 msgr2=0x55abc69dafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69dafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 -- 172.21.15.31:0/3759066952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 msgr2=0x55abc69e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c5ffb700 1 -- 172.21.15.31:0/1576550620 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fd6bc014070 con 0x7fd6c81074e0 2023-10-06T21:39:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 -- 172.21.15.31:0/3759066952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc6759d80 con 0x55abc69e63d0 2023-10-06T21:39:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 0x7fd6b80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.085+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009fd0 tx=0x7fd6b4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24eddf5fe6111948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.086+0000 7fd6b2ffd700 1 -- 172.21.15.31:0/3759066952 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b401d040 con 0x55abc69e63d0 2023-10-06T21:39:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.086+0000 7fd6b2ffd700 1 -- 172.21.15.31:0/3759066952 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6b400b790 con 0x55abc69e63d0 2023-10-06T21:39:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.086+0000 7fd6b2ffd700 1 -- 172.21.15.31:0/3759066952 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40039a0 con 0x55abc69e63d0 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 -- 172.21.15.31:0/3759066952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 msgr2=0x55abc69e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009fd0 tx=0x7fd6b4009510 comp rx=0 tx=0).stop 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 0x7fd6b80a3b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009fd0 tx=0x7fd6c0004b40 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 -- 172.21.15.31:0/3759066952 shutdown_connections 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69e63d0 0x55abc69e87c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69dafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/3759066952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.087+0000 7fd6e2716b80 1 -- 172.21.15.31:0/3759066952 >> 172.21.15.31:0/3759066952 conn(0x55abc6950200 msgr2=0x55abc69d7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.088+0000 7fd6e2716b80 1 -- 172.21.15.31:0/3759066952 shutdown_connections 2023-10-06T21:39:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.088+0000 7fd6e2716b80 1 -- 172.21.15.31:0/3759066952 wait complete. 2023-10-06T21:39:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.088+0000 7fd6e2716b80 1 -- start start 2023-10-06T21:39:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.089+0000 7fd6e2716b80 1 Processor -- start 2023-10-06T21:39:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69e63d0 0x55abc69e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc692edd0 con 0x55abc69db520 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc68cb6b0 con 0x55abc69e63d0 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc68cb350 con 0x55abc69dac00 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6e2716b80 10 client.0 _ll_register_callbacks cb 0x55abc67287a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6ccb9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53358/0 (socket says 172.21.15.31:53358) 2023-10-06T21:39:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6c7fff700 1 -- 172.21.15.31:0/277936460 learned_addr learned my addr 172.21.15.31:0/277936460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.090+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69e63d0 0x55abc69e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69e63d0 msgr2=0x55abc69e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69e63d0 0x55abc69e37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/277936460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 msgr2=0x55abc69e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/277936460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c8114c60 con 0x55abc69dac00 2023-10-06T21:39:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e5e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0caa40 tx=0x7fd6bc099cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17eb6ccf62bbfcab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc09a2d0 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fd6bc0cc050 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.091+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abc69ced50 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6bc09e420 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 10 client.27259 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55abc6742c00 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55abc69ced50 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6bc0a21d0 con 0x55abc69dac00 2023-10-06T21:39:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 10 client.27259 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 10 client.27259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 20 client.27259 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc69f5e40 0x55abc69f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 4 client.27259 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.092+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abc6927db0 con 0x55abc69f5e40 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.093+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc69f5e40 0x55abc69f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.093+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc69f5e40 0x55abc69f8260 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c67b38506d7b4b4 server_cookie=3396556bd72bad6f in_seq=0 out_seq=0 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.093+0000 7fd6a7fff700 10 client.27259 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6a7fff700 1 -- 172.21.15.31:0/277936460 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abc6927db0 con 0x55abc69f5e40 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6a7fff700 10 client.27259 handle_command_reply: tid=0 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 client.27259 shutdown 2023-10-06T21:39:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 msgr2=0x55abc69e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e5e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0caa40 tx=0x7fd6bc099cb0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 shutdown_connections 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69e63d0 0x55abc69e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e5e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.094+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc69f5e40 0x55abc69f8260 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/277936460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69db520 0x55abc69e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 >> 172.21.15.31:0/277936460 conn(0x55abc6950200 msgr2=0x55abc69d6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6a7fff700 0 client.27259 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 shutdown_connections 2023-10-06T21:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6e2716b80 1 -- 172.21.15.31:0/277936460 wait complete. 2023-10-06T21:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.095+0000 7fd6e2716b80 20 client.27259 trim_cache size 0 max 16384 2023-10-06T21:39:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.122+0000 7fd6e2716b80 1 Processor -- start 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.122+0000 7fd6e2716b80 1 -- start start 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.122+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.122+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 0x55abc69e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.122+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc67af8c0 con 0x55abc69e63d0 2023-10-06T21:39:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69def40 con 0x55abc69dac00 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69e32a0 con 0x55abc69db520 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 0x55abc69e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 0x55abc69e87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:53044/0 (socket says 172.21.15.31:53044) 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54130/0 (socket says 172.21.15.31:54130) 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6c7fff700 1 -- 172.21.15.31:0/1352958413 learned_addr learned my addr 172.21.15.31:0/1352958413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1352958413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 msgr2=0x55abc69e87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 0x55abc69e87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1352958413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 msgr2=0x55abc69f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.123+0000 7fd6ccb9b700 1 -- 172.21.15.31:0/1352958413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc67801f0 con 0x55abc69dac00 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.124+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc098620 tx=0x7fd6bc0cd370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb2ea351992dc1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.124+0000 7fd6b37fe700 1 -- 172.21.15.31:0/1352958413 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b0070 con 0x55abc69dac00 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.124+0000 7fd6b37fe700 1 -- 172.21.15.31:0/1352958413 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6bc099a10 con 0x55abc69dac00 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.124+0000 7fd6b37fe700 1 -- 172.21.15.31:0/1352958413 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc09e120 con 0x55abc69dac00 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1352958413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 msgr2=0x55abc69ddbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc098620 tx=0x7fd6bc0cd370 comp rx=0 tx=0).stop 2023-10-06T21:39:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1352958413 shutdown_connections 2023-10-06T21:39:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69dac00 0x55abc69ddbe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.125+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1352958413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69db520 0x55abc69e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.126+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1352958413 >> 172.21.15.31:0/1352958413 conn(0x55abc6950200 msgr2=0x55abc69fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.126+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1352958413 shutdown_connections 2023-10-06T21:39:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.126+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1352958413 wait complete. 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.126+0000 7fd6e2716b80 1 -- start start 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.127+0000 7fd6e2716b80 1 Processor -- start 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69dfdd0 con 0x55abc69e63d0 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69dff40 con 0x55abc69db520 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc69e0100 con 0x55abc69dac00 2023-10-06T21:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6e2716b80 10 client.0 _ll_register_callbacks cb 0x7fd6bc09a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:54132/0 (socket says 172.21.15.31:54132) 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 -- 172.21.15.31:0/1784471448 learned_addr learned my addr 172.21.15.31:0/1784471448 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6cd39c700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 -- 172.21.15.31:0/1784471448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 msgr2=0x55abc69e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 -- 172.21.15.31:0/1784471448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 msgr2=0x55abc69df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.128+0000 7fd6c7fff700 1 -- 172.21.15.31:0/1784471448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abc6742c00 con 0x55abc69db520 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.129+0000 7fd6c7fff700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009990 tx=0x7fd6b4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e5405d96349b6c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.129+0000 7fd6a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.129+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b401d040 con 0x55abc69db520 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.129+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6b400b790 con 0x55abc69db520 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.129+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8681+0+0 (secure 0 0 0) 0x7fd6b4030130 con 0x55abc69db520 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6b4030310 con 0x55abc69db520 2023-10-06T21:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 10 client.27332 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55abc6758250 con 0x55abc69db520 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6b400b790 con 0x55abc69db520 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6b40038f0 con 0x55abc69db520 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 10 client.27332 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 10 client.27332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 20 client.27332 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc6a4abd0 0x55abc6a4cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 4 client.27332 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.130+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abc6927ad0 con 0x55abc6a4abd0 2023-10-06T21:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.131+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc6a4abd0 0x55abc6a4cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.131+0000 7fd6ccb9b700 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc6a4abd0 0x55abc6a4cff0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3547494fe11fdb13 server_cookie=93b96c0a15e4bbf6 in_seq=0 out_seq=0 2023-10-06T21:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.131+0000 7fd6a7fff700 10 client.27332 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.190+0000 7fd6a7fff700 1 -- 172.21.15.31:0/1784471448 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abc6927ad0 con 0x55abc6a4abd0 2023-10-06T21:39:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.190+0000 7fd6a7fff700 10 client.27332 handle_command_reply: tid=0 2023-10-06T21:39:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.190+0000 7fd6e2716b80 1 client.27332 shutdown 2023-10-06T21:39:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.191+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 msgr2=0x55abc69df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.191+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009990 tx=0x7fd6b4009580 comp rx=0 tx=0).stop 2023-10-06T21:39:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 shutdown_connections 2023-10-06T21:39:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55abc6a4abd0 0x55abc6a4cff0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55abc69e63d0 0x55abc69df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abc69dac00 0x55abc69e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 --2- 172.21.15.31:0/1784471448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abc69db520 0x55abc69df350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 >> 172.21.15.31:0/1784471448 conn(0x55abc6950200 msgr2=0x55abc69f2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6a7fff700 0 client.27332 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 shutdown_connections 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 1 -- 172.21.15.31:0/1784471448 wait complete. 2023-10-06T21:39:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.192+0000 7fd6e2716b80 20 client.27332 trim_cache size 0 max 16384 2023-10-06T21:39:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:39:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:39:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:39:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:39:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 msgr2=0x7fd6b80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 0x7fd6b80a3b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009fd0 tx=0x7fd6c0004b40 comp rx=0 tx=0).stop 2023-10-06T21:39:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 msgr2=0x7fd6c819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00e9a0 tx=0x7fd6bc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 shutdown_connections 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fd6b80a16b0 0x7fd6b80a3b70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6c810d5a0 0x7fd6c81a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6c81074e0 0x7fd6c819d490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 --2- 172.21.15.31:0/1576550620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c8107ea0 0x7fd6c819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 >> 172.21.15.31:0/1576550620 conn(0x7fd6c8076cd0 msgr2=0x7fd6c8077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 shutdown_connections 2023-10-06T21:39:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:08.195+0000 7fd6cedff700 1 -- 172.21.15.31:0/1576550620 wait complete. 2023-10-06T21:39:08.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35741 6.41 MB/sec execute 177 sec latency 251.669 ms 2023-10-06T21:39:09.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35841 6.38 MB/sec execute 178 sec latency 73.321 ms 2023-10-06T21:39:10.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:10 smithi142 ceph-mon[102951]: pgmap v576: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:10.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:10 smithi031 ceph-mon[98539]: pgmap v576: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:10 smithi146 ceph-mon[102809]: pgmap v576: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:10.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 35970 6.43 MB/sec execute 179 sec latency 133.564 ms 2023-10-06T21:39:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:11 smithi142 ceph-mon[102951]: pgmap v577: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:11.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:11 smithi031 ceph-mon[98539]: pgmap v577: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:11.770 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36070 6.41 MB/sec execute 180 sec latency 285.645 ms 2023-10-06T21:39:11.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:11 smithi146 ceph-mon[102809]: pgmap v577: 97 pgs: 97 active+clean; 694 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2023-10-06T21:39:12.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36187 6.39 MB/sec execute 181 sec latency 65.102 ms 2023-10-06T21:39:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:13 smithi142 ceph-mon[102951]: pgmap v578: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:13 smithi031 ceph-mon[98539]: pgmap v578: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:13.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36315 6.40 MB/sec execute 182 sec latency 63.508 ms 2023-10-06T21:39:13.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:13 smithi146 ceph-mon[102809]: pgmap v578: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:14.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36438 6.39 MB/sec execute 183 sec latency 74.958 ms 2023-10-06T21:39:15.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:15 smithi142 ceph-mon[102951]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:39:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:15 smithi031 ceph-mon[98539]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:39:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:15 smithi146 ceph-mon[102809]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:39:15.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36569 6.36 MB/sec execute 184 sec latency 102.552 ms 2023-10-06T21:39:16.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:16 smithi031 ceph-mon[98539]: pgmap v580: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:16 smithi146 ceph-mon[102809]: pgmap v580: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:16.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:16 smithi142 ceph-mon[102951]: pgmap v580: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:16.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36622 6.34 MB/sec execute 185 sec latency 621.721 ms 2023-10-06T21:39:17.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:17 smithi142 ceph-mon[102951]: pgmap v581: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:17.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:17 smithi031 ceph-mon[98539]: pgmap v581: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:17.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36737 6.33 MB/sec execute 186 sec latency 71.036 ms 2023-10-06T21:39:17.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:17 smithi146 ceph-mon[102809]: pgmap v581: 97 pgs: 97 active+clean; 705 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2023-10-06T21:39:18.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36872 6.34 MB/sec execute 187 sec latency 69.639 ms 2023-10-06T21:39:19.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 36989 6.34 MB/sec execute 188 sec latency 71.868 ms 2023-10-06T21:39:20.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:20 smithi142 ceph-mon[102951]: pgmap v582: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:20.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:20 smithi031 ceph-mon[98539]: pgmap v582: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:20.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:20 smithi146 ceph-mon[102809]: pgmap v582: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:20.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37126 6.34 MB/sec execute 189 sec latency 74.357 ms 2023-10-06T21:39:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2719444975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38073c00 msgr2=0x7f5c3810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2719444975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38073c00 0x7f5c3810d160 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009990 tx=0x7f5c20009ca0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2719444975 shutdown_connections 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2719444975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c3810d6a0 0x7f5c3810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2719444975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c380732e0 0x7f5c380736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2719444975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38073c00 0x7f5c3810d160 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2719444975 >> 172.21.15.31:0/2719444975 conn(0x7f5c380fc930 msgr2=0x7f5c380fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2719444975 shutdown_connections 2023-10-06T21:39:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.356+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2719444975 wait complete. 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.357+0000 7f5c3e3ff700 1 Processor -- start 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.357+0000 7f5c3e3ff700 1 -- start start 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38073c00 0x7f5c38199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3810d6a0 0x7f5c3819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3819e300 con 0x7f5c380732e0 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3819e470 con 0x7f5c38073c00 2023-10-06T21:39:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3e3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3819e630 con 0x7f5c3810d6a0 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51468/0 (socket says 172.21.15.31:51468) 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c37fff700 1 -- 172.21.15.31:0/2657085436 learned_addr learned my addr 172.21.15.31:0/2657085436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38073c00 0x7f5c38199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3810d6a0 0x7f5c3819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c37fff700 1 -- 172.21.15.31:0/2657085436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3810d6a0 msgr2=0x7f5c3819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.358+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3810d6a0 0x7f5c3819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c37fff700 1 -- 172.21.15.31:0/2657085436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38073c00 msgr2=0x7f5c38199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38073c00 0x7f5c38199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c37fff700 1 -- 172.21.15.31:0/2657085436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200095f0 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800d980 tx=0x7f5c2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c35ffb700 1 -- 172.21.15.31:0/2657085436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28004a30 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c35ffb700 1 -- 172.21.15.31:0/2657085436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c280054b0 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3819e910 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c35ffb700 1 -- 172.21.15.31:0/2657085436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800fab0 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.359+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c381a6150 con 0x7f5c380732e0 2023-10-06T21:39:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.362+0000 7f5c35ffb700 1 -- 172.21.15.31:0/2657085436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5c2800fc10 con 0x7f5c380732e0 2023-10-06T21:39:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c35ffb700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 0x7f5c240a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 Processor -- start 2023-10-06T21:39:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c35ffb700 1 -- 172.21.15.31:0/2657085436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5c280ccff0 con 0x7f5c380732e0 2023-10-06T21:39:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 -- start start 2023-10-06T21:39:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 0x7f5c240a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b5ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b6d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969ac9400 con 0x563969b60520 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969ababc0 con 0x563969b6b3d0 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.364+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969ab84e0 con 0x563969b5fc00 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51484/0 (socket says 172.21.15.31:51484) 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 -- 172.21.15.31:0/252878486 learned_addr learned my addr 172.21.15.31:0/252878486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c37fff700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b5ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b6d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 0x7f5c240a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c3819a930 tx=0x7f5c2000b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 -- 172.21.15.31:0/252878486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 msgr2=0x563969b5ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b5ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 -- 172.21.15.31:0/252878486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 msgr2=0x563969b6d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b6d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c2ffff700 1 -- 172.21.15.31:0/252878486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639698ded80 con 0x563969b60520 2023-10-06T21:39:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.365+0000 7f5c37fff700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b5ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5c20015000 tx=0x7f5c20020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1ec2313e4489d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c2e7fc700 1 -- 172.21.15.31:0/252878486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20004700 con 0x563969b60520 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c2e7fc700 1 -- 172.21.15.31:0/252878486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c2001e040 con 0x563969b60520 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c2e7fc700 1 -- 172.21.15.31:0/252878486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000f920 con 0x563969b60520 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/252878486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 msgr2=0x563969b6ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.366+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5c20015000 tx=0x7f5c20020000 comp rx=0 tx=0).stop 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/252878486 shutdown_connections 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b6d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b5ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/252878486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b6ae90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/252878486 >> 172.21.15.31:0/252878486 conn(0x563969ad5200 msgr2=0x563969b5c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/252878486 shutdown_connections 2023-10-06T21:39:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/252878486 wait complete. 2023-10-06T21:39:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 -- start start 2023-10-06T21:39:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.367+0000 7f5c51dc0b80 1 Processor -- start 2023-10-06T21:39:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 0x563969b686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b6b3d0 0x563969b68c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969ab3dd0 con 0x563969b5fc00 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969a506b0 con 0x563969b60520 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969a50350 con 0x563969b6b3d0 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c51dc0b80 10 client.0 _ll_register_callbacks cb 0x5639698ad7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51500/0 (socket says 172.21.15.31:51500) 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c37fff700 1 -- 172.21.15.31:0/4141316106 learned_addr learned my addr 172.21.15.31:0/4141316106 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 0x563969b686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.368+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b6b3d0 0x563969b68c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 -- 172.21.15.31:0/4141316106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b6b3d0 msgr2=0x563969b68c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b6b3d0 0x563969b68c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 -- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 msgr2=0x563969b686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 0x563969b686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 -- 172.21.15.31:0/4141316106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c381a6150 con 0x563969b5fc00 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 0x563969b686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c37fff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5c28096990 tx=0x7f5c28005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a769b8277dce2ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c177fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2809a380 con 0x563969b5fc00 2023-10-06T21:39:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563969b53d50 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5c280a22b0 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c28099410 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c51dc0b80 10 client.17613 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.369+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f5c381107d0 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563969b53d50 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5c2809d070 con 0x563969b5fc00 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 20 client.17613 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969b7aef0 0x563969b7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 4 client.17613 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563969aacdb0 con 0x563969b7aef0 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.370+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969b7aef0 0x563969b7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.371+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969b7aef0 0x563969b7d2e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc862d9ee824ffa7 server_cookie=4a0b350e2364d298 in_seq=0 out_seq=0 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.371+0000 7f5c177fe700 10 client.17613 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c177fe700 1 -- 172.21.15.31:0/4141316106 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563969aacdb0 con 0x563969b7aef0 2023-10-06T21:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c177fe700 10 client.17613 handle_command_reply: tid=0 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 client.17613 shutdown 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 msgr2=0x563969b681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5c28096990 tx=0x7f5c28005750 comp rx=0 tx=0).stop 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 shutdown_connections 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b6b3d0 0x563969b68c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969b7aef0 0x563969b7d2e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b5fc00 0x563969b681a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/4141316106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b60520 0x563969b686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c177fe700 0 client.17613 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 >> 172.21.15.31:0/4141316106 conn(0x563969ad5200 msgr2=0x563969b5bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 shutdown_connections 2023-10-06T21:39:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/4141316106 wait complete. 2023-10-06T21:39:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.373+0000 7f5c51dc0b80 20 client.17613 trim_cache size 0 max 16384 2023-10-06T21:39:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.390+0000 7f5c51dc0b80 1 Processor -- start 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.390+0000 7f5c51dc0b80 1 -- start start 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b77120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639699348c0 con 0x563969b60520 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969b6da10 con 0x563969b6b3d0 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969b6dbd0 con 0x563969b5fc00 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51510/0 (socket says 172.21.15.31:51510) 2023-10-06T21:39:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 -- 172.21.15.31:0/636456437 learned_addr learned my addr 172.21.15.31:0/636456437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c37fff700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b77120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 -- 172.21.15.31:0/636456437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 msgr2=0x563969b62aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b62aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 -- 172.21.15.31:0/636456437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 msgr2=0x563969b77120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b77120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c2ffff700 1 -- 172.21.15.31:0/636456437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639698c7c00 con 0x563969b60520 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b77120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:39:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.391+0000 7f5c37fff700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b62aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5c200057a0 tx=0x7f5c20009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd2794b92ecca77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c2effd700 1 -- 172.21.15.31:0/636456437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20035040 con 0x563969b60520 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c2effd700 1 -- 172.21.15.31:0/636456437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c20029040 con 0x563969b60520 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c2effd700 1 -- 172.21.15.31:0/636456437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000fc30 con 0x563969b60520 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/636456437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 msgr2=0x563969b68a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5c200057a0 tx=0x7f5c20009290 comp rx=0 tx=0).stop 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/636456437 shutdown_connections 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b77120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b62aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/636456437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b68a50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.392+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/636456437 >> 172.21.15.31:0/636456437 conn(0x563969ad5200 msgr2=0x563969b76610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.393+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/636456437 shutdown_connections 2023-10-06T21:39:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.393+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/636456437 wait complete. 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.393+0000 7f5c51dc0b80 1 -- start start 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.393+0000 7f5c51dc0b80 1 Processor -- start 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b70920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969bcddd0 con 0x563969b60520 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969bcdf40 con 0x563969b6b3d0 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563969bce100 con 0x563969b5fc00 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c51dc0b80 10 client.0 _ll_register_callbacks cb 0x7f5c28099970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51518/0 (socket says 172.21.15.31:51518) 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c2ffff700 1 -- 172.21.15.31:0/2930158240 learned_addr learned my addr 172.21.15.31:0/2930158240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.394+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c3c99c700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b70920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 -- 172.21.15.31:0/2930158240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 msgr2=0x563969b6fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b6fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 -- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 msgr2=0x563969b70920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b70920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 -- 172.21.15.31:0/2930158240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639698ded80 con 0x563969b60520 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b6fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c2ffff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000b460 tx=0x7f5c2003a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a28ae80810ee130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c177fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20035040 con 0x563969b60520 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c2001e040 con 0x563969b60520 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5c2000f620 con 0x563969b60520 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c2000f7e0 con 0x563969b60520 2023-10-06T21:39:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c51dc0b80 10 client.17619 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.395+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5639699051f0 con 0x563969b60520 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c2001e040 con 0x563969b60520 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5c20004b50 con 0x563969b60520 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 10 client.17619 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 20 client.17619 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969bcf2b0 0x563969bd16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 4 client.17619 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:39:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.396+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563969aacad0 con 0x563969bcf2b0 2023-10-06T21:39:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.397+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969bcf2b0 0x563969bd16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.397+0000 7f5c37fff700 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969bcf2b0 0x563969bd16a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f25c34fd3f7723 server_cookie=b18962599f0cfdab in_seq=0 out_seq=0 2023-10-06T21:39:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.397+0000 7f5c177fe700 10 client.17619 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.446+0000 7f5c177fe700 1 -- 172.21.15.31:0/2930158240 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563969aacad0 con 0x563969bcf2b0 2023-10-06T21:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.446+0000 7f5c177fe700 10 client.17619 handle_command_reply: tid=0 2023-10-06T21:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.446+0000 7f5c51dc0b80 1 client.17619 shutdown 2023-10-06T21:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 msgr2=0x563969b703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000b460 tx=0x7f5c2003a5c0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 shutdown_connections 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563969b6b3d0 0x563969b70920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563969b5fc00 0x563969b6fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563969bcf2b0 0x563969bd16a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 --2- 172.21.15.31:0/2930158240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563969b60520 0x563969b703e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 >> 172.21.15.31:0/2930158240 conn(0x563969ad5200 msgr2=0x563969b77120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c177fe700 0 client.17619 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.447+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 shutdown_connections 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.449+0000 7f5c51dc0b80 1 -- 172.21.15.31:0/2930158240 wait complete. 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.451+0000 7f5c51dc0b80 20 client.17619 trim_cache size 0 max 16384 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.453+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 msgr2=0x7f5c240a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.453+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 0x7f5c240a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c3819a930 tx=0x7f5c2000b260 comp rx=0 tx=0).stop 2023-10-06T21:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.453+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 msgr2=0x7f5c38199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.453+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800d980 tx=0x7f5c2800dc90 comp rx=0 tx=0).stop 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 shutdown_connections 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3810d6a0 0x7f5c3819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c240a1670 0x7f5c240a3b30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c380732e0 0x7f5c38199110 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 --2- 172.21.15.31:0/2657085436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38073c00 0x7f5c38199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 >> 172.21.15.31:0/2657085436 conn(0x7f5c380fc930 msgr2=0x7f5c3810f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.454+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 shutdown_connections 2023-10-06T21:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:21.455+0000 7f5c3e3ff700 1 -- 172.21.15.31:0/2657085436 wait complete. 2023-10-06T21:39:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:21 smithi142 ceph-mon[102951]: pgmap v583: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:21 smithi031 ceph-mon[98539]: pgmap v583: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:21.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37211 6.35 MB/sec execute 190 sec latency 448.516 ms 2023-10-06T21:39:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:21 smithi146 ceph-mon[102809]: pgmap v583: 97 pgs: 97 active+clean; 713 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2023-10-06T21:39:22.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37368 6.40 MB/sec execute 191 sec latency 66.203 ms 2023-10-06T21:39:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:23 smithi142 ceph-mon[102951]: pgmap v584: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 216 op/s 2023-10-06T21:39:23.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:23 smithi031 ceph-mon[98539]: pgmap v584: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 216 op/s 2023-10-06T21:39:23.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37534 6.40 MB/sec execute 192 sec latency 60.971 ms 2023-10-06T21:39:23.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:23 smithi146 ceph-mon[102809]: pgmap v584: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 216 op/s 2023-10-06T21:39:24.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37684 6.43 MB/sec execute 193 sec latency 72.238 ms 2023-10-06T21:39:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:25 smithi142 ceph-mon[102951]: pgmap v585: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 194 op/s 2023-10-06T21:39:25.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:25 smithi031 ceph-mon[98539]: pgmap v585: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 194 op/s 2023-10-06T21:39:25.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37781 6.42 MB/sec execute 194 sec latency 61.326 ms 2023-10-06T21:39:25.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:25 smithi146 ceph-mon[102809]: pgmap v585: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 194 op/s 2023-10-06T21:39:26.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 37881 6.40 MB/sec execute 195 sec latency 64.675 ms 2023-10-06T21:39:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:27 smithi142 ceph-mon[102951]: pgmap v586: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2023-10-06T21:39:27.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:27 smithi031 ceph-mon[98539]: pgmap v586: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2023-10-06T21:39:27.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38011 6.43 MB/sec execute 196 sec latency 69.230 ms 2023-10-06T21:39:27.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:27 smithi146 ceph-mon[102809]: pgmap v586: 97 pgs: 97 active+clean; 722 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2023-10-06T21:39:28.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38105 6.42 MB/sec execute 197 sec latency 421.893 ms 2023-10-06T21:39:29.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38243 6.44 MB/sec execute 198 sec latency 72.251 ms 2023-10-06T21:39:30.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:30 smithi142 ceph-mon[102951]: pgmap v587: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:39:30.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:30 smithi031 ceph-mon[98539]: pgmap v587: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:39:30.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:30 smithi146 ceph-mon[102809]: pgmap v587: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-06T21:39:30.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38346 6.43 MB/sec execute 199 sec latency 67.586 ms 2023-10-06T21:39:31.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:31 smithi142 ceph-mon[102951]: pgmap v588: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2023-10-06T21:39:31.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:31 smithi031 ceph-mon[98539]: pgmap v588: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2023-10-06T21:39:31.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38473 6.43 MB/sec execute 200 sec latency 61.556 ms 2023-10-06T21:39:31.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:31 smithi146 ceph-mon[102809]: pgmap v588: 97 pgs: 97 active+clean; 735 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2023-10-06T21:39:32.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38566 6.42 MB/sec execute 201 sec latency 65.998 ms 2023-10-06T21:39:33.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:33 smithi142 ceph-mon[102951]: pgmap v589: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2023-10-06T21:39:33.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:33 smithi031 ceph-mon[98539]: pgmap v589: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2023-10-06T21:39:33.771 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38671 6.42 MB/sec execute 202 sec latency 65.181 ms 2023-10-06T21:39:33.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:33 smithi146 ceph-mon[102809]: pgmap v589: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2023-10-06T21:39:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 -- 172.21.15.31:0/3691801826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de01032f0 msgr2=0x7f6de01036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 --2- 172.21.15.31:0/3691801826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de01032f0 0x7f6de01036d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009990 tx=0x7f6dd4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 -- 172.21.15.31:0/3691801826 shutdown_connections 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 --2- 172.21.15.31:0/3691801826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de0108410 0x7f6de010ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 --2- 172.21.15.31:0/3691801826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de01032f0 0x7f6de01036d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 --2- 172.21.15.31:0/3691801826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0103ca0 0x7f6de0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 -- 172.21.15.31:0/3691801826 >> 172.21.15.31:0/3691801826 conn(0x7f6de00feb30 msgr2=0x7f6de0100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.682+0000 7f6de6fff700 1 -- 172.21.15.31:0/3691801826 shutdown_connections 2023-10-06T21:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.683+0000 7f6de6fff700 1 -- 172.21.15.31:0/3691801826 wait complete. 2023-10-06T21:39:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.684+0000 7f6de6fff700 1 Processor -- start 2023-10-06T21:39:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.684+0000 7f6de6fff700 1 -- start start 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 0x7f6de010fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0108410 0x7f6de010cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01a9dd0 con 0x7f6de01032f0 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01a9f40 con 0x7f6de0108410 2023-10-06T21:39:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.685+0000 7f6de6fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01aa100 con 0x7f6de010d000 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.686+0000 7f6de559c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.686+0000 7f6de559c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56210/0 (socket says 172.21.15.31:56210) 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.686+0000 7f6de559c700 1 -- 172.21.15.31:0/2036085158 learned_addr learned my addr 172.21.15.31:0/2036085158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.686+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 0x7f6de010fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.686+0000 7f6ddffff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0108410 0x7f6de010cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 -- 172.21.15.31:0/2036085158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0108410 msgr2=0x7f6de010cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0108410 0x7f6de010cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 -- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 msgr2=0x7f6de010fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 0x7f6de010fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 -- 172.21.15.31:0/2036085158 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40095f0 con 0x7f6de010d000 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 0x7f6de010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de559c700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007ee0 tx=0x7f6dd800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6dddffb700 1 -- 172.21.15.31:0/2036085158 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8017070 con 0x7f6de010d000 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6dddffb700 1 -- 172.21.15.31:0/2036085158 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dd800f040 con 0x7f6de010d000 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6dddffb700 1 -- 172.21.15.31:0/2036085158 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8005710 con 0x7f6de010d000 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.687+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de01aa3e0 con 0x7f6de010d000 2023-10-06T21:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.688+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de01aa8a0 con 0x7f6de010d000 2023-10-06T21:39:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.692+0000 7f6dddffb700 1 -- 172.21.15.31:0/2036085158 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6dd8005870 con 0x7f6de010d000 2023-10-06T21:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.694+0000 7f6dddffb700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 0x7f6dd00a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.694+0000 7f6dddffb700 1 -- 172.21.15.31:0/2036085158 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6dd80cc8b0 con 0x7f6de010d000 2023-10-06T21:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.694+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 0x7f6dd00a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 Processor -- start 2023-10-06T21:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 -- start start 2023-10-06T21:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2d877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2ce3400 con 0x5571f2d7a520 2023-10-06T21:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2cd4bc0 con 0x5571f2d853d0 2023-10-06T21:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.695+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2cd24e0 con 0x5571f2d79c00 2023-10-06T21:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 0x7f6dd00a3b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd400b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56222/0 (socket says 172.21.15.31:56222) 2023-10-06T21:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 -- 172.21.15.31:0/3133094508 learned_addr learned my addr 172.21.15.31:0/3133094508 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6ddffff700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de559c700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2d877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 -- 172.21.15.31:0/3133094508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 msgr2=0x5571f2d877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2d877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 -- 172.21.15.31:0/3133094508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 msgr2=0x5571f2d84e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d84e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.696+0000 7f6de4d9b700 1 -- 172.21.15.31:0/3133094508 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571f2af8d80 con 0x5571f2d79c00 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.697+0000 7f6ddffff700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.697+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4015000 tx=0x7f6dd401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e5eba62a0cb507 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.697+0000 7f6dcaffd700 1 -- 172.21.15.31:0/3133094508 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd402b070 con 0x5571f2d79c00 2023-10-06T21:39:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.697+0000 7f6dcaffd700 1 -- 172.21.15.31:0/3133094508 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dd401e040 con 0x5571f2d79c00 2023-10-06T21:39:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.697+0000 7f6dcaffd700 1 -- 172.21.15.31:0/3133094508 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004e50 con 0x5571f2d79c00 2023-10-06T21:39:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3133094508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 msgr2=0x5571f2d79fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4015000 tx=0x7f6dd401f000 comp rx=0 tx=0).stop 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3133094508 shutdown_connections 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2d877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d79c00 0x5571f2d79fe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3133094508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.698+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3133094508 >> 172.21.15.31:0/3133094508 conn(0x5571f2cef200 msgr2=0x5571f2d76080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.699+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3133094508 shutdown_connections 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.699+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3133094508 wait complete. 2023-10-06T21:39:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.700+0000 7f6dfa924b80 1 -- start start 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.700+0000 7f6dfa924b80 1 Processor -- start 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d84e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d853d0 0x5571f2d82840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2ccddd0 con 0x5571f2d7a520 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2c6a6b0 con 0x5571f2d79c00 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2c6a350 con 0x5571f2d853d0 2023-10-06T21:39:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6dfa924b80 10 client.0 _ll_register_callbacks cb 0x5571f2ac77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37280/0 (socket says 172.21.15.31:37280) 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.701+0000 7f6ddffff700 1 -- 172.21.15.31:0/1302136597 learned_addr learned my addr 172.21.15.31:0/1302136597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d84e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de559c700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d853d0 0x5571f2d82840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 -- 172.21.15.31:0/1302136597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d853d0 msgr2=0x5571f2d82840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d853d0 0x5571f2d82840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 -- 172.21.15.31:0/1302136597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 msgr2=0x5571f2d82100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6de4d9b700 1 -- 172.21.15.31:0/1302136597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de0110800 con 0x5571f2d79c00 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.702+0000 7f6ddffff700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d84e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4005950 tx=0x7f6dd4004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f6a5f03e35fdb37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6dbffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd402b030 con 0x5571f2d79c00 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571f2d6dd50 con 0x5571f2d79c00 2023-10-06T21:39:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6dd402c070 con 0x5571f2d79c00 2023-10-06T21:39:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.703+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd4029070 con 0x5571f2d79c00 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 10 client.27359 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5571f2ae1c00 con 0x5571f2d79c00 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5571f2d6dd50 con 0x5571f2d79c00 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6dd400f460 con 0x5571f2d79c00 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 10 client.27359 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 10 client.27359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 20 client.27359 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2d94a40 0x5571f2d96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 4 client.27359 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.704+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571f2cc6db0 con 0x5571f2d94a40 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.705+0000 7f6ddffff700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2d94a40 0x5571f2d96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.706+0000 7f6ddffff700 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2d94a40 0x5571f2d96e60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6d7e059f26a045 server_cookie=f2877f297140c968 in_seq=0 out_seq=0 2023-10-06T21:39:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.706+0000 7f6dbffff700 10 client.27359 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dbffff700 1 -- 172.21.15.31:0/1302136597 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5571f2cc6db0 con 0x5571f2d94a40 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dbffff700 10 client.27359 handle_command_reply: tid=0 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 client.27359 shutdown 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 msgr2=0x5571f2d84e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d84e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4005950 tx=0x7f6dd4004da0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 shutdown_connections 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d853d0 0x5571f2d82840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d84e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2d94a40 0x5571f2d96e60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/1302136597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d7a520 0x5571f2d82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 >> 172.21.15.31:0/1302136597 conn(0x5571f2cef200 msgr2=0x5571f2d75c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.707+0000 7f6dbffff700 0 client.27359 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.708+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 shutdown_connections 2023-10-06T21:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.708+0000 7f6dfa924b80 1 -- 172.21.15.31:0/1302136597 wait complete. 2023-10-06T21:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.708+0000 7f6dfa924b80 20 client.27359 trim_cache size 0 max 16384 2023-10-06T21:39:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.734+0000 7f6dfa924b80 1 Processor -- start 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 -- start start 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2d818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 0x5571f2d90640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2b4e8c0 con 0x5571f2d853d0 2023-10-06T21:39:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2d80660 con 0x5571f2d79c00 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2d7df50 con 0x5571f2d7a520 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.735+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2d818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2d818c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56236/0 (socket says 172.21.15.31:56236) 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6ddffff700 1 -- 172.21.15.31:0/3416644412 learned_addr learned my addr 172.21.15.31:0/3416644412 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 0x5571f2d90640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 -- 172.21.15.31:0/3416644412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 msgr2=0x5571f2d818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2d818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 -- 172.21.15.31:0/3416644412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 msgr2=0x5571f2d877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.736+0000 7f6de559c700 1 -- 172.21.15.31:0/3416644412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571f2b1f1f0 con 0x5571f2d853d0 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.737+0000 7f6de559c700 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 0x5571f2d90640 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80c9f30 tx=0x7f6dd8084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897292c55c3bd61f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.737+0000 7f6dcb7fe700 1 -- 172.21.15.31:0/3416644412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800dc60 con 0x5571f2d853d0 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.737+0000 7f6dcb7fe700 1 -- 172.21.15.31:0/3416644412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dd8099ec0 con 0x5571f2d853d0 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.737+0000 7f6dcb7fe700 1 -- 172.21.15.31:0/3416644412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8095ed0 con 0x5571f2d853d0 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3416644412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 msgr2=0x5571f2d90640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 0x5571f2d90640 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80c9f30 tx=0x7f6dd8084040 comp rx=0 tx=0).stop 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3416644412 shutdown_connections 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d853d0 0x5571f2d90640 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d79c00 0x5571f2d877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3416644412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2d818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3416644412 >> 172.21.15.31:0/3416644412 conn(0x5571f2cef200 msgr2=0x5571f2d747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.738+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3416644412 shutdown_connections 2023-10-06T21:39:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.739+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3416644412 wait complete. 2023-10-06T21:39:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.739+0000 7f6dfa924b80 1 -- start start 2023-10-06T21:39:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.739+0000 7f6dfa924b80 1 Processor -- start 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 0x5571f2d8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2de3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2de3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2de4580 con 0x5571f2d79c00 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2d89350 con 0x5571f2d853d0 2023-10-06T21:39:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f2d89510 con 0x5571f2d7a520 2023-10-06T21:39:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6dfa924b80 10 client.0 _ll_register_callbacks cb 0x5571f2ac77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2de3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2de3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56238/0 (socket says 172.21.15.31:56238) 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.740+0000 7f6ddffff700 1 -- 172.21.15.31:0/3456282061 learned_addr learned my addr 172.21.15.31:0/3456282061 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2de3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 0x5571f2d8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 -- 172.21.15.31:0/3456282061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 msgr2=0x5571f2de3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2de3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 -- 172.21.15.31:0/3456282061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 msgr2=0x5571f2d8bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 0x5571f2d8bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de559c700 1 -- 172.21.15.31:0/3456282061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de01aa8a0 con 0x5571f2d853d0 2023-10-06T21:39:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.741+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 0x5571f2d8bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:39:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6de559c700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2de3e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8099980 tx=0x7f6dd80c9f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f2a0be946659f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6dbffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80997c0 con 0x5571f2d853d0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6dd80a1b50 con 0x5571f2d853d0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dd401e040 con 0x5571f2d853d0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.742+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd809d910 con 0x5571f2d853d0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 10 client.27365 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5571f2af7250 con 0x5571f2d853d0 2023-10-06T21:39:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dd401e040 con 0x5571f2d853d0 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6dd8098d40 con 0x5571f2d853d0 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 10 client.27365 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 20 client.27365 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2df9d70 0x5571f2dfc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.743+0000 7f6dfa924b80 4 client.27365 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.744+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571f2cc6ad0 con 0x5571f2df9d70 2023-10-06T21:39:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.744+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2df9d70 0x5571f2dfc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.745+0000 7f6de4d9b700 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2df9d70 0x5571f2dfc190 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc78e269f6d112de server_cookie=a029076fcaace634 in_seq=0 out_seq=0 2023-10-06T21:39:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.745+0000 7f6dbffff700 10 client.27365 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:34.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38784 6.41 MB/sec execute 203 sec latency 77.850 ms 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.792+0000 7f6dbffff700 1 -- 172.21.15.31:0/3456282061 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571f2cc6ad0 con 0x5571f2df9d70 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.792+0000 7f6dbffff700 10 client.27365 handle_command_reply: tid=0 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.792+0000 7f6dfa924b80 1 client.27365 shutdown 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 msgr2=0x5571f2de3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2de3e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8099980 tx=0x7f6dd80c9f00 comp rx=0 tx=0).stop 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 shutdown_connections 2023-10-06T21:39:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571f2d853d0 0x5571f2de3e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5571f2df9d70 0x5571f2dfc190 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f2d79c00 0x5571f2d8bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dbffff700 0 client.27365 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 --2- 172.21.15.31:0/3456282061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571f2d7a520 0x5571f2de3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 >> 172.21.15.31:0/3456282061 conn(0x5571f2cef200 msgr2=0x5571f2d90640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 shutdown_connections 2023-10-06T21:39:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 1 -- 172.21.15.31:0/3456282061 wait complete. 2023-10-06T21:39:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.793+0000 7f6dfa924b80 20 client.27365 trim_cache size 0 max 16384 2023-10-06T21:39:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:39:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:39:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:39:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 msgr2=0x7f6dd00a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 0x7f6dd00a3b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd400b260 comp rx=0 tx=0).stop 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 msgr2=0x7f6de010d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007ee0 tx=0x7f6dd800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 shutdown_connections 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de010d000 0x7f6de010d4b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6dd00a16a0 0x7f6dd00a3b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de01032f0 0x7f6de010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 --2- 172.21.15.31:0/2036085158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6de0108410 0x7f6de010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.794+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 >> 172.21.15.31:0/2036085158 conn(0x7f6de00feb30 msgr2=0x7f6de00ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.795+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 shutdown_connections 2023-10-06T21:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:34.795+0000 7f6de6fff700 1 -- 172.21.15.31:0/2036085158 wait complete. 2023-10-06T21:39:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:35 smithi142 ceph-mon[102951]: pgmap v590: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:35.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:35 smithi031 ceph-mon[98539]: pgmap v590: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:35.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 38934 6.41 MB/sec execute 204 sec latency 84.518 ms 2023-10-06T21:39:35.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:35 smithi146 ceph-mon[102809]: pgmap v590: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:36.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39050 6.41 MB/sec execute 205 sec latency 73.544 ms 2023-10-06T21:39:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:37 smithi142 ceph-mon[102951]: pgmap v591: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:37.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:37 smithi031 ceph-mon[98539]: pgmap v591: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:37.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39166 6.40 MB/sec execute 206 sec latency 72.746 ms 2023-10-06T21:39:37.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:37 smithi146 ceph-mon[102809]: pgmap v591: 97 pgs: 97 active+clean; 743 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 161 op/s 2023-10-06T21:39:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:38 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:39:38.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:38 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:39:38.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39268 6.39 MB/sec execute 207 sec latency 86.470 ms 2023-10-06T21:39:38.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:38 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:39:39.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:39 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:39:39.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:39 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:39:39.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:39 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:39:39.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:39 smithi142 ceph-mon[102951]: pgmap v592: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T21:39:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:39 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:39:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:39 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:39:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:39 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:39:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:39 smithi031 ceph-mon[98539]: pgmap v592: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T21:39:39.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39392 6.41 MB/sec execute 208 sec latency 75.202 ms 2023-10-06T21:39:39.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:39 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:39:39.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:39 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:39:39.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:39 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:39:39.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:39 smithi146 ceph-mon[102809]: pgmap v592: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T21:39:40.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39507 6.41 MB/sec execute 209 sec latency 70.375 ms 2023-10-06T21:39:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:41 smithi142 ceph-mon[102951]: pgmap v593: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2023-10-06T21:39:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:41 smithi031 ceph-mon[98539]: pgmap v593: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2023-10-06T21:39:41.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39628 6.41 MB/sec execute 210 sec latency 75.229 ms 2023-10-06T21:39:41.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:41 smithi146 ceph-mon[102809]: pgmap v593: 97 pgs: 97 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2023-10-06T21:39:42.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39743 6.40 MB/sec execute 211 sec latency 77.635 ms 2023-10-06T21:39:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:43 smithi142 ceph-mon[102951]: pgmap v594: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 290 op/s 2023-10-06T21:39:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:43 smithi031 ceph-mon[98539]: pgmap v594: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 290 op/s 2023-10-06T21:39:43.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39865 6.39 MB/sec execute 212 sec latency 77.852 ms 2023-10-06T21:39:43.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:43 smithi146 ceph-mon[102809]: pgmap v594: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 290 op/s 2023-10-06T21:39:44.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 39977 6.38 MB/sec execute 213 sec latency 70.445 ms 2023-10-06T21:39:45.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:45 smithi142 ceph-mon[102951]: pgmap v595: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:45 smithi031 ceph-mon[98539]: pgmap v595: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:45.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40105 6.37 MB/sec execute 214 sec latency 67.146 ms 2023-10-06T21:39:45.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:45 smithi146 ceph-mon[102809]: pgmap v595: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:46.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40222 6.36 MB/sec execute 215 sec latency 68.416 ms 2023-10-06T21:39:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:47 smithi142 ceph-mon[102951]: pgmap v596: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:47 smithi031 ceph-mon[98539]: pgmap v596: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:47.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40368 6.37 MB/sec execute 216 sec latency 70.447 ms 2023-10-06T21:39:47.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:47 smithi146 ceph-mon[102809]: pgmap v596: 97 pgs: 97 active+clean; 749 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.001+0000 7f71ba7ff700 1 -- 172.21.15.31:0/1077543175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41061f0 msgr2=0x7f71b410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.001+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/1077543175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41061f0 0x7f71b410dd70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b0a0 tx=0x7f71ac00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 -- 172.21.15.31:0/1077543175 shutdown_connections 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/1077543175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41061f0 0x7f71b410dd70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/1077543175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41010d0 0x7f71b41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/1077543175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b4101a80 0x7f71b4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 -- 172.21.15.31:0/1077543175 >> 172.21.15.31:0/1077543175 conn(0x7f71b40fc930 msgr2=0x7f71b40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 -- 172.21.15.31:0/1077543175 shutdown_connections 2023-10-06T21:39:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.002+0000 7f71ba7ff700 1 -- 172.21.15.31:0/1077543175 wait complete. 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 Processor -- start 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 -- start start 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 0x7f71b40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 0x7f71b41a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b41a5c60 con 0x7f71b41010d0 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b41a5da0 con 0x7f71b41061f0 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.003+0000 7f71ba7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b41a5f10 con 0x7f71b41a3330 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56432/0 (socket says 172.21.15.31:56432) 2023-10-06T21:39:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 -- 172.21.15.31:0/3319105727 learned_addr learned my addr 172.21.15.31:0/3319105727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 0x7f71b41a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b37fe700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 0x7f71b40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 -- 172.21.15.31:0/3319105727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 msgr2=0x7f71b41a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 0x7f71b41a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 -- 172.21.15.31:0/3319105727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 msgr2=0x7f71b40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 0x7f71b40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 -- 172.21.15.31:0/3319105727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac009d00 con 0x7f71b41010d0 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 0x7f71b41a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b37fe700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 0x7f71b40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.004+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b530 tx=0x7f71a400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.005+0000 7f71b17fa700 1 -- 172.21.15.31:0/3319105727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011830 con 0x7f71b41010d0 2023-10-06T21:39:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.005+0000 7f71b17fa700 1 -- 172.21.15.31:0/3319105727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a4011e20 con 0x7f71b41010d0 2023-10-06T21:39:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.005+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b41a61f0 con 0x7f71b41010d0 2023-10-06T21:39:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.005+0000 7f71b17fa700 1 -- 172.21.15.31:0/3319105727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4010dd0 con 0x7f71b41010d0 2023-10-06T21:39:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.005+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b41a6700 con 0x7f71b41010d0 2023-10-06T21:39:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.008+0000 7f71b17fa700 1 -- 172.21.15.31:0/3319105727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f71a4011990 con 0x7f71b41010d0 2023-10-06T21:39:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.009+0000 7f71ce105b80 1 Processor -- start 2023-10-06T21:39:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 -- start start 2023-10-06T21:39:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b17fa700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 0x7f719c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b17fa700 1 -- 172.21.15.31:0/3319105727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f71a40ccc50 con 0x7f71b41010d0 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 0x7f719c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98f520 0x55684f999e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f99c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f8f8400 con 0x55684f99a3d0 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f8e9bc0 con 0x55684f98f520 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71ce105b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f8e74e0 con 0x55684f98ec00 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f99c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55378/0 (socket says 172.21.15.31:55378) 2023-10-06T21:39:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.010+0000 7f71b3fff700 1 -- 172.21.15.31:0/1925468523 learned_addr learned my addr 172.21.15.31:0/1925468523 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b37fe700 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98f520 0x55684f999e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 -- 172.21.15.31:0/1925468523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98f520 msgr2=0x55684f999e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98f520 0x55684f999e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 -- 172.21.15.31:0/1925468523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 msgr2=0x55684f99c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f99c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 -- 172.21.15.31:0/1925468523 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684f70dd80 con 0x55684f98ec00 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f99c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 0x7f719c0a3ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009fd0 tx=0x7f71ac004cd0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f71b3fff700 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a40966e0 tx=0x7f71a40960b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361596102e088d0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f719a7fc700 1 -- 172.21.15.31:0/1925468523 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40995d0 con 0x55684f98ec00 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f719a7fc700 1 -- 172.21.15.31:0/1925468523 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a4010510 con 0x55684f98ec00 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.011+0000 7f719a7fc700 1 -- 172.21.15.31:0/1925468523 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a409a0d0 con 0x55684f98ec00 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 -- 172.21.15.31:0/1925468523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 msgr2=0x55684f98efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a40966e0 tx=0x7f71a40960b0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 -- 172.21.15.31:0/1925468523 shutdown_connections 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f99c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98ec00 0x55684f98efe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 --2- 172.21.15.31:0/1925468523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98f520 0x55684f999e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 -- 172.21.15.31:0/1925468523 >> 172.21.15.31:0/1925468523 conn(0x55684f904200 msgr2=0x55684f98b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 -- 172.21.15.31:0/1925468523 shutdown_connections 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.012+0000 7f71ce105b80 1 -- 172.21.15.31:0/1925468523 wait complete. 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.013+0000 7f71ce105b80 1 -- start start 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.013+0000 7f71ce105b80 1 Processor -- start 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.013+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f997130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.013+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f997670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71ce105b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f8e2dd0 con 0x55684f99a3d0 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71ce105b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f87f6b0 con 0x55684f98ec00 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71ce105b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f87f350 con 0x55684f98f520 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71ce105b80 10 client.0 _ll_register_callbacks cb 0x55684f6dc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56446/0 (socket says 172.21.15.31:56446) 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 -- 172.21.15.31:0/256270479 learned_addr learned my addr 172.21.15.31:0/256270479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b37fe700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f997670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b3fff700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f997130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 -- 172.21.15.31:0/256270479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 msgr2=0x55684f997670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f997670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 -- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 msgr2=0x55684f997130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f997130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b8d9c700 1 -- 172.21.15.31:0/256270479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b410e5e0 con 0x55684f99a3d0 2023-10-06T21:39:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.014+0000 7f71b3fff700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f997130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f71ac020700 tx=0x7f71ac00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af392b57e7b7505d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71937fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00e070 con 0x55684f99a3d0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55684f982d50 con 0x55684f99a3d0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f71ac02d660 con 0x55684f99a3d0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71ac02d820 con 0x55684f99a3d0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71ce105b80 10 client.17646 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f71b41a6700 con 0x55684f99a3d0 2023-10-06T21:39:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.015+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55684f982d50 con 0x55684f99a3d0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71ac01e7d0 con 0x55684f99a3d0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 10 client.17646 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 20 client.17646 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9a9fe0 0x55684f9ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 4 client.17646 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55684f8dbdb0 con 0x55684f9a9fe0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.016+0000 7f71b3fff700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9a9fe0 0x55684f9ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.017+0000 7f71b3fff700 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9a9fe0 0x55684f9ac400 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98804343378163fc server_cookie=b05e036916ee7973 in_seq=0 out_seq=0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.017+0000 7f71937fe700 10 client.17646 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.017+0000 7f71937fe700 1 -- 172.21.15.31:0/256270479 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55684f8dbdb0 con 0x55684f9a9fe0 2023-10-06T21:39:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.017+0000 7f71937fe700 10 client.17646 handle_command_reply: tid=0 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.017+0000 7f71ce105b80 1 client.17646 shutdown 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 msgr2=0x55684f997bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f71ac020700 tx=0x7f71ac00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 shutdown_connections 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9a9fe0 0x55684f9ac400 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f997bb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f997130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71937fe700 0 client.17646 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 --2- 172.21.15.31:0/256270479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f997670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 >> 172.21.15.31:0/256270479 conn(0x55684f904200 msgr2=0x55684f98ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 shutdown_connections 2023-10-06T21:39:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 1 -- 172.21.15.31:0/256270479 wait complete. 2023-10-06T21:39:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.018+0000 7f71ce105b80 20 client.17646 trim_cache size 0 max 16384 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.033+0000 7f71ce105b80 1 Processor -- start 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.033+0000 7f71ce105b80 1 -- start start 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f991aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f7638c0 con 0x55684f99a3d0 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f987860 con 0x55684f98ec00 2023-10-06T21:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71ce105b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f9a1460 con 0x55684f98f520 2023-10-06T21:39:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56454/0 (socket says 172.21.15.31:56454) 2023-10-06T21:39:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 -- 172.21.15.31:0/66574811 learned_addr learned my addr 172.21.15.31:0/66574811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b37fe700 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b3fff700 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f991aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 -- 172.21.15.31:0/66574811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 msgr2=0x55684f9a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 -- 172.21.15.31:0/66574811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 msgr2=0x55684f991aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.034+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f991aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.035+0000 7f71b8d9c700 1 -- 172.21.15.31:0/66574811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684f6f6c00 con 0x55684f99a3d0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.035+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01eb30 tx=0x7f71ac007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8797281b0b39bf6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.035+0000 7f719affd700 1 -- 172.21.15.31:0/66574811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00e040 con 0x55684f99a3d0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.035+0000 7f719affd700 1 -- 172.21.15.31:0/66574811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ac01e8c0 con 0x55684f99a3d0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.035+0000 7f719affd700 1 -- 172.21.15.31:0/66574811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac02d9e0 con 0x55684f99a3d0 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- 172.21.15.31:0/66574811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 msgr2=0x55684f9976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01eb30 tx=0x7f71ac007970 comp rx=0 tx=0).stop 2023-10-06T21:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- 172.21.15.31:0/66574811 shutdown_connections 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9976a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f991aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 --2- 172.21.15.31:0/66574811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- 172.21.15.31:0/66574811 >> 172.21.15.31:0/66574811 conn(0x55684f904200 msgr2=0x55684f992430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- 172.21.15.31:0/66574811 shutdown_connections 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- 172.21.15.31:0/66574811 wait complete. 2023-10-06T21:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.036+0000 7f71ce105b80 1 -- start start 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 Processor -- start 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f9f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f9f2ae0 con 0x55684f99a3d0 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f9f2c50 con 0x55684f98ec00 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f9fd3e0 con 0x55684f98f520 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71ce105b80 10 client.0 _ll_register_callbacks cb 0x55684f6dc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f9f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71b8d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56466/0 (socket says 172.21.15.31:56466) 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71b37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.037+0000 7f71b8d9c700 1 -- 172.21.15.31:0/3683536930 learned_addr learned my addr 172.21.15.31:0/3683536930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 -- 172.21.15.31:0/3683536930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 msgr2=0x55684f9f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 -- 172.21.15.31:0/3683536930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 msgr2=0x55684f9fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 -- 172.21.15.31:0/3683536930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b41a6700 con 0x55684f98ec00 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b8d9c700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b37fe700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71b3fff700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f9f1da0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71a4099c00 tx=0x7f71a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e88d9741adca5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:39:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71937fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4099360 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71a4010510 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.038+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f71a40b2040 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a40a2000 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 10 client.27377 fetch_fsmap learned FSMap version 49 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55684f7341f0 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a4010510 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71a4099f60 con 0x55684f98ec00 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 10 client.27377 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 10 client.27377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 20 client.27377 populate_metadata read hostname 'smithi031' 2023-10-06T21:39:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9fe940 0x55684fa00d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 4 client.27377 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55684f8dbad0 con 0x55684f9fe940 2023-10-06T21:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.039+0000 7f71b37fe700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9fe940 0x55684fa00d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.040+0000 7f71b37fe700 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9fe940 0x55684fa00d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e121465d1aa3a6f2 server_cookie=422cba8784628c25 in_seq=0 out_seq=0 2023-10-06T21:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.040+0000 7f71937fe700 10 client.27377 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.290+0000 7f71937fe700 1 -- 172.21.15.31:0/3683536930 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55684f8dbad0 con 0x55684f9fe940 2023-10-06T21:39:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.290+0000 7f71937fe700 10 client.27377 handle_command_reply: tid=0 2023-10-06T21:39:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.290+0000 7f71ce105b80 1 client.27377 shutdown 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 msgr2=0x55684f9f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f9f1da0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71a4099c00 tx=0x7f71a4004750 comp rx=0 tx=0).stop 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 shutdown_connections 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55684f9fe940 0x55684fa00d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55684f99a3d0 0x55684f9fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55684f98ec00 0x55684f9f1da0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 --2- 172.21.15.31:0/3683536930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55684f98f520 0x55684f9f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71937fe700 0 client.27377 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.291+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 >> 172.21.15.31:0/3683536930 conn(0x55684f904200 msgr2=0x55684f9976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.292+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 shutdown_connections 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.292+0000 7f71ce105b80 1 -- 172.21.15.31:0/3683536930 wait complete. 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.292+0000 7f71ce105b80 20 client.27377 trim_cache size 0 max 16384 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:39:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:39:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 msgr2=0x7f719c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 0x7f719c0a3ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009fd0 tx=0x7f71ac004cd0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 msgr2=0x7f71b4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:39:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b530 tx=0x7f71a400b8f0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 shutdown_connections 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71b41a3330 0x7f71b41a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f719c0a15f0 0x7f719c0a3ab0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b41010d0 0x7f71b4073390 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 --2- 172.21.15.31:0/3319105727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b41061f0 0x7f71b40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 >> 172.21.15.31:0/3319105727 conn(0x7f71b40fc930 msgr2=0x7f71b40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 shutdown_connections 2023-10-06T21:39:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:39:48.293+0000 7f71ba7ff700 1 -- 172.21.15.31:0/3319105727 wait complete. 2023-10-06T21:39:48.772 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40441 6.35 MB/sec execute 217 sec latency 453.686 ms 2023-10-06T21:39:49.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40560 6.35 MB/sec execute 218 sec latency 116.723 ms 2023-10-06T21:39:50.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:50 smithi031 ceph-mon[98539]: pgmap v597: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-10-06T21:39:50.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:50 smithi146 ceph-mon[102809]: pgmap v597: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-10-06T21:39:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:50 smithi142 ceph-mon[102951]: pgmap v597: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-10-06T21:39:50.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40656 6.34 MB/sec execute 219 sec latency 262.324 ms 2023-10-06T21:39:51.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:51 smithi142 ceph-mon[102951]: pgmap v598: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 173 op/s 2023-10-06T21:39:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:51 smithi031 ceph-mon[98539]: pgmap v598: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 173 op/s 2023-10-06T21:39:51.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40790 6.38 MB/sec execute 220 sec latency 259.531 ms 2023-10-06T21:39:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:51 smithi146 ceph-mon[102809]: pgmap v598: 97 pgs: 97 active+clean; 757 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 173 op/s 2023-10-06T21:39:52.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 40942 6.39 MB/sec execute 221 sec latency 64.513 ms 2023-10-06T21:39:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:53 smithi142 ceph-mon[102951]: pgmap v599: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-10-06T21:39:55.654 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:53 smithi146 ceph-mon[102809]: pgmap v599: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-10-06T21:39:55.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:53 smithi031 ceph-mon[98539]: pgmap v599: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2023-10-06T21:39:55.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:55 smithi142 ceph-mon[102951]: pgmap v600: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:55.654 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41105 6.42 MB/sec execute 222 sec latency 67.259 ms 2023-10-06T21:39:55.655 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41230 6.44 MB/sec execute 223 sec latency 69.991 ms 2023-10-06T21:39:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:55 smithi031 ceph-mon[98539]: pgmap v600: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:56.112 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:55 smithi146 ceph-mon[102809]: pgmap v600: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:56.112 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41323 6.42 MB/sec execute 224 sec latency 65.100 ms 2023-10-06T21:39:56.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41374 6.41 MB/sec execute 225 sec latency 389.852 ms 2023-10-06T21:39:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:39:57 smithi142 ceph-mon[102951]: pgmap v601: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:58.127 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:39:57 smithi146 ceph-mon[102809]: pgmap v601: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:58.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:39:57 smithi031 ceph-mon[98539]: pgmap v601: 97 pgs: 97 active+clean; 764 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 180 op/s 2023-10-06T21:39:58.127 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41480 6.42 MB/sec execute 226 sec latency 451.315 ms 2023-10-06T21:39:58.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41600 6.42 MB/sec execute 227 sec latency 73.627 ms 2023-10-06T21:39:59.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41726 6.43 MB/sec execute 228 sec latency 64.956 ms 2023-10-06T21:40:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:39:59.999+0000 7f51d5f22700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:40:00.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:00 smithi031 ceph-mon[98539]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:40:00.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:00 smithi146 ceph-mon[102809]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:40:00.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:00 smithi142 ceph-mon[102951]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-10-06T21:40:00.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41839 6.41 MB/sec execute 229 sec latency 108.118 ms 2023-10-06T21:40:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:01 smithi031 ceph-mon[98539]: pgmap v602: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2023-10-06T21:40:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.525+0000 7f2fb6dff700 1 -- 172.21.15.31:0/917816989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb0068bf0 msgr2=0x7f2fb010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.525+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/917816989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb0068bf0 0x7f2fb010d5e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0099c0 tx=0x7f2f9c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 -- 172.21.15.31:0/917816989 shutdown_connections 2023-10-06T21:40:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/917816989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb010db20 0x7f2fb010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/917816989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb00682d0 0x7f2fb00686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/917816989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb0068bf0 0x7f2fb010d5e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 -- 172.21.15.31:0/917816989 >> 172.21.15.31:0/917816989 conn(0x7f2fb0076b70 msgr2=0x7f2fb0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 -- 172.21.15.31:0/917816989 shutdown_connections 2023-10-06T21:40:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.526+0000 7f2fb6dff700 1 -- 172.21.15.31:0/917816989 wait complete. 2023-10-06T21:40:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.527+0000 7f2fb6dff700 1 Processor -- start 2023-10-06T21:40:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.527+0000 7f2fb6dff700 1 -- start start 2023-10-06T21:40:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 0x7f2fb0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb010db20 0x7f2fb019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019e320 con 0x7f2fb0068bf0 2023-10-06T21:40:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019e490 con 0x7f2fb010db20 2023-10-06T21:40:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.528+0000 7f2fb6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019e650 con 0x7f2fb00682d0 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 0x7f2fb0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34064/0 (socket says 172.21.15.31:34064) 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 -- 172.21.15.31:0/2910571433 learned_addr learned my addr 172.21.15.31:0/2910571433 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2fb539c700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb010db20 0x7f2fb019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 -- 172.21.15.31:0/2910571433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 msgr2=0x7f2fb0199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 0x7f2fb0199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 -- 172.21.15.31:0/2910571433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb010db20 msgr2=0x7f2fb019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb010db20 0x7f2fb019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2faffff700 1 -- 172.21.15.31:0/2910571433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0095f0 con 0x7f2fb0068bf0 2023-10-06T21:40:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.529+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 0x7f2fb0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.530+0000 7f2faffff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005910 tx=0x7f2f9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.530+0000 7f2fadffb700 1 -- 172.21.15.31:0/2910571433 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c01d070 con 0x7f2fb0068bf0 2023-10-06T21:40:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.530+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb019e8d0 con 0x7f2fb0068bf0 2023-10-06T21:40:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.530+0000 7f2fadffb700 1 -- 172.21.15.31:0/2910571433 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f9c005050 con 0x7f2fb0068bf0 2023-10-06T21:40:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.530+0000 7f2fadffb700 1 -- 172.21.15.31:0/2910571433 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004a00 con 0x7f2fb0068bf0 2023-10-06T21:40:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.531+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb01a6170 con 0x7f2fb0068bf0 2023-10-06T21:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.537+0000 7f2fadffb700 1 -- 172.21.15.31:0/2910571433 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2f9c0051c0 con 0x7f2fb0068bf0 2023-10-06T21:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.537+0000 7f2fca781b80 1 Processor -- start 2023-10-06T21:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.537+0000 7f2fca781b80 1 -- start start 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e215c400 con 0x55e9e21fe3d0 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e214dbc0 con 0x55e9e21f3520 2023-10-06T21:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2fca781b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e214b4e0 con 0x55e9e21f2c00 2023-10-06T21:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.538+0000 7f2faffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34072/0 (socket says 172.21.15.31:34072) 2023-10-06T21:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 -- 172.21.15.31:0/4070902594 learned_addr learned my addr 172.21.15.31:0/4070902594 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2fb539c700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 -- 172.21.15.31:0/4070902594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 msgr2=0x55e9e21f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 -- 172.21.15.31:0/4070902594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 msgr2=0x55e9e21fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2faffff700 1 -- 172.21.15.31:0/4070902594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e1f71d80 con 0x55e9e21fe3d0 2023-10-06T21:40:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.539+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.540+0000 7f2faffff700 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0cb530 tx=0x7f2f9c0d63c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90fc0cb0e3866b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.540+0000 7f2fadffb700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 0x7f2fa00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.540+0000 7f2fadffb700 1 -- 172.21.15.31:0/2910571433 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2f9c0cdee0 con 0x7f2fb0068bf0 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.540+0000 7f2fb539c700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 0x7f2fa00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.540+0000 7f2f9affd700 1 -- 172.21.15.31:0/4070902594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c097220 con 0x55e9e21fe3d0 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.541+0000 7f2f9affd700 1 -- 172.21.15.31:0/4070902594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f9c09acb0 con 0x55e9e21fe3d0 2023-10-06T21:40:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.541+0000 7f2f9affd700 1 -- 172.21.15.31:0/4070902594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c09b5e0 con 0x55e9e21fe3d0 2023-10-06T21:40:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.541+0000 7f2fca781b80 1 -- 172.21.15.31:0/4070902594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 msgr2=0x55e9e22007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.541+0000 7f2fca781b80 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0cb530 tx=0x7f2f9c0d63c0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.541+0000 7f2fb539c700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 0x7f2fa00a3b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0073e30 tx=0x7f2fa8009050 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 -- 172.21.15.31:0/4070902594 shutdown_connections 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e22007c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 --2- 172.21.15.31:0/4070902594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 -- 172.21.15.31:0/4070902594 >> 172.21.15.31:0/4070902594 conn(0x55e9e2168200 msgr2=0x55e9e21ef080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 -- 172.21.15.31:0/4070902594 shutdown_connections 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 -- 172.21.15.31:0/4070902594 wait complete. 2023-10-06T21:40:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.542+0000 7f2fca781b80 1 -- start start 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 Processor -- start 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e2209830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e2146dd0 con 0x55e9e21fe3d0 2023-10-06T21:40:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e20e36b0 con 0x55e9e21f3520 2023-10-06T21:40:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.543+0000 7f2fca781b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e20e3350 con 0x55e9e21f2c00 2023-10-06T21:40:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fca781b80 10 client.0 _ll_register_callbacks cb 0x55e9e1f407a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:33562/0 (socket says 172.21.15.31:33562) 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fb4b9b700 1 -- 172.21.15.31:0/2695387755 learned_addr learned my addr 172.21.15.31:0/2695387755 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fb539c700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e2209830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 -- 172.21.15.31:0/2695387755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 msgr2=0x55e9e21fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 -- 172.21.15.31:0/2695387755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 msgr2=0x55e9e2209830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e2209830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2faffff700 1 -- 172.21.15.31:0/2695387755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4009c90 con 0x55e9e21f3520 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.544+0000 7f2fb539c700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e2209830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2faffff700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fecf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c000c00 tx=0x7f2f9c009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c354e8e0e9c0fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2f8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0d1b20 con 0x55e9e21f3520 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2f9c09b5e0 con 0x55e9e21f3520 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9e21e6d50 con 0x55e9e21f3520 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.545+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f9c097220 con 0x55e9e21f3520 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 10 client.27389 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f2fb01a6170 con 0x55e9e21f3520 2023-10-06T21:40:01.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:01 smithi146 ceph-mon[102809]: pgmap v602: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f9c0b7020 con 0x55e9e21f3520 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f9c09e020 con 0x55e9e21f3520 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 10 client.27389 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 10 client.27389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 20 client.27389 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e220d600 0x55e9e220f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 4 client.27389 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.546+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9e213fdb0 con 0x55e9e220d600 2023-10-06T21:40:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.547+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e220d600 0x55e9e220f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.547+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e220d600 0x55e9e220f9f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3ec0081c695090 server_cookie=b1a1fb653c7a5f3d in_seq=0 out_seq=0 2023-10-06T21:40:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.547+0000 7f2f8ffff700 10 client.27389 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.549+0000 7f2f8ffff700 1 -- 172.21.15.31:0/2695387755 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9e213fdb0 con 0x55e9e220d600 2023-10-06T21:40:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.549+0000 7f2f8ffff700 10 client.27389 handle_command_reply: tid=0 2023-10-06T21:40:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.549+0000 7f2fca781b80 1 client.27389 shutdown 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.549+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 msgr2=0x55e9e21fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.549+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fecf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c000c00 tx=0x7f2f9c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 shutdown_connections 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e220d600 0x55e9e220f9f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21fe3d0 0x55e9e2209830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e21fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2695387755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21fecf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 >> 172.21.15.31:0/2695387755 conn(0x55e9e2168200 msgr2=0x55e9e21eec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2f8ffff700 0 client.27389 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 shutdown_connections 2023-10-06T21:40:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 1 -- 172.21.15.31:0/2695387755 wait complete. 2023-10-06T21:40:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.550+0000 7f2fca781b80 20 client.27389 trim_cache size 0 max 16384 2023-10-06T21:40:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.576+0000 7f2fca781b80 1 Processor -- start 2023-10-06T21:40:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.576+0000 7f2fca781b80 1 -- start start 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.576+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e2203790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.576+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 0x55e9e220f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 0x55e9e220fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fca781b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e1fc78c0 con 0x55e9e21f3520 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fca781b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e21fdbb0 con 0x55e9e21fe3d0 2023-10-06T21:40:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fca781b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e21eb180 con 0x55e9e21f2c00 2023-10-06T21:40:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2faffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 0x55e9e220f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fb539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 0x55e9e220fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e2203790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fb539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 0x55e9e220fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:47288/0 (socket says 172.21.15.31:47288) 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.577+0000 7f2fb539c700 1 -- 172.21.15.31:0/2174283616 learned_addr learned my addr 172.21.15.31:0/2174283616 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 -- 172.21.15.31:0/2174283616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 msgr2=0x55e9e220fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 0x55e9e220fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 -- 172.21.15.31:0/2174283616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 msgr2=0x55e9e220f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 0x55e9e220f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 -- 172.21.15.31:0/2174283616 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e2214580 con 0x55e9e21f2c00 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2faffff700 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 0x55e9e220f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.578+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e2203790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e9e2210230 tx=0x7f2fa4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76497172464a242d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.579+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/2174283616 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa401e270 con 0x55e9e21f2c00 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.579+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/2174283616 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa4022070 con 0x55e9e21f2c00 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.579+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/2174283616 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40053e0 con 0x55e9e21f2c00 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 -- 172.21.15.31:0/2174283616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 msgr2=0x55e9e2203790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e2203790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e9e2210230 tx=0x7f2fa4004750 comp rx=0 tx=0).stop 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 -- 172.21.15.31:0/2174283616 shutdown_connections 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21fe3d0 0x55e9e220fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21f2c00 0x55e9e2203790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 --2- 172.21.15.31:0/2174283616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f3520 0x55e9e220f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 -- 172.21.15.31:0/2174283616 >> 172.21.15.31:0/2174283616 conn(0x55e9e2168200 msgr2=0x55e9e21fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 -- 172.21.15.31:0/2174283616 shutdown_connections 2023-10-06T21:40:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.580+0000 7f2fca781b80 1 -- 172.21.15.31:0/2174283616 wait complete. 2023-10-06T21:40:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.581+0000 7f2fca781b80 1 -- start start 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.581+0000 7f2fca781b80 1 Processor -- start 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 0x55e9e21ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21fe3d0 0x55e9e2260350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e22003a0 con 0x55e9e21f2c00 2023-10-06T21:40:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e2200510 con 0x55e9e21f3520 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e2260890 con 0x55e9e21fe3d0 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fca781b80 10 client.0 _ll_register_callbacks cb 0x7f2fa4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 0x55e9e21ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fb4b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 0x55e9e21ff6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34100/0 (socket says 172.21.15.31:34100) 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fb4b9b700 1 -- 172.21.15.31:0/1979603357 learned_addr learned my addr 172.21.15.31:0/1979603357 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2faffff700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.582+0000 7f2fb539c700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21fe3d0 0x55e9e2260350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 -- 172.21.15.31:0/1979603357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21fe3d0 msgr2=0x55e9e2260350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21fe3d0 0x55e9e2260350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 -- 172.21.15.31:0/1979603357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 msgr2=0x55e9e21ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 0x55e9e21ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 -- 172.21.15.31:0/1979603357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb01a6170 con 0x55e9e21f3520 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2faffff700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21ffc00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00e040 tx=0x7f2f9c00fd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5249eb459cfa8cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.583+0000 7f2f8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0d1b20 con 0x55e9e21f3520 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2f9c0b8080 con 0x55e9e21f3520 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f9c09acb0 con 0x55e9e21f3520 2023-10-06T21:40:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f9c097220 con 0x55e9e21f3520 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2fca781b80 10 client.27395 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.584+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f2fa40132c0 con 0x55e9e21f3520 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f9c09acb0 con 0x55e9e21f3520 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f9c09e070 con 0x55e9e21f3520 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 10 client.27395 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 10 client.27395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 20 client.27395 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e2261b20 0x55e9e2263f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 4 client.27395 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9e213fad0 con 0x55e9e2261b20 2023-10-06T21:40:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.585+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e2261b20 0x55e9e2263f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.586+0000 7f2fb4b9b700 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e2261b20 0x55e9e2263f10 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1aff14ba5c023 server_cookie=35dbe07095692c90 in_seq=0 out_seq=0 2023-10-06T21:40:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.586+0000 7f2f8ffff700 10 client.27395 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.645+0000 7f2f8ffff700 1 -- 172.21.15.31:0/1979603357 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9e213fad0 con 0x55e9e2261b20 2023-10-06T21:40:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.645+0000 7f2f8ffff700 10 client.27395 handle_command_reply: tid=0 2023-10-06T21:40:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.645+0000 7f2fca781b80 1 client.27395 shutdown 2023-10-06T21:40:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 msgr2=0x55e9e21ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21ffc00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00e040 tx=0x7f2f9c00fd00 comp rx=0 tx=0).stop 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 shutdown_connections 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e9e21fe3d0 0x55e9e2260350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e9e2261b20 0x55e9e2263f10 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9e21f2c00 0x55e9e21ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 --2- 172.21.15.31:0/1979603357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9e21f3520 0x55e9e21ffc00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2f8ffff700 0 client.27395 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 >> 172.21.15.31:0/1979603357 conn(0x55e9e2168200 msgr2=0x55e9e21fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 shutdown_connections 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.646+0000 7f2fca781b80 1 -- 172.21.15.31:0/1979603357 wait complete. 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.647+0000 7f2fca781b80 20 client.27395 trim_cache size 0 max 16384 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:40:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 msgr2=0x7f2fa00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 0x7f2fa00a3b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0073e30 tx=0x7f2fa8009050 comp rx=0 tx=0).stop 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 msgr2=0x7f2fb0199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005910 tx=0x7f2f9c00f690 comp rx=0 tx=0).stop 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 shutdown_connections 2023-10-06T21:40:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb010db20 0x7f2fb019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fb00682d0 0x7f2fb0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2fa00a1670 0x7f2fa00a3b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 --2- 172.21.15.31:0/2910571433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb0068bf0 0x7f2fb0199670 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.648+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 >> 172.21.15.31:0/2910571433 conn(0x7f2fb0076b70 msgr2=0x7f2fb010f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.649+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 shutdown_connections 2023-10-06T21:40:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:01.649+0000 7f2fb6dff700 1 -- 172.21.15.31:0/2910571433 wait complete. 2023-10-06T21:40:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:01 smithi142 ceph-mon[102951]: pgmap v602: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2023-10-06T21:40:01.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 41955 6.40 MB/sec execute 230 sec latency 72.519 ms 2023-10-06T21:40:02.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42056 6.39 MB/sec execute 231 sec latency 66.307 ms 2023-10-06T21:40:03.255 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:03 smithi146 ceph-mon[102809]: pgmap v603: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2023-10-06T21:40:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:03 smithi031 ceph-mon[98539]: pgmap v603: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2023-10-06T21:40:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:03 smithi142 ceph-mon[102951]: pgmap v603: 97 pgs: 97 active+clean; 774 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2023-10-06T21:40:03.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42150 6.37 MB/sec execute 232 sec latency 94.100 ms 2023-10-06T21:40:04.773 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42264 6.36 MB/sec execute 233 sec latency 91.185 ms 2023-10-06T21:40:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:05 smithi031 ceph-mon[98539]: pgmap v604: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 277 op/s 2023-10-06T21:40:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:05 smithi031 ceph-mon[98539]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:40:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:05 smithi146 ceph-mon[102809]: pgmap v604: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 277 op/s 2023-10-06T21:40:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:05 smithi146 ceph-mon[102809]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:40:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:05 smithi142 ceph-mon[102951]: pgmap v604: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 277 op/s 2023-10-06T21:40:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:05 smithi142 ceph-mon[102951]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:40:05.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42402 6.38 MB/sec execute 234 sec latency 79.672 ms 2023-10-06T21:40:06.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42537 6.39 MB/sec execute 235 sec latency 73.641 ms 2023-10-06T21:40:07.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:07 smithi146 ceph-mon[102809]: pgmap v606: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:07 smithi031 ceph-mon[98539]: pgmap v606: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:07 smithi142 ceph-mon[102951]: pgmap v606: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:07.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42670 6.39 MB/sec execute 236 sec latency 69.101 ms 2023-10-06T21:40:08.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42823 6.39 MB/sec execute 237 sec latency 68.446 ms 2023-10-06T21:40:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:09 smithi031 ceph-mon[98539]: pgmap v607: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:09 smithi146 ceph-mon[102809]: pgmap v607: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:09 smithi142 ceph-mon[102951]: pgmap v607: 97 pgs: 97 active+clean; 779 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2023-10-06T21:40:09.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 42948 6.40 MB/sec execute 238 sec latency 69.759 ms 2023-10-06T21:40:10.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43051 6.39 MB/sec execute 239 sec latency 70.409 ms 2023-10-06T21:40:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:11 smithi031 ceph-mon[98539]: pgmap v608: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:11 smithi146 ceph-mon[102809]: pgmap v608: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:11 smithi142 ceph-mon[102951]: pgmap v608: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:11.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43147 6.38 MB/sec execute 240 sec latency 235.188 ms 2023-10-06T21:40:12.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43246 6.38 MB/sec execute 241 sec latency 304.205 ms 2023-10-06T21:40:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:13 smithi031 ceph-mon[98539]: pgmap v609: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:13 smithi146 ceph-mon[102809]: pgmap v609: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:13 smithi142 ceph-mon[102951]: pgmap v609: 97 pgs: 97 active+clean; 778 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2023-10-06T21:40:13.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43354 6.37 MB/sec execute 242 sec latency 219.573 ms 2023-10-06T21:40:14.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43474 6.36 MB/sec execute 243 sec latency 70.454 ms 2023-10-06T21:40:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 -- 172.21.15.31:0/1391840566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241010d0 msgr2=0x7f89241014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 --2- 172.21.15.31:0/1391840566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241010d0 0x7f89241014b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f891c009990 tx=0x7f891c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 -- 172.21.15.31:0/1391840566 shutdown_connections 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 --2- 172.21.15.31:0/1391840566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241061f0 0x7f892410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 --2- 172.21.15.31:0/1391840566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241010d0 0x7f89241014b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 --2- 172.21.15.31:0/1391840566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8924101a80 0x7f8924105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 -- 172.21.15.31:0/1391840566 >> 172.21.15.31:0/1391840566 conn(0x7f89240fc950 msgr2=0x7f89240fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 -- 172.21.15.31:0/1391840566 shutdown_connections 2023-10-06T21:40:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.414+0000 7f892bfff700 1 -- 172.21.15.31:0/1391840566 wait complete. 2023-10-06T21:40:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.416+0000 7f892bfff700 1 Processor -- start 2023-10-06T21:40:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.416+0000 7f892bfff700 1 -- start start 2023-10-06T21:40:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.416+0000 7f892bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241010d0 0x7f89241a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.416+0000 7f892bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.416+0000 7f892bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 0x7f892410cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892410d590 con 0x7f89241061f0 2023-10-06T21:40:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892bfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892410d6d0 con 0x7f892410cb10 2023-10-06T21:40:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892410d890 con 0x7f89241010d0 2023-10-06T21:40:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33728/0 (socket says 172.21.15.31:33728) 2023-10-06T21:40:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892959a700 1 -- 172.21.15.31:0/3352097617 learned_addr learned my addr 172.21.15.31:0/3352097617 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 0x7f892410cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.417+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241010d0 0x7f89241a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 -- 172.21.15.31:0/3352097617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241010d0 msgr2=0x7f89241a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241010d0 0x7f89241a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 -- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 msgr2=0x7f89241a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 -- 172.21.15.31:0/3352097617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0095f0 con 0x7f892410cb10 2023-10-06T21:40:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892959a700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 0x7f892410cfc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f891400efc0 tx=0x7f891400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f8922ffd700 1 -- 172.21.15.31:0/3352097617 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914018070 con 0x7f892410cb10 2023-10-06T21:40:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f8922ffd700 1 -- 172.21.15.31:0/3352097617 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f891400f040 con 0x7f892410cb10 2023-10-06T21:40:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.418+0000 7f8922ffd700 1 -- 172.21.15.31:0/3352097617 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400cba0 con 0x7f892410cb10 2023-10-06T21:40:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.419+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241aa4b0 con 0x7f892410cb10 2023-10-06T21:40:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.419+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89241aa970 con 0x7f892410cb10 2023-10-06T21:40:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.424+0000 7f8922ffd700 1 -- 172.21.15.31:0/3352097617 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8914003a00 con 0x7f892410cb10 2023-10-06T21:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.425+0000 7f893f958b80 1 Processor -- start 2023-10-06T21:40:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.425+0000 7f893f958b80 1 -- start start 2023-10-06T21:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5decfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5d56400 con 0x55c6a5decc00 2023-10-06T21:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5d47bc0 con 0x55c6a5ded520 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f893f958b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5d454e0 con 0x55c6a5df83d0 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.426+0000 7f8922ffd700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 0x7f890c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5decfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5decfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33740/0 (socket says 172.21.15.31:33740) 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 -- 172.21.15.31:0/910603089 learned_addr learned my addr 172.21.15.31:0/910603089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f8929d9b700 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 0x7f890c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f8922ffd700 1 -- 172.21.15.31:0/3352097617 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f89140ccce0 con 0x7f892410cb10 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 -- 172.21.15.31:0/910603089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 msgr2=0x55c6a5df7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 -- 172.21.15.31:0/910603089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 msgr2=0x55c6a5decfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5decfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.427+0000 7f892959a700 1 -- 172.21.15.31:0/910603089 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a5b6bd80 con 0x55c6a5df83d0 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.428+0000 7f892959a700 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfa7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8918015b00 tx=0x7f8918009210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de596eea099f83da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.428+0000 7f890bfff700 1 -- 172.21.15.31:0/910603089 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918017900 con 0x55c6a5df83d0 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.428+0000 7f890bfff700 1 -- 172.21.15.31:0/910603089 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8918015630 con 0x55c6a5df83d0 2023-10-06T21:40:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.428+0000 7f890bfff700 1 -- 172.21.15.31:0/910603089 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918018c40 con 0x55c6a5df83d0 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 -- 172.21.15.31:0/910603089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 msgr2=0x55c6a5dfa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfa7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8918015b00 tx=0x7f8918009210 comp rx=0 tx=0).stop 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f892a59c700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 0x7f890c0a3ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f891401b000 tx=0x7f8914095920 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 -- 172.21.15.31:0/910603089 shutdown_connections 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfa7c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5decfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 --2- 172.21.15.31:0/910603089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 -- 172.21.15.31:0/910603089 >> 172.21.15.31:0/910603089 conn(0x55c6a5d62200 msgr2=0x55c6a5de9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 -- 172.21.15.31:0/910603089 shutdown_connections 2023-10-06T21:40:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.429+0000 7f893f958b80 1 -- 172.21.15.31:0/910603089 wait complete. 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.430+0000 7f893f958b80 1 -- start start 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.430+0000 7f893f958b80 1 Processor -- start 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5df50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5df57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5d40dd0 con 0x55c6a5df83d0 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5cdd6b0 con 0x55c6a5decc00 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5cdd350 con 0x55c6a5ded520 2023-10-06T21:40:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f893f958b80 10 client.0 _ll_register_callbacks cb 0x55c6a5b3a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f892959a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f892959a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42236/0 (socket says 172.21.15.31:42236) 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.431+0000 7f892959a700 1 -- 172.21.15.31:0/4079326225 learned_addr learned my addr 172.21.15.31:0/4079326225 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f8929d9b700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5df50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 -- 172.21.15.31:0/4079326225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 msgr2=0x55c6a5df50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5df50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 -- 172.21.15.31:0/4079326225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 msgr2=0x55c6a5df57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5df57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 -- 172.21.15.31:0/4079326225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89180157e0 con 0x55c6a5decc00 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f892959a700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8918015f40 tx=0x7f8918015f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa31696a7b73d68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.432+0000 7f89097fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180099b0 con 0x55c6a5decc00 2023-10-06T21:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8918017900 con 0x55c6a5decc00 2023-10-06T21:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a5de0d50 con 0x55c6a5decc00 2023-10-06T21:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8918017ac0 con 0x55c6a5decc00 2023-10-06T21:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f893f958b80 10 client.27404 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.433+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c6a5b54c00 con 0x55c6a5decc00 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c6a5de0d50 con 0x55c6a5decc00 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8918029aa0 con 0x55c6a5decc00 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 10 client.27404 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 10 client.27404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 20 client.27404 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e07d40 0x55c6a5e0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 4 client.27404 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.434+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a5d39db0 con 0x55c6a5e07d40 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.435+0000 7f8929d9b700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e07d40 0x55c6a5e0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.436+0000 7f8929d9b700 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e07d40 0x55c6a5e0a130 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d36e3aec86640b server_cookie=c0233a0190e1a38 in_seq=0 out_seq=0 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.436+0000 7f89097fa700 10 client.27404 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.436+0000 7f89097fa700 1 -- 172.21.15.31:0/4079326225 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6a5d39db0 con 0x55c6a5e07d40 2023-10-06T21:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.436+0000 7f89097fa700 10 client.27404 handle_command_reply: tid=0 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 client.27404 shutdown 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 msgr2=0x55c6a5df7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8918015f40 tx=0x7f8918015f70 comp rx=0 tx=0).stop 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 shutdown_connections 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e07d40 0x55c6a5e0a130 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5df57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5df7e40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 --2- 172.21.15.31:0/4079326225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5df50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 >> 172.21.15.31:0/4079326225 conn(0x55c6a5d62200 msgr2=0x55c6a5de8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f89097fa700 0 client.27404 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.437+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 shutdown_connections 2023-10-06T21:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.438+0000 7f893f958b80 1 -- 172.21.15.31:0/4079326225 wait complete. 2023-10-06T21:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.438+0000 7f893f958b80 20 client.27404 trim_cache size 0 max 16384 2023-10-06T21:40:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 Processor -- start 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 -- start start 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5df79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5df0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5bc18c0 con 0x55c6a5decc00 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5de9dc0 con 0x55c6a5ded520 2023-10-06T21:40:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.464+0000 7f893f958b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5df3660 con 0x55c6a5df83d0 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5df79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42242/0 (socket says 172.21.15.31:42242) 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 -- 172.21.15.31:0/3016687277 learned_addr learned my addr 172.21.15.31:0/3016687277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f892959a700 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5df0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 -- 172.21.15.31:0/3016687277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 msgr2=0x55c6a5df0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5df0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 -- 172.21.15.31:0/3016687277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 msgr2=0x55c6a5df79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5df79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.465+0000 7f8929d9b700 1 -- 172.21.15.31:0/3016687277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a5b921f0 con 0x55c6a5ded520 2023-10-06T21:40:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.466+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f891c009990 tx=0x7f891c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c9195e0a202bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.467+0000 7f8920ff9700 1 -- 172.21.15.31:0/3016687277 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c01d040 con 0x55c6a5ded520 2023-10-06T21:40:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.467+0000 7f8920ff9700 1 -- 172.21.15.31:0/3016687277 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f891c01e040 con 0x55c6a5ded520 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.467+0000 7f8920ff9700 1 -- 172.21.15.31:0/3016687277 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0052a0 con 0x55c6a5ded520 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 -- 172.21.15.31:0/3016687277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 msgr2=0x55c6a5df0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f891c009990 tx=0x7f891c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 -- 172.21.15.31:0/3016687277 shutdown_connections 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5df83d0 0x55c6a5df0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5decc00 0x55c6a5df79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 --2- 172.21.15.31:0/3016687277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5ded520 0x55c6a5df0430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 -- 172.21.15.31:0/3016687277 >> 172.21.15.31:0/3016687277 conn(0x55c6a5d62200 msgr2=0x55c6a5de77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 -- 172.21.15.31:0/3016687277 shutdown_connections 2023-10-06T21:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.468+0000 7f893f958b80 1 -- 172.21.15.31:0/3016687277 wait complete. 2023-10-06T21:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.469+0000 7f893f958b80 1 -- start start 2023-10-06T21:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.469+0000 7f893f958b80 1 Processor -- start 2023-10-06T21:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5dfeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5dff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5dfd7b0 con 0x55c6a5df83d0 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5dfd920 con 0x55c6a5decc00 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a5dfdae0 con 0x55c6a5ded520 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f893f958b80 10 client.0 _ll_register_callbacks cb 0x7f89180185d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f892959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33774/0 (socket says 172.21.15.31:33774) 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f8929d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5dff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.470+0000 7f892959a700 1 -- 172.21.15.31:0/3361161125 learned_addr learned my addr 172.21.15.31:0/3361161125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f892959a700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5dfeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 -- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 msgr2=0x55c6a5dfeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5dfeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 -- 172.21.15.31:0/3361161125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 msgr2=0x55c6a5dfd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 -- 172.21.15.31:0/3361161125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892410e5e0 con 0x55c6a5ded520 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f8929d9b700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5dff030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55c6a5df6750 tx=0x7f891c005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef84fdb64a7ccbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.471+0000 7f89097fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c01d030 con 0x55c6a5ded520 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f891c0280c0 con 0x55c6a5ded520 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8918015630 con 0x55c6a5ded520 2023-10-06T21:40:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f891c0282a0 con 0x55c6a5ded520 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f893f958b80 10 client.27349 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.472+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c6a5b6a250 con 0x55c6a5ded520 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8918015630 con 0x55c6a5ded520 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f891c003810 con 0x55c6a5ded520 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 10 client.27349 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 10 client.27349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 20 client.27349 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e5c970 0x55c6a5e5ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 4 client.27349 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:40:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.473+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6a5d39ad0 con 0x55c6a5e5c970 2023-10-06T21:40:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.474+0000 7f892959a700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e5c970 0x55c6a5e5ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.474+0000 7f892959a700 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e5c970 0x55c6a5e5ed60 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f764d6221cc27e6 server_cookie=2031b6533eabd150 in_seq=0 out_seq=0 2023-10-06T21:40:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.474+0000 7f89097fa700 10 client.27349 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f89097fa700 1 -- 172.21.15.31:0/3361161125 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6a5d39ad0 con 0x55c6a5e5c970 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f89097fa700 10 client.27349 handle_command_reply: tid=0 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 client.27349 shutdown 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 msgr2=0x55c6a5dff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5dff030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55c6a5df6750 tx=0x7f891c005a60 comp rx=0 tx=0).stop 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 shutdown_connections 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c6a5e5c970 0x55c6a5e5ed60 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a5df83d0 0x55c6a5dfd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5decc00 0x55c6a5dfeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 --2- 172.21.15.31:0/3361161125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6a5ded520 0x55c6a5dff030 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 >> 172.21.15.31:0/3361161125 conn(0x55c6a5d62200 msgr2=0x55c6a5df0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f89097fa700 0 client.27349 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.524+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 shutdown_connections 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.525+0000 7f893f958b80 1 -- 172.21.15.31:0/3361161125 wait complete. 2023-10-06T21:40:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.525+0000 7f893f958b80 20 client.27349 trim_cache size 0 max 16384 2023-10-06T21:40:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:40:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:40:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:40:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 msgr2=0x7f890c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 0x7f890c0a3ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f891401b000 tx=0x7f8914095920 comp rx=0 tx=0).stop 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 msgr2=0x7f892410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 0x7f892410cfc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f891400efc0 tx=0x7f891400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 shutdown_connections 2023-10-06T21:40:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f892410cb10 0x7f892410cfc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89241010d0 0x7f89241a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f890c0a15f0 0x7f890c0a3ab0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 --2- 172.21.15.31:0/3352097617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89241061f0 0x7f89241a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 >> 172.21.15.31:0/3352097617 conn(0x7f89240fc950 msgr2=0x7f89240fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 shutdown_connections 2023-10-06T21:40:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:15.526+0000 7f892bfff700 1 -- 172.21.15.31:0/3352097617 wait complete. 2023-10-06T21:40:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:15 smithi146 ceph-mon[102809]: pgmap v610: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:15 smithi146 ceph-mon[102809]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:40:15.560 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:15 smithi031 ceph-mon[98539]: pgmap v610: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:15.560 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:15 smithi031 ceph-mon[98539]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:40:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:15 smithi142 ceph-mon[102951]: pgmap v610: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:15.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:15 smithi142 ceph-mon[102951]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:40:15.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43583 6.35 MB/sec execute 244 sec latency 107.446 ms 2023-10-06T21:40:16.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43722 6.36 MB/sec execute 245 sec latency 69.041 ms 2023-10-06T21:40:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:17 smithi031 ceph-mon[98539]: pgmap v612: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:17 smithi146 ceph-mon[102809]: pgmap v612: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:17 smithi142 ceph-mon[102951]: pgmap v612: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:17.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43850 6.36 MB/sec execute 246 sec latency 69.889 ms 2023-10-06T21:40:18.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 43991 6.36 MB/sec execute 247 sec latency 68.710 ms 2023-10-06T21:40:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:19 smithi031 ceph-mon[98539]: pgmap v613: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:19 smithi146 ceph-mon[102809]: pgmap v613: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:19.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:19 smithi142 ceph-mon[102951]: pgmap v613: 97 pgs: 97 active+clean; 785 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 226 op/s 2023-10-06T21:40:19.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44107 6.38 MB/sec execute 248 sec latency 69.714 ms 2023-10-06T21:40:20.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44233 6.38 MB/sec execute 249 sec latency 74.737 ms 2023-10-06T21:40:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:21 smithi031 ceph-mon[98539]: pgmap v614: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:21 smithi146 ceph-mon[102809]: pgmap v614: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:21 smithi142 ceph-mon[102951]: pgmap v614: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:21.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44399 6.40 MB/sec execute 250 sec latency 73.818 ms 2023-10-06T21:40:22.774 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44535 6.43 MB/sec execute 251 sec latency 198.110 ms 2023-10-06T21:40:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:23 smithi146 ceph-mon[102809]: pgmap v615: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:23.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:23 smithi031 ceph-mon[98539]: pgmap v615: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:23 smithi142 ceph-mon[102951]: pgmap v615: 97 pgs: 97 active+clean; 793 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2023-10-06T21:40:23.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44656 6.42 MB/sec execute 252 sec latency 70.338 ms 2023-10-06T21:40:24.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44753 6.41 MB/sec execute 253 sec latency 77.250 ms 2023-10-06T21:40:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:25 smithi031 ceph-mon[98539]: pgmap v616: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 222 op/s 2023-10-06T21:40:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:25 smithi146 ceph-mon[102809]: pgmap v616: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 222 op/s 2023-10-06T21:40:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:25 smithi142 ceph-mon[102951]: pgmap v616: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 222 op/s 2023-10-06T21:40:25.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44875 6.43 MB/sec execute 254 sec latency 63.985 ms 2023-10-06T21:40:26.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 44994 6.44 MB/sec execute 255 sec latency 64.843 ms 2023-10-06T21:40:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:27 smithi031 ceph-mon[98539]: pgmap v617: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2023-10-06T21:40:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:27 smithi146 ceph-mon[102809]: pgmap v617: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2023-10-06T21:40:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:27 smithi142 ceph-mon[102951]: pgmap v617: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2023-10-06T21:40:27.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45094 6.43 MB/sec execute 256 sec latency 65.165 ms 2023-10-06T21:40:28.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45219 6.44 MB/sec execute 257 sec latency 74.898 ms 2023-10-06T21:40:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:29 smithi031 ceph-mon[98539]: pgmap v618: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 185 op/s 2023-10-06T21:40:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:29 smithi146 ceph-mon[102809]: pgmap v618: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 185 op/s 2023-10-06T21:40:29.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:29 smithi142 ceph-mon[102951]: pgmap v618: 97 pgs: 97 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 185 op/s 2023-10-06T21:40:29.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45320 6.43 MB/sec execute 258 sec latency 88.600 ms 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.386+0000 7f20787ff700 1 -- 172.21.15.31:0/696494611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20700fff70 msgr2=0x7f20701003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.386+0000 7f20787ff700 1 --2- 172.21.15.31:0/696494611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20700fff70 0x7f20701003f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2060009920 tx=0x7f2060009c30 comp rx=0 tx=0).stop 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 -- 172.21.15.31:0/696494611 shutdown_connections 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 --2- 172.21.15.31:0/696494611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070100930 0x7f207010cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 --2- 172.21.15.31:0/696494611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2070105ef0 0x7f20701062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 --2- 172.21.15.31:0/696494611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20700fff70 0x7f20701003f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 -- 172.21.15.31:0/696494611 >> 172.21.15.31:0/696494611 conn(0x7f2070076b70 msgr2=0x7f2070076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 -- 172.21.15.31:0/696494611 shutdown_connections 2023-10-06T21:40:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.387+0000 7f20787ff700 1 -- 172.21.15.31:0/696494611 wait complete. 2023-10-06T21:40:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.388+0000 7f20787ff700 1 Processor -- start 2023-10-06T21:40:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.389+0000 7f20787ff700 1 -- start start 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.389+0000 7f20787ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.389+0000 7f20787ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2070100930 0x7f2070199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f20787ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070105ef0 0x7f207019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f20787ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019e3e0 con 0x7f2070105ef0 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f20787ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019e550 con 0x7f2070100930 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f20787ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019e710 con 0x7f20700fff70 2023-10-06T21:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48828/0 (socket says 172.21.15.31:48828) 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f207659b700 1 -- 172.21.15.31:0/2386063202 learned_addr learned my addr 172.21.15.31:0/2386063202 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2070100930 0x7f2070199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.390+0000 7f2076d9c700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070105ef0 0x7f207019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 -- 172.21.15.31:0/2386063202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2070100930 msgr2=0x7f2070199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2070100930 0x7f2070199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 -- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070105ef0 msgr2=0x7f207019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070105ef0 0x7f207019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 -- 172.21.15.31:0/2386063202 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068009d00 con 0x7f20700fff70 2023-10-06T21:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f207659b700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f206c00e9a0 tx=0x7f206c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f20677fe700 1 -- 172.21.15.31:0/2386063202 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c009790 con 0x7f20700fff70 2023-10-06T21:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f20677fe700 1 -- 172.21.15.31:0/2386063202 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206c010430 con 0x7f20700fff70 2023-10-06T21:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.391+0000 7f20677fe700 1 -- 172.21.15.31:0/2386063202 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00cea0 con 0x7f20700fff70 2023-10-06T21:40:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.392+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2060009520 con 0x7f20700fff70 2023-10-06T21:40:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.392+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20701a6050 con 0x7f20700fff70 2023-10-06T21:40:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.396+0000 7f20677fe700 1 -- 172.21.15.31:0/2386063202 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f206c0105a0 con 0x7f20700fff70 2023-10-06T21:40:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.398+0000 7f20677fe700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 0x7f205c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.399+0000 7f20677fe700 1 -- 172.21.15.31:0/2386063202 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f206c014070 con 0x7f20700fff70 2023-10-06T21:40:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.400+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 0x7f205c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.401+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 0x7f205c0a3b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f20600094f0 tx=0x7f2060019040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.421+0000 7f208c1cbb80 1 Processor -- start 2023-10-06T21:40:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 -- start start 2023-10-06T21:40:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364e9410 0x55ff364e97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364f4be0 0x55ff364f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff3644a400 con 0x55ff364e9410 2023-10-06T21:40:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff3643bbc0 con 0x55ff364e9d30 2023-10-06T21:40:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.422+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff364394e0 con 0x55ff364f4be0 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49458/0 (socket says 172.21.15.31:49458) 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 -- 172.21.15.31:0/1587273589 learned_addr learned my addr 172.21.15.31:0/1587273589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f207659b700 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364e9410 0x55ff364e97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2076d9c700 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364f4be0 0x55ff364f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 -- 172.21.15.31:0/1587273589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364f4be0 msgr2=0x55ff364f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364f4be0 0x55ff364f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 -- 172.21.15.31:0/1587273589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364e9410 msgr2=0x55ff364e97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364e9410 0x55ff364e97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.423+0000 7f2075d9a700 1 -- 172.21.15.31:0/1587273589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff3625fd80 con 0x55ff364e9d30 2023-10-06T21:40:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.424+0000 7f2075d9a700 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2060009920 tx=0x7f206001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b529ee0152ec57f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.424+0000 7f2064ff9700 1 -- 172.21.15.31:0/1587273589 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206002a070 con 0x55ff364e9d30 2023-10-06T21:40:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.425+0000 7f2064ff9700 1 -- 172.21.15.31:0/1587273589 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206001d040 con 0x55ff364e9d30 2023-10-06T21:40:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.425+0000 7f2064ff9700 1 -- 172.21.15.31:0/1587273589 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060005260 con 0x55ff364e9d30 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 -- 172.21.15.31:0/1587273589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 msgr2=0x55ff364f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2060009920 tx=0x7f206001e000 comp rx=0 tx=0).stop 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 -- 172.21.15.31:0/1587273589 shutdown_connections 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364f4be0 0x55ff364f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364e9410 0x55ff364e97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/1587273589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f46a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.426+0000 7f208c1cbb80 1 -- 172.21.15.31:0/1587273589 >> 172.21.15.31:0/1587273589 conn(0x55ff36456200 msgr2=0x55ff364dcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.427+0000 7f208c1cbb80 1 -- 172.21.15.31:0/1587273589 shutdown_connections 2023-10-06T21:40:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.427+0000 7f208c1cbb80 1 -- 172.21.15.31:0/1587273589 wait complete. 2023-10-06T21:40:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.427+0000 7f208c1cbb80 1 -- start start 2023-10-06T21:40:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.428+0000 7f208c1cbb80 1 Processor -- start 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36434dd0 con 0x55ff364f4be0 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff363d16b0 con 0x55ff364e9d30 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff363d1350 con 0x55ff364e9410 2023-10-06T21:40:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f208c1cbb80 10 client.0 _ll_register_callbacks cb 0x55ff3622e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f2075d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48836/0 (socket says 172.21.15.31:48836) 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.429+0000 7f207659b700 1 -- 172.21.15.31:0/2350162763 learned_addr learned my addr 172.21.15.31:0/2350162763 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f2076d9c700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 -- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 msgr2=0x55ff364f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 -- 172.21.15.31:0/2350162763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 msgr2=0x55ff364f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 -- 172.21.15.31:0/2350162763 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207010d560 con 0x55ff364e9410 2023-10-06T21:40:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f2076d9c700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f207659b700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f206c099e90 tx=0x7f206c099b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595c75e4a50fa575 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.430+0000 7f2059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0b1070 con 0x55ff364e9410 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f206c09a890 con 0x55ff364e9410 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff364d4d50 con 0x55ff364e9410 2023-10-06T21:40:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206c09a190 con 0x55ff364e9410 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f208c1cbb80 10 client.27361 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ff36248c00 con 0x55ff364e9410 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff364d4d50 con 0x55ff364e9410 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f206c0a2090 con 0x55ff364e9410 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.431+0000 7f208c1cbb80 10 client.27361 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f208c1cbb80 10 client.27361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f208c1cbb80 20 client.27361 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff365045e0 0x55ff36506a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f208c1cbb80 4 client.27361 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff3642ddb0 con 0x55ff365045e0 2023-10-06T21:40:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.432+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff365045e0 0x55ff36506a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.433+0000 7f2059ffb700 10 client.27361 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.433+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff365045e0 0x55ff36506a00 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e0320260698c9e server_cookie=1f03787e3da9f188 in_seq=0 out_seq=0 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f2059ffb700 1 -- 172.21.15.31:0/2350162763 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff3642ddb0 con 0x55ff365045e0 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f2059ffb700 10 client.27361 handle_command_reply: tid=0 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 client.27361 shutdown 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 msgr2=0x55ff364f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f206c099e90 tx=0x7f206c099b60 comp rx=0 tx=0).stop 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 shutdown_connections 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff365045e0 0x55ff36506a00 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f4650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2350162763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 >> 172.21.15.31:0/2350162763 conn(0x55ff36456200 msgr2=0x55ff364dcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f2059ffb700 0 client.27361 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 shutdown_connections 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.434+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2350162763 wait complete. 2023-10-06T21:40:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.435+0000 7f208c1cbb80 20 client.27361 trim_cache size 0 max 16384 2023-10-06T21:40:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.461+0000 7f208c1cbb80 1 Processor -- start 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.461+0000 7f208c1cbb80 1 -- start start 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 0x55ff364f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff362b58c0 con 0x55ff364f4be0 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff364f2df0 con 0x55ff364e9410 2023-10-06T21:40:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff364fbba0 con 0x55ff364e9d30 2023-10-06T21:40:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f207659b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f207659b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:49476/0 (socket says 172.21.15.31:49476) 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.462+0000 7f207659b700 1 -- 172.21.15.31:0/459030182 learned_addr learned my addr 172.21.15.31:0/459030182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f2076d9c700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f2075d9a700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 0x55ff364f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f207659b700 1 -- 172.21.15.31:0/459030182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 msgr2=0x55ff364f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f207659b700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 0x55ff364f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f207659b700 1 -- 172.21.15.31:0/459030182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 msgr2=0x55ff364ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f207659b700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.463+0000 7f207659b700 1 -- 172.21.15.31:0/459030182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff362861f0 con 0x55ff364e9410 2023-10-06T21:40:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.464+0000 7f2075d9a700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 0x55ff364f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:40:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.465+0000 7f207659b700 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c2d0 tx=0x7f206c09a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679dd55237ca523b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.466+0000 7f20657fa700 1 -- 172.21.15.31:0/459030182 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0b1070 con 0x55ff364e9410 2023-10-06T21:40:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.466+0000 7f20657fa700 1 -- 172.21.15.31:0/459030182 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206c09a890 con 0x55ff364e9410 2023-10-06T21:40:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.466+0000 7f20657fa700 1 -- 172.21.15.31:0/459030182 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c09e2e0 con 0x55ff364e9410 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 -- 172.21.15.31:0/459030182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 msgr2=0x55ff364fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c2d0 tx=0x7f206c09a090 comp rx=0 tx=0).stop 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 -- 172.21.15.31:0/459030182 shutdown_connections 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff364ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9410 0x55ff364fb660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/459030182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9d30 0x55ff364f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.467+0000 7f208c1cbb80 1 -- 172.21.15.31:0/459030182 >> 172.21.15.31:0/459030182 conn(0x55ff36456200 msgr2=0x55ff364db7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.468+0000 7f208c1cbb80 1 -- 172.21.15.31:0/459030182 shutdown_connections 2023-10-06T21:40:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.468+0000 7f208c1cbb80 1 -- 172.21.15.31:0/459030182 wait complete. 2023-10-06T21:40:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.469+0000 7f208c1cbb80 1 -- start start 2023-10-06T21:40:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.470+0000 7f208c1cbb80 1 Processor -- start 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff365573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff364f82c0 con 0x55ff364f4be0 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff364f8430 con 0x55ff364e9d30 2023-10-06T21:40:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36568910 con 0x55ff364e9410 2023-10-06T21:40:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.471+0000 7f208c1cbb80 10 client.0 _ll_register_callbacks cb 0x7f206c095e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.472+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.472+0000 7f207659b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48850/0 (socket says 172.21.15.31:48850) 2023-10-06T21:40:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.472+0000 7f207659b700 1 -- 172.21.15.31:0/2171753486 learned_addr learned my addr 172.21.15.31:0/2171753486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.472+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f2076d9c700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff365573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f207659b700 1 -- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 msgr2=0x55ff364f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f207659b700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f207659b700 1 -- 172.21.15.31:0/2171753486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 msgr2=0x55ff365573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f207659b700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff365573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.473+0000 7f207659b700 1 -- 172.21.15.31:0/2171753486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff36248c00 con 0x55ff364e9410 2023-10-06T21:40:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.474+0000 7f207659b700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f206c0a2490 tx=0x7f206c0b8580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9907961d9e052730 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.474+0000 7f2059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:40:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.474+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0b1070 con 0x55ff364e9410 2023-10-06T21:40:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.475+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f206c0a2520 con 0x55ff364e9410 2023-10-06T21:40:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.475+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f206001d040 con 0x55ff364e9410 2023-10-06T21:40:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.476+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206c09e2e0 con 0x55ff364e9410 2023-10-06T21:40:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.476+0000 7f208c1cbb80 10 client.27364 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.476+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ff3625e250 con 0x55ff364e9410 2023-10-06T21:40:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.476+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206c0a6030 con 0x55ff364e9410 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.476+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f206c0991c0 con 0x55ff364e9410 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 10 client.27364 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 10 client.27364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 20 client.27364 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff36569b80 0x55ff3656bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 4 client.27364 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.477+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff3642dad0 con 0x55ff36569b80 2023-10-06T21:40:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.478+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff36569b80 0x55ff3656bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.482+0000 7f2075d9a700 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff36569b80 0x55ff3656bfa0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9438375d1c96fb76 server_cookie=f91e92d627fe072d in_seq=0 out_seq=0 2023-10-06T21:40:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.483+0000 7f2059ffb700 10 client.27364 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f2059ffb700 1 -- 172.21.15.31:0/2171753486 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff3642dad0 con 0x55ff36569b80 2023-10-06T21:40:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f2059ffb700 10 client.27364 handle_command_reply: tid=0 2023-10-06T21:40:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 client.27364 shutdown 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 msgr2=0x55ff364f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f206c0a2490 tx=0x7f206c0b8580 comp rx=0 tx=0).stop 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 shutdown_connections 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55ff36569b80 0x55ff3656bfa0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff364f4be0 0x55ff365573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff364e9410 0x55ff364f7610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 --2- 172.21.15.31:0/2171753486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff364e9d30 0x55ff364f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 >> 172.21.15.31:0/2171753486 conn(0x55ff36456200 msgr2=0x55ff364ecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f2059ffb700 0 client.27364 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.538+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 shutdown_connections 2023-10-06T21:40:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.539+0000 7f208c1cbb80 1 -- 172.21.15.31:0/2171753486 wait complete. 2023-10-06T21:40:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.539+0000 7f208c1cbb80 20 client.27364 trim_cache size 0 max 16384 2023-10-06T21:40:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:40:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 msgr2=0x7f205c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 0x7f205c0a3b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f20600094f0 tx=0x7f2060019040 comp rx=0 tx=0).stop 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 msgr2=0x7f20701991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f206c00e9a0 tx=0x7f206c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 shutdown_connections 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f205c0a1670 0x7f205c0a3b30 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f20600094f0 tx=0x7f2060019040 comp rx=0 tx=0).stop 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2070105ef0 0x7f207019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20700fff70 0x7f20701991f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 --2- 172.21.15.31:0/2386063202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2070100930 0x7f2070199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 >> 172.21.15.31:0/2386063202 conn(0x7f2070076b70 msgr2=0x7f2070068350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 shutdown_connections 2023-10-06T21:40:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:30.540+0000 7f20787ff700 1 -- 172.21.15.31:0/2386063202 wait complete. 2023-10-06T21:40:30.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45429 6.42 MB/sec execute 259 sec latency 108.192 ms 2023-10-06T21:40:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:31 smithi031 ceph-mon[98539]: pgmap v619: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2023-10-06T21:40:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:31 smithi146 ceph-mon[102809]: pgmap v619: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2023-10-06T21:40:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:31 smithi142 ceph-mon[102951]: pgmap v619: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2023-10-06T21:40:31.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45536 6.40 MB/sec execute 260 sec latency 62.955 ms 2023-10-06T21:40:32.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45638 6.38 MB/sec execute 261 sec latency 77.910 ms 2023-10-06T21:40:33.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:33 smithi031 ceph-mon[98539]: pgmap v620: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 173 op/s 2023-10-06T21:40:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:33 smithi146 ceph-mon[102809]: pgmap v620: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 173 op/s 2023-10-06T21:40:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:33 smithi142 ceph-mon[102951]: pgmap v620: 97 pgs: 97 active+clean; 817 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 173 op/s 2023-10-06T21:40:33.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45763 6.38 MB/sec execute 262 sec latency 70.712 ms 2023-10-06T21:40:34.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 45883 6.38 MB/sec execute 263 sec latency 71.029 ms 2023-10-06T21:40:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:35 smithi031 ceph-mon[98539]: pgmap v621: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 272 op/s 2023-10-06T21:40:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:35 smithi146 ceph-mon[102809]: pgmap v621: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 272 op/s 2023-10-06T21:40:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:35 smithi142 ceph-mon[102951]: pgmap v621: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 272 op/s 2023-10-06T21:40:35.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46012 6.39 MB/sec execute 264 sec latency 70.666 ms 2023-10-06T21:40:36.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46138 6.38 MB/sec execute 265 sec latency 87.193 ms 2023-10-06T21:40:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:37 smithi031 ceph-mon[98539]: pgmap v622: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:37 smithi146 ceph-mon[102809]: pgmap v622: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:37 smithi142 ceph-mon[102951]: pgmap v622: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:37.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46265 6.40 MB/sec execute 266 sec latency 66.804 ms 2023-10-06T21:40:38.775 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46351 6.39 MB/sec execute 267 sec latency 325.598 ms 2023-10-06T21:40:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:39 smithi031 ceph-mon[98539]: pgmap v623: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:39 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:40:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:39 smithi146 ceph-mon[102809]: pgmap v623: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:39 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:40:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:39 smithi142 ceph-mon[102951]: pgmap v623: 97 pgs: 97 active+clean; 818 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-10-06T21:40:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:39 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:40:39.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46477 6.39 MB/sec execute 268 sec latency 297.885 ms 2023-10-06T21:40:40.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46585 6.38 MB/sec execute 269 sec latency 78.872 ms 2023-10-06T21:40:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:41 smithi031 ceph-mon[98539]: pgmap v624: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 277 op/s 2023-10-06T21:40:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:41 smithi146 ceph-mon[102809]: pgmap v624: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 277 op/s 2023-10-06T21:40:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:41 smithi142 ceph-mon[102951]: pgmap v624: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 277 op/s 2023-10-06T21:40:41.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46687 6.37 MB/sec execute 270 sec latency 376.158 ms 2023-10-06T21:40:42.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46771 6.36 MB/sec execute 271 sec latency 244.814 ms 2023-10-06T21:40:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:43 smithi031 ceph-mon[98539]: pgmap v625: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2023-10-06T21:40:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:43 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:40:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:43 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:40:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:43 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:40:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:43 smithi146 ceph-mon[102809]: pgmap v625: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2023-10-06T21:40:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:43 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:40:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:43 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:40:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:43 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:40:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:43 smithi142 ceph-mon[102951]: pgmap v625: 97 pgs: 97 active+clean; 822 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2023-10-06T21:40:43.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:43 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:40:43.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:43 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:40:43.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:43 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:40:43.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 46909 6.36 MB/sec execute 272 sec latency 76.515 ms 2023-10-06T21:40:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 -- 172.21.15.31:0/1596406340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c0753e0 msgr2=0x7f946c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 --2- 172.21.15.31:0/1596406340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c0753e0 0x7f946c0757c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9460009990 tx=0x7f9460009ca0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 -- 172.21.15.31:0/1596406340 shutdown_connections 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 --2- 172.21.15.31:0/1596406340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 0x7f946c114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 --2- 172.21.15.31:0/1596406340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c0753e0 0x7f946c0757c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 --2- 172.21.15.31:0/1596406340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c075d00 0x7f946c111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 -- 172.21.15.31:0/1596406340 >> 172.21.15.31:0/1596406340 conn(0x7f946c0feba0 msgr2=0x7f946c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 -- 172.21.15.31:0/1596406340 shutdown_connections 2023-10-06T21:40:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.798+0000 7f94741ff700 1 -- 172.21.15.31:0/1596406340 wait complete. 2023-10-06T21:40:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.800+0000 7f94741ff700 1 Processor -- start 2023-10-06T21:40:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.800+0000 7f94741ff700 1 -- start start 2023-10-06T21:40:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.800+0000 7f94741ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 0x7f946c1a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.800+0000 7f94741ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f94741ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 0x7f946c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f94741ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c19da70 con 0x7f946c075d00 2023-10-06T21:40:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f94741ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c19dbe0 con 0x7f946c111d50 2023-10-06T21:40:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f94741ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c19dda0 con 0x7f946c0753e0 2023-10-06T21:40:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f947179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 0x7f946c1a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f947179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52760/0 (socket says 172.21.15.31:52760) 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f947179a700 1 -- 172.21.15.31:0/193983954 learned_addr learned my addr 172.21.15.31:0/193983954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f947279c700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 0x7f946c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 -- 172.21.15.31:0/193983954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 msgr2=0x7f946c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 0x7f946c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 -- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 msgr2=0x7f946c1a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f9471f9b700 1 -- 172.21.15.31:0/193983954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600095f0 con 0x7f946c0753e0 2023-10-06T21:40:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.801+0000 7f947179a700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.802+0000 7f9471f9b700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 0x7f946c1a35a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9460006010 tx=0x7f9460004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.802+0000 7f945effd700 1 -- 172.21.15.31:0/193983954 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946001d070 con 0x7f946c0753e0 2023-10-06T21:40:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.802+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c19e020 con 0x7f946c0753e0 2023-10-06T21:40:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.803+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1aa590 con 0x7f946c0753e0 2023-10-06T21:40:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.803+0000 7f945effd700 1 -- 172.21.15.31:0/193983954 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9460003ec0 con 0x7f946c0753e0 2023-10-06T21:40:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.803+0000 7f945effd700 1 -- 172.21.15.31:0/193983954 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000fac0 con 0x7f946c0753e0 2023-10-06T21:40:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.808+0000 7f945effd700 1 -- 172.21.15.31:0/193983954 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9460004030 con 0x7f946c0753e0 2023-10-06T21:40:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.809+0000 7f9487b63b80 1 Processor -- start 2023-10-06T21:40:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.811+0000 7f945effd700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 0x7f94580a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.811+0000 7f945effd700 1 -- 172.21.15.31:0/193983954 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f94600cde90 con 0x7f946c0753e0 2023-10-06T21:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 -- start start 2023-10-06T21:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f947279c700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 0x7f94580a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1cd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 0x5561a1ce37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1c3f400 con 0x5561a1cd5c00 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1c30bc0 con 0x5561a1ce13d0 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.812+0000 7f9487b63b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1c2e4e0 con 0x5561a1cd6520 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:51776/0 (socket says 172.21.15.31:51776) 2023-10-06T21:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 -- 172.21.15.31:0/4292556992 learned_addr learned my addr 172.21.15.31:0/4292556992 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947279c700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 0x5561a1ce37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 -- 172.21.15.31:0/4292556992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 msgr2=0x5561a1ce37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f9471f9b700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1cd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 0x5561a1ce37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 -- 172.21.15.31:0/4292556992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 msgr2=0x5561a1cd5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1cd5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947179a700 1 -- 172.21.15.31:0/4292556992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a1a54d80 con 0x5561a1cd6520 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947279c700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 0x5561a1ce37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f9471f9b700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1cd5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.813+0000 7f947279c700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 0x7f94580a3bb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f946c1a42a0 tx=0x7f9464009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.814+0000 7f947179a700 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9468010610 tx=0x7f9468010920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3326013d3e3e23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.814+0000 7f9457fff700 1 -- 172.21.15.31:0/4292556992 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468011700 con 0x5561a1cd6520 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.814+0000 7f9457fff700 1 -- 172.21.15.31:0/4292556992 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9468015070 con 0x5561a1cd6520 2023-10-06T21:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.814+0000 7f9457fff700 1 -- 172.21.15.31:0/4292556992 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468019b10 con 0x5561a1cd6520 2023-10-06T21:40:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 -- 172.21.15.31:0/4292556992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 msgr2=0x5561a1ce0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9468010610 tx=0x7f9468010920 comp rx=0 tx=0).stop 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 -- 172.21.15.31:0/4292556992 shutdown_connections 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1ce13d0 0x5561a1ce37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1cd5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 --2- 172.21.15.31:0/4292556992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1cd6520 0x5561a1ce0e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 -- 172.21.15.31:0/4292556992 >> 172.21.15.31:0/4292556992 conn(0x5561a1c4b200 msgr2=0x5561a1cd2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 -- 172.21.15.31:0/4292556992 shutdown_connections 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.815+0000 7f9487b63b80 1 -- 172.21.15.31:0/4292556992 wait complete. 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.816+0000 7f9487b63b80 1 -- start start 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.816+0000 7f9487b63b80 1 Processor -- start 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1ce0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cde540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1c29dd0 con 0x5561a1cd5c00 2023-10-06T21:40:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1bc66b0 con 0x5561a1cd6520 2023-10-06T21:40:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1bc6350 con 0x5561a1ce13d0 2023-10-06T21:40:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f9487b63b80 10 client.0 _ll_register_callbacks cb 0x5561a1a237a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f947179a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f947179a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45158/0 (socket says 172.21.15.31:45158) 2023-10-06T21:40:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.817+0000 7f947179a700 1 -- 172.21.15.31:0/2009744248 learned_addr learned my addr 172.21.15.31:0/2009744248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947279c700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cde540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f9471f9b700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1ce0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 -- 172.21.15.31:0/2009744248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 msgr2=0x5561a1cde540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cde540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 -- 172.21.15.31:0/2009744248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 msgr2=0x5561a1ce0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1ce0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 -- 172.21.15.31:0/2009744248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a1a3dc00 con 0x5561a1cd6520 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.818+0000 7f947179a700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9468019730 tx=0x7f9468005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb232e8338fbdbb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f94557fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680054e0 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a1cc9d50 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9468005640 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9468004050 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f9487b63b80 10 client.27437 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.819+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5561a1a7b1f0 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5561a1cc9d50 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9468004750 con 0x5561a1cd6520 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 10 client.27437 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 10 client.27437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 20 client.27437 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1cf0ca0 0x5561a1cf30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 4 client.27437 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:40:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.820+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a1c22db0 con 0x5561a1cf0ca0 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.821+0000 7f9471f9b700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1cf0ca0 0x5561a1cf30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.822+0000 7f9471f9b700 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1cf0ca0 0x5561a1cf30c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9557c2cd143b74e server_cookie=6d16b4bca70e6c21 in_seq=0 out_seq=0 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.822+0000 7f94557fa700 10 client.27437 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.823+0000 7f94557fa700 1 -- 172.21.15.31:0/2009744248 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561a1c22db0 con 0x5561a1cf0ca0 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.823+0000 7f94557fa700 10 client.27437 handle_command_reply: tid=0 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.823+0000 7f9487b63b80 1 client.27437 shutdown 2023-10-06T21:40:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 msgr2=0x5561a1cddf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9468019730 tx=0x7f9468005040 comp rx=0 tx=0).stop 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 shutdown_connections 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cde540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1cf0ca0 0x5561a1cf30c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cd5c00 0x5561a1ce0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 --2- 172.21.15.31:0/2009744248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd6520 0x5561a1cddf80 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9468019730 tx=0x7f9468005040 comp rx=0 tx=0).stop 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 >> 172.21.15.31:0/2009744248 conn(0x5561a1c4b200 msgr2=0x5561a1cd1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f94557fa700 0 client.27437 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 shutdown_connections 2023-10-06T21:40:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.824+0000 7f9487b63b80 1 -- 172.21.15.31:0/2009744248 wait complete. 2023-10-06T21:40:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.825+0000 7f9487b63b80 20 client.27437 trim_cache size 0 max 16384 2023-10-06T21:40:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 Processor -- start 2023-10-06T21:40:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 -- start start 2023-10-06T21:40:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cd9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1cd97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1aaa8c0 con 0x5561a1cecc10 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1cdc530 con 0x5561a1cd5c00 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.851+0000 7f9487b63b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1cd9e20 con 0x5561a1ce13d0 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45174/0 (socket says 172.21.15.31:45174) 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 -- 172.21.15.31:0/3186010053 learned_addr learned my addr 172.21.15.31:0/3186010053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f947279c700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1cd97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f947179a700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cd9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 -- 172.21.15.31:0/3186010053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 msgr2=0x5561a1cd9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cd9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 -- 172.21.15.31:0/3186010053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 msgr2=0x5561a1cd97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1cd97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f9471f9b700 1 -- 172.21.15.31:0/3186010053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a1cf7cf0 con 0x5561a1cd5c00 2023-10-06T21:40:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.852+0000 7f947179a700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cd9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:40:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.853+0000 7f9471f9b700 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f946009ab30 tx=0x7f94600a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84456037bc4c89c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.853+0000 7f945cff9700 1 -- 172.21.15.31:0/3186010053 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946009ae30 con 0x5561a1cd5c00 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.853+0000 7f945cff9700 1 -- 172.21.15.31:0/3186010053 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f946009ef80 con 0x5561a1cd5c00 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.853+0000 7f945cff9700 1 -- 172.21.15.31:0/3186010053 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000fe80 con 0x5561a1cd5c00 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 -- 172.21.15.31:0/3186010053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 msgr2=0x5561a1cec6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f946009ab30 tx=0x7f94600a1040 comp rx=0 tx=0).stop 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 -- 172.21.15.31:0/3186010053 shutdown_connections 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1cd97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1cec6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 --2- 172.21.15.31:0/3186010053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1cd9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 -- 172.21.15.31:0/3186010053 >> 172.21.15.31:0/3186010053 conn(0x5561a1c4b200 msgr2=0x5561a1cd07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 -- 172.21.15.31:0/3186010053 shutdown_connections 2023-10-06T21:40:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.854+0000 7f9487b63b80 1 -- 172.21.15.31:0/3186010053 wait complete. 2023-10-06T21:40:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.855+0000 7f9487b63b80 1 -- start start 2023-10-06T21:40:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.855+0000 7f9487b63b80 1 Processor -- start 2023-10-06T21:40:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1d3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1d3e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1ce3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1ce44c0 con 0x5561a1cecc10 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1ce4630 con 0x5561a1cd5c00 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a1ce47f0 con 0x5561a1ce13d0 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9487b63b80 10 client.0 _ll_register_callbacks cb 0x5561a1a237a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f947279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1ce3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f947279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1ce3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52796/0 (socket says 172.21.15.31:52796) 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f947279c700 1 -- 172.21.15.31:0/171901688 learned_addr learned my addr 172.21.15.31:0/171901688 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f9471f9b700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1d3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.856+0000 7f947179a700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1d3e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 -- 172.21.15.31:0/171901688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 msgr2=0x5561a1d3e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1d3e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 -- 172.21.15.31:0/171901688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 msgr2=0x5561a1ce3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1ce3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 -- 172.21.15.31:0/171901688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a1a7b1f0 con 0x5561a1cd5c00 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f9471f9b700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1d3dd60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f946009adc0 tx=0x7f946009aa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7dae74cc9d1b5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f94557fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000fe30 con 0x5561a1cd5c00 2023-10-06T21:40:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.857+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f946009b090 con 0x5561a1cd5c00 2023-10-06T21:40:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.858+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9468015070 con 0x5561a1cd5c00 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.858+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946009b2b0 con 0x5561a1cd5c00 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.858+0000 7f9487b63b80 10 client.27443 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.858+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f946c1aa590 con 0x5561a1cd5c00 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.858+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9468015070 con 0x5561a1cd5c00 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f94600aeb80 con 0x5561a1cd5c00 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 10 client.27443 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 10 client.27443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 20 client.27443 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1d45680 0x5561a1d47a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 4 client.27443 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:40:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561a1c22ad0 con 0x5561a1d45680 2023-10-06T21:40:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.859+0000 7f947179a700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1d45680 0x5561a1d47a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.861+0000 7f947179a700 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1d45680 0x5561a1d47a70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231a12fffd43254e server_cookie=e0a61b35eff3f688 in_seq=0 out_seq=0 2023-10-06T21:40:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.861+0000 7f94557fa700 10 client.27443 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.917+0000 7f94557fa700 1 -- 172.21.15.31:0/171901688 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561a1c22ad0 con 0x5561a1d45680 2023-10-06T21:40:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.917+0000 7f94557fa700 10 client.27443 handle_command_reply: tid=0 2023-10-06T21:40:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.917+0000 7f9487b63b80 1 client.27443 shutdown 2023-10-06T21:40:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 msgr2=0x5561a1d3dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1d3dd60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f946009adc0 tx=0x7f946009aa30 comp rx=0 tx=0).stop 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 shutdown_connections 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5561a1d45680 0x5561a1d47a70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561a1cecc10 0x5561a1ce3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561a1cd5c00 0x5561a1d3dd60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 --2- 172.21.15.31:0/171901688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561a1ce13d0 0x5561a1d3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 >> 172.21.15.31:0/171901688 conn(0x5561a1c4b200 msgr2=0x5561a1cd9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.918+0000 7f94557fa700 0 client.27443 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.919+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 shutdown_connections 2023-10-06T21:40:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.919+0000 7f9487b63b80 1 -- 172.21.15.31:0/171901688 wait complete. 2023-10-06T21:40:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.920+0000 7f9487b63b80 20 client.27443 trim_cache size 0 max 16384 2023-10-06T21:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:40:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.921+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 msgr2=0x7f94580a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.921+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 0x7f94580a3bb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f946c1a42a0 tx=0x7f9464009040 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.921+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 msgr2=0x7f946c1a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.921+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 0x7f946c1a35a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9460006010 tx=0x7f9460004730 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 shutdown_connections 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f946c111d50 0x7f946c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0753e0 0x7f946c1a35a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f94580a16f0 0x7f94580a3bb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 --2- 172.21.15.31:0/193983954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c075d00 0x7f946c1a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.922+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 >> 172.21.15.31:0/193983954 conn(0x7f946c0feba0 msgr2=0x7f946c1129c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.923+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 shutdown_connections 2023-10-06T21:40:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:43.923+0000 7f94741ff700 1 -- 172.21.15.31:0/193983954 wait complete. 2023-10-06T21:40:44.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47018 6.34 MB/sec execute 273 sec latency 95.851 ms 2023-10-06T21:40:45.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:45 smithi031 ceph-mon[98539]: pgmap v626: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 271 op/s 2023-10-06T21:40:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:45 smithi146 ceph-mon[102809]: pgmap v626: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 271 op/s 2023-10-06T21:40:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:45 smithi142 ceph-mon[102951]: pgmap v626: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 271 op/s 2023-10-06T21:40:45.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47145 6.35 MB/sec execute 274 sec latency 84.890 ms 2023-10-06T21:40:46.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47236 6.34 MB/sec execute 275 sec latency 314.642 ms 2023-10-06T21:40:47.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:47 smithi031 ceph-mon[98539]: pgmap v627: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:47.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:47 smithi146 ceph-mon[102809]: pgmap v627: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:47 smithi142 ceph-mon[102951]: pgmap v627: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:47.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47376 6.36 MB/sec execute 276 sec latency 69.686 ms 2023-10-06T21:40:48.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47500 6.37 MB/sec execute 277 sec latency 66.109 ms 2023-10-06T21:40:49.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:49 smithi031 ceph-mon[98539]: pgmap v628: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:49.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:49 smithi146 ceph-mon[102809]: pgmap v628: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:49 smithi142 ceph-mon[102951]: pgmap v628: 97 pgs: 97 active+clean; 827 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 173 op/s 2023-10-06T21:40:49.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47642 6.38 MB/sec execute 278 sec latency 69.776 ms 2023-10-06T21:40:50.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47781 6.39 MB/sec execute 279 sec latency 97.347 ms 2023-10-06T21:40:51.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:51 smithi031 ceph-mon[98539]: pgmap v629: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2023-10-06T21:40:51.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:51 smithi146 ceph-mon[102809]: pgmap v629: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2023-10-06T21:40:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:51 smithi142 ceph-mon[102951]: pgmap v629: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2023-10-06T21:40:51.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 47918 6.40 MB/sec execute 280 sec latency 66.511 ms 2023-10-06T21:40:52.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48050 6.41 MB/sec execute 281 sec latency 74.149 ms 2023-10-06T21:40:53.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:53 smithi031 ceph-mon[98539]: pgmap v630: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2023-10-06T21:40:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:53 smithi146 ceph-mon[102809]: pgmap v630: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2023-10-06T21:40:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:53 smithi142 ceph-mon[102951]: pgmap v630: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2023-10-06T21:40:53.776 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48136 6.41 MB/sec execute 282 sec latency 339.297 ms 2023-10-06T21:40:54.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48268 6.43 MB/sec execute 283 sec latency 391.113 ms 2023-10-06T21:40:55.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:55 smithi031 ceph-mon[98539]: pgmap v631: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 252 op/s 2023-10-06T21:40:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:55 smithi146 ceph-mon[102809]: pgmap v631: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 252 op/s 2023-10-06T21:40:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:55 smithi142 ceph-mon[102951]: pgmap v631: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 252 op/s 2023-10-06T21:40:55.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48373 6.42 MB/sec execute 284 sec latency 212.232 ms 2023-10-06T21:40:56.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48492 6.43 MB/sec execute 285 sec latency 78.336 ms 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.158+0000 7fc8889ff700 1 -- 172.21.15.31:0/811946088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 msgr2=0x7fc88411c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.158+0000 7fc8889ff700 1 --2- 172.21.15.31:0/811946088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc88411c940 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00b0a0 tx=0x7fc87c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 -- 172.21.15.31:0/811946088 shutdown_connections 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 --2- 172.21.15.31:0/811946088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc88411c940 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 --2- 172.21.15.31:0/811946088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 0x7fc88406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 --2- 172.21.15.31:0/811946088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 0x7fc88411a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 -- 172.21.15.31:0/811946088 >> 172.21.15.31:0/811946088 conn(0x7fc88406bcc0 msgr2=0x7fc884109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 -- 172.21.15.31:0/811946088 shutdown_connections 2023-10-06T21:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.159+0000 7fc8889ff700 1 -- 172.21.15.31:0/811946088 wait complete. 2023-10-06T21:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.160+0000 7fc8889ff700 1 Processor -- start 2023-10-06T21:40:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.160+0000 7fc8889ff700 1 -- start start 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 0x7fc8841a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 0x7fc8841a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841aaf80 con 0x7fc88411a550 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841ab0f0 con 0x7fc88406c5e0 2023-10-06T21:40:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc8889ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841ab2b0 con 0x7fc88406cf00 2023-10-06T21:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.161+0000 7fc882d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49112/0 (socket says 172.21.15.31:49112) 2023-10-06T21:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 -- 172.21.15.31:0/3495650059 learned_addr learned my addr 172.21.15.31:0/3495650059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 0x7fc8841a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc88259c700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 0x7fc8841a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 -- 172.21.15.31:0/3495650059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 msgr2=0x7fc8841a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 0x7fc8841a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 -- 172.21.15.31:0/3495650059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 msgr2=0x7fc8841a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 0x7fc8841a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.162+0000 7fc882d9d700 1 -- 172.21.15.31:0/3495650059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009520 con 0x7fc88411a550 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.163+0000 7fc882d9d700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0062a0 tx=0x7fc87c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.163+0000 7fc87b7fe700 1 -- 172.21.15.31:0/3495650059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00e040 con 0x7fc88411a550 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.163+0000 7fc87b7fe700 1 -- 172.21.15.31:0/3495650059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc87c003e10 con 0x7fc88411a550 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.164+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c009d00 con 0x7fc88411a550 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.164+0000 7fc87b7fe700 1 -- 172.21.15.31:0/3495650059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c005040 con 0x7fc88411a550 2023-10-06T21:40:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.164+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841b2bf0 con 0x7fc88411a550 2023-10-06T21:40:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.168+0000 7fc87b7fe700 1 -- 172.21.15.31:0/3495650059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc87c01d480 con 0x7fc88411a550 2023-10-06T21:40:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.170+0000 7fc89c2c3b80 1 Processor -- start 2023-10-06T21:40:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.170+0000 7fc89c2c3b80 1 -- start start 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.170+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9ef1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.170+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9eff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9e5b400 con 0x5617c9ef1c00 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9e4cbc0 con 0x5617c9ef2520 2023-10-06T21:40:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9e4a4e0 con 0x5617c9efd3d0 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc88259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc87b7fe700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 0x7fc8700a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9ef1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc88259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41242/0 (socket says 172.21.15.31:41242) 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc88259c700 1 -- 172.21.15.31:0/367986286 learned_addr learned my addr 172.21.15.31:0/367986286 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc87b7fe700 1 -- 172.21.15.31:0/3495650059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc87c0ce1f0 con 0x7fc88411a550 2023-10-06T21:40:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.171+0000 7fc881d9b700 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9eff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc88259c700 1 -- 172.21.15.31:0/367986286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 msgr2=0x5617c9eff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc88259c700 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9eff7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc88259c700 1 -- 172.21.15.31:0/367986286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 msgr2=0x5617c9ef1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc88259c700 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9ef1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc88259c700 1 -- 172.21.15.31:0/367986286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c9c70d80 con 0x5617c9ef2520 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.172+0000 7fc882d9d700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 0x7fc8700a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.173+0000 7fc88259c700 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc87400bd70 tx=0x7fc8740106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd0905bac2a99f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.173+0000 7fc878ff9700 1 -- 172.21.15.31:0/367986286 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874010e10 con 0x5617c9ef2520 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.173+0000 7fc878ff9700 1 -- 172.21.15.31:0/367986286 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc874015070 con 0x5617c9ef2520 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.173+0000 7fc878ff9700 1 -- 172.21.15.31:0/367986286 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740116a0 con 0x5617c9ef2520 2023-10-06T21:40:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.173+0000 7fc882d9d700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 0x7fc8700a3b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0a2f60 tx=0x7fc87c009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/367986286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 msgr2=0x5617c9efce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc87400bd70 tx=0x7fc8740106b0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/367986286 shutdown_connections 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9eff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9ef1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/367986286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef2520 0x5617c9efce90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/367986286 >> 172.21.15.31:0/367986286 conn(0x5617c9e67200 msgr2=0x5617c9eee080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/367986286 shutdown_connections 2023-10-06T21:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.174+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/367986286 wait complete. 2023-10-06T21:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.175+0000 7fc89c2c3b80 1 -- start start 2023-10-06T21:40:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.175+0000 7fc89c2c3b80 1 Processor -- start 2023-10-06T21:40:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.177+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.177+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9efa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9efa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9e45dd0 con 0x5617c9ef2520 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9de26b0 con 0x5617c9ef1c00 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9de2350 con 0x5617c9efd3d0 2023-10-06T21:40:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc89c2c3b80 10 client.0 _ll_register_callbacks cb 0x5617c9c3f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41258/0 (socket says 172.21.15.31:41258) 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 -- 172.21.15.31:0/2419393666 learned_addr learned my addr 172.21.15.31:0/2419393666 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9efa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 -- 172.21.15.31:0/2419393666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 msgr2=0x5617c9efa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.178+0000 7fc881d9b700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9efa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc881d9b700 1 -- 172.21.15.31:0/2419393666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 msgr2=0x5617c9efa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc881d9b700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9efa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc881d9b700 1 -- 172.21.15.31:0/2419393666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87400b9a0 con 0x5617c9ef1c00 2023-10-06T21:40:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc881d9b700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc86c013f70 tx=0x7fc86c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b160d406d27fa229 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00fd30 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c9ee5d50 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.179+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc86c010430 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc86c00fe90 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 10 client.27455 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5617c9c59c00 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617c9ee5d50 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc86c023400 con 0x5617c9ef1c00 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 10 client.27455 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 10 client.27455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 20 client.27455 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f0cf20 0x5617c9f0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 4 client.27455 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.180+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617c9e3edb0 con 0x5617c9f0cf20 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.181+0000 7fc88259c700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f0cf20 0x5617c9f0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.182+0000 7fc869ffb700 10 client.27455 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.182+0000 7fc88259c700 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f0cf20 0x5617c9f0f340 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ea0e4864f7939d server_cookie=c804aa41347bd772 in_seq=0 out_seq=0 2023-10-06T21:40:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.183+0000 7fc869ffb700 1 -- 172.21.15.31:0/2419393666 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617c9e3edb0 con 0x5617c9f0cf20 2023-10-06T21:40:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.183+0000 7fc869ffb700 10 client.27455 handle_command_reply: tid=0 2023-10-06T21:40:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.183+0000 7fc89c2c3b80 1 client.27455 shutdown 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.183+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 msgr2=0x5617c9efce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.183+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc86c013f70 tx=0x7fc86c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 shutdown_connections 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9efa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9efce60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f0cf20 0x5617c9f0f340 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/2419393666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9efa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc869ffb700 0 client.27455 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 >> 172.21.15.31:0/2419393666 conn(0x5617c9e67200 msgr2=0x5617c9eedc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 shutdown_connections 2023-10-06T21:40:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.184+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/2419393666 wait complete. 2023-10-06T21:40:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.185+0000 7fc89c2c3b80 20 client.27455 trim_cache size 0 max 16384 2023-10-06T21:40:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.214+0000 7fc89c2c3b80 1 Processor -- start 2023-10-06T21:40:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 -- start start 2023-10-06T21:40:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9f03f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9f08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9cc68c0 con 0x5617c9ef2520 2023-10-06T21:40:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9efb6c0 con 0x5617c9ef1c00 2023-10-06T21:40:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9f04490 con 0x5617c9efd3d0 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41270/0 (socket says 172.21.15.31:41270) 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.215+0000 7fc881d9b700 1 -- 172.21.15.31:0/4253994756 learned_addr learned my addr 172.21.15.31:0/4253994756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9f08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc88259c700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9f03f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 -- 172.21.15.31:0/4253994756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 msgr2=0x5617c9f08670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9f08670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 -- 172.21.15.31:0/4253994756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 msgr2=0x5617c9f03f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9f03f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc881d9b700 1 -- 172.21.15.31:0/4253994756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c9c971f0 con 0x5617c9ef1c00 2023-10-06T21:40:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.216+0000 7fc88259c700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9f03f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:40:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.217+0000 7fc881d9b700 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc86c013c00 tx=0x7fc86c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570601033a427560 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.217+0000 7fc8797fa700 1 -- 172.21.15.31:0/4253994756 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00fd30 con 0x5617c9ef1c00 2023-10-06T21:40:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.217+0000 7fc8797fa700 1 -- 172.21.15.31:0/4253994756 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86c00fe90 con 0x5617c9ef1c00 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.217+0000 7fc8797fa700 1 -- 172.21.15.31:0/4253994756 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c023950 con 0x5617c9ef1c00 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/4253994756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 msgr2=0x5617c9ef4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc86c013c00 tx=0x7fc86c004750 comp rx=0 tx=0).stop 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/4253994756 shutdown_connections 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9efd3d0 0x5617c9f08670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9ef1c00 0x5617c9ef4c10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/4253994756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef2520 0x5617c9f03f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.218+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/4253994756 >> 172.21.15.31:0/4253994756 conn(0x5617c9e67200 msgr2=0x5617c9eec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.219+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/4253994756 shutdown_connections 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.219+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/4253994756 wait complete. 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.219+0000 7fc89c2c3b80 1 -- start start 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 Processor -- start 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9f03a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 0x5617c9f01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9efd3d0 0x5617c9f022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9f02820 con 0x5617c9ef1c00 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9f02990 con 0x5617c9efd3d0 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.220+0000 7fc89c2c3b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9f02b50 con 0x5617c9ef2520 2023-10-06T21:40:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc89c2c3b80 10 client.0 _ll_register_callbacks cb 0x7fc86c013cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc88259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 0x5617c9f01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9f03a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc881d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9f03a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49158/0 (socket says 172.21.15.31:49158) 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc881d9b700 1 -- 172.21.15.31:0/431516376 learned_addr learned my addr 172.21.15.31:0/431516376 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.221+0000 7fc881d9b700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9efd3d0 0x5617c9f022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 -- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9efd3d0 msgr2=0x5617c9f022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9efd3d0 0x5617c9f022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 -- 172.21.15.31:0/431516376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 msgr2=0x5617c9f03a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:40:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9f03a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 -- 172.21.15.31:0/431516376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617c9c59c00 con 0x5617c9ef2520 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc88259c700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 0x5617c9f01da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b690 tx=0x7fc87400b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbfc04698d404d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874022040 con 0x5617c9ef2520 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc874017070 con 0x5617c9ef2520 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.222+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc874015070 con 0x5617c9ef2520 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.223+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc87400bec0 con 0x5617c9ef2520 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.223+0000 7fc89c2c3b80 10 client.27400 fetch_fsmap learned FSMap version 49 2023-10-06T21:40:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.223+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5617c9c6f250 con 0x5617c9ef2520 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.223+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc874015070 con 0x5617c9ef2520 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.223+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc8740050b0 con 0x5617c9ef2520 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 10 client.27400 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 10 client.27400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 20 client.27400 populate_metadata read hostname 'smithi031' 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f62000 0x5617c9f64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:40:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 4 client.27400 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:40:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617c9e3ead0 con 0x5617c9f62000 2023-10-06T21:40:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.224+0000 7fc881d9b700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f62000 0x5617c9f64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:40:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.225+0000 7fc881d9b700 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f62000 0x5617c9f64420 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d182526e3fe63d4 server_cookie=fd87e1c8d1d4f54c in_seq=0 out_seq=0 2023-10-06T21:40:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.225+0000 7fc869ffb700 10 client.27400 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.281+0000 7fc869ffb700 1 -- 172.21.15.31:0/431516376 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617c9e3ead0 con 0x5617c9f62000 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.281+0000 7fc869ffb700 10 client.27400 handle_command_reply: tid=0 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.281+0000 7fc89c2c3b80 1 client.27400 shutdown 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 msgr2=0x5617c9f01da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 0x5617c9f01da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b690 tx=0x7fc87400b710 comp rx=0 tx=0).stop 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 shutdown_connections 2023-10-06T21:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617c9efd3d0 0x5617c9f022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x5617c9f62000 0x5617c9f64420 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617c9ef1c00 0x5617c9f03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 --2- 172.21.15.31:0/431516376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5617c9ef2520 0x5617c9f01da0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc869ffb700 0 client.27400 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 >> 172.21.15.31:0/431516376 conn(0x5617c9e67200 msgr2=0x5617c9f08670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 shutdown_connections 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.282+0000 7fc89c2c3b80 1 -- 172.21.15.31:0/431516376 wait complete. 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.283+0000 7fc89c2c3b80 20 client.27400 trim_cache size 0 max 16384 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 msgr2=0x7fc8700a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 0x7fc8700a3b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0a2f60 tx=0x7fc87c009040 comp rx=0 tx=0).stop 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 msgr2=0x7fc8841aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0062a0 tx=0x7fc87c00bce0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 shutdown_connections 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc8700a16b0 0x7fc8700a3b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88411a550 0x7fc8841aa940 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88406c5e0 0x7fc8841a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 --2- 172.21.15.31:0/3495650059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc88406cf00 0x7fc8841a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 >> 172.21.15.31:0/3495650059 conn(0x7fc88406bcc0 msgr2=0x7fc88411b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.284+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 shutdown_connections 2023-10-06T21:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:40:57.285+0000 7fc8889ff700 1 -- 172.21.15.31:0/3495650059 wait complete. 2023-10-06T21:40:57.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:57 smithi031 ceph-mon[98539]: pgmap v632: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:57.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:57 smithi146 ceph-mon[102809]: pgmap v632: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:57 smithi142 ceph-mon[102951]: pgmap v632: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:57.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48581 6.41 MB/sec execute 286 sec latency 86.631 ms 2023-10-06T21:40:58.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48692 6.41 MB/sec execute 287 sec latency 68.287 ms 2023-10-06T21:40:59.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:40:59 smithi031 ceph-mon[98539]: pgmap v633: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:59.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:40:59 smithi146 ceph-mon[102809]: pgmap v633: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:59.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:40:59 smithi142 ceph-mon[102951]: pgmap v633: 97 pgs: 97 active+clean; 849 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-10-06T21:40:59.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48804 6.41 MB/sec execute 288 sec latency 67.142 ms 2023-10-06T21:41:00.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 48906 6.40 MB/sec execute 289 sec latency 87.636 ms 2023-10-06T21:41:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:01 smithi031 ceph-mon[98539]: pgmap v634: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 266 op/s 2023-10-06T21:41:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:01 smithi146 ceph-mon[102809]: pgmap v634: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 266 op/s 2023-10-06T21:41:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:01 smithi142 ceph-mon[102951]: pgmap v634: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 266 op/s 2023-10-06T21:41:01.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49028 6.39 MB/sec execute 290 sec latency 79.967 ms 2023-10-06T21:41:02.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49149 6.38 MB/sec execute 291 sec latency 79.369 ms 2023-10-06T21:41:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:03 smithi031 ceph-mon[98539]: pgmap v635: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2023-10-06T21:41:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:03 smithi146 ceph-mon[102809]: pgmap v635: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2023-10-06T21:41:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:03 smithi142 ceph-mon[102951]: pgmap v635: 97 pgs: 97 active+clean; 855 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2023-10-06T21:41:03.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49240 6.38 MB/sec execute 292 sec latency 153.116 ms 2023-10-06T21:41:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:04 smithi146 ceph-mon[102809]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:41:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:04 smithi142 ceph-mon[102951]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:41:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:04 smithi031 ceph-mon[98539]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:41:04.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49348 6.37 MB/sec execute 293 sec latency 164.545 ms 2023-10-06T21:41:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:05 smithi146 ceph-mon[102809]: pgmap v637: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:05 smithi142 ceph-mon[102951]: pgmap v637: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:05 smithi031 ceph-mon[98539]: pgmap v637: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:05.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49472 6.37 MB/sec execute 294 sec latency 76.930 ms 2023-10-06T21:41:06.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49590 6.37 MB/sec execute 295 sec latency 175.642 ms 2023-10-06T21:41:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:07 smithi146 ceph-mon[102809]: pgmap v638: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:07 smithi142 ceph-mon[102951]: pgmap v638: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:07.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:07 smithi031 ceph-mon[98539]: pgmap v638: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:07.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49704 6.38 MB/sec execute 296 sec latency 360.480 ms 2023-10-06T21:41:08.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49822 6.38 MB/sec execute 297 sec latency 70.729 ms 2023-10-06T21:41:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:09 smithi146 ceph-mon[102809]: pgmap v639: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:09.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:09 smithi142 ceph-mon[102951]: pgmap v639: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:09 smithi031 ceph-mon[98539]: pgmap v639: 97 pgs: 2 active+clean+snaptrim_wait, 93 active+clean, 2 active+clean+snaptrim; 867 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 236 op/s 2023-10-06T21:41:09.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 49947 6.37 MB/sec execute 298 sec latency 68.301 ms 2023-10-06T21:41:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/1054583833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9810d630 msgr2=0x7f6d9810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/1054583833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9810d630 0x7f6d9810da10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6d940099c0 tx=0x7f6d94009cd0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/1054583833 shutdown_connections 2023-10-06T21:41:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/1054583833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 0x7f6d98114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/1054583833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9810d630 0x7f6d9810da10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/1054583833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d98107cf0 0x7f6d98108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/1054583833 >> 172.21.15.31:0/1054583833 conn(0x7f6d98076c70 msgr2=0x7f6d98079090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/1054583833 shutdown_connections 2023-10-06T21:41:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.522+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/1054583833 wait complete. 2023-10-06T21:41:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.524+0000 7f6d9f5ff700 1 Processor -- start 2023-10-06T21:41:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.524+0000 7f6d9f5ff700 1 -- start start 2023-10-06T21:41:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.524+0000 7f6d9f5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9f5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 0x7f6d980838b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9f5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9810d630 0x7f6d9807c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9f5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9807c9f0 con 0x7f6d98108690 2023-10-06T21:41:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9f5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9807cb60 con 0x7f6d9810d630 2023-10-06T21:41:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9f5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9807cd20 con 0x7f6d98107cf0 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56762/0 (socket says 172.21.15.31:56762) 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 -- 172.21.15.31:0/772202681 learned_addr learned my addr 172.21.15.31:0/772202681 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 0x7f6d980838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9810d630 0x7f6d9807c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 -- 172.21.15.31:0/772202681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9810d630 msgr2=0x7f6d9807c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9810d630 0x7f6d9807c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 -- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 msgr2=0x7f6d980838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.525+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 0x7f6d980838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d9d39b700 1 -- 172.21.15.31:0/772202681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940095f0 con 0x7f6d98107cf0 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005910 tx=0x7f6d9400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d8e7fc700 1 -- 172.21.15.31:0/772202681 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9401d070 con 0x7f6d98107cf0 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d8e7fc700 1 -- 172.21.15.31:0/772202681 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d940054e0 con 0x7f6d98107cf0 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d8e7fc700 1 -- 172.21.15.31:0/772202681 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004a90 con 0x7f6d98107cf0 2023-10-06T21:41:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.526+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9807cfa0 con 0x7f6d98107cf0 2023-10-06T21:41:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.527+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d98081100 con 0x7f6d98107cf0 2023-10-06T21:41:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.532+0000 7f6d8e7fc700 1 -- 172.21.15.31:0/772202681 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6d9400fb80 con 0x7f6d98107cf0 2023-10-06T21:41:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.534+0000 7f6db2f35b80 1 Processor -- start 2023-10-06T21:41:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.534+0000 7f6db2f35b80 1 -- start start 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.534+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b43e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2aa2400 con 0x55e0c2b443d0 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2a93bc0 con 0x55e0c2b39520 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2a914e0 con 0x55e0c2b38c00 2023-10-06T21:41:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d8e7fc700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 0x7f6d840a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d8e7fc700 1 -- 172.21.15.31:0/772202681 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f6d940cdda0 con 0x7f6d98107cf0 2023-10-06T21:41:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d9db9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d9db9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56778/0 (socket says 172.21.15.31:56778) 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2883999463 learned_addr learned my addr 172.21.15.31:0/2883999463 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.535+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 0x7f6d840a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b43e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2883999463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 msgr2=0x55e0c2b43e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b43e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2883999463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 msgr2=0x55e0c2b467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2883999463 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0c28b7d80 con 0x55e0c2b38c00 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.536+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.537+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d90019500 tx=0x7f6d90019810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33df966a68ce3e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.537+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 0x7f6d840a3bb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d90019c10 tx=0x7f6d9001d000 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.537+0000 7f6d837fe700 1 -- 172.21.15.31:0/2883999463 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90025070 con 0x55e0c2b38c00 2023-10-06T21:41:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.537+0000 7f6d837fe700 1 -- 172.21.15.31:0/2883999463 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d90009da0 con 0x55e0c2b38c00 2023-10-06T21:41:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.537+0000 7f6d837fe700 1 -- 172.21.15.31:0/2883999463 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000e040 con 0x55e0c2b38c00 2023-10-06T21:41:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2883999463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 msgr2=0x55e0c2b38fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d90019500 tx=0x7f6d90019810 comp rx=0 tx=0).stop 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2883999463 shutdown_connections 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b38fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2883999463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b43e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2883999463 >> 172.21.15.31:0/2883999463 conn(0x55e0c2aae200 msgr2=0x55e0c2b35080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2883999463 shutdown_connections 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.538+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2883999463 wait complete. 2023-10-06T21:41:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.539+0000 7f6db2f35b80 1 -- start start 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.539+0000 7f6db2f35b80 1 Processor -- start 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b38c00 0x55e0c2b41120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b443d0 0x55e0c2b41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2a8cdd0 con 0x55e0c2b38c00 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2a296b0 con 0x55e0c2b443d0 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2a29350 con 0x55e0c2b39520 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.540+0000 7f6db2f35b80 10 client.0 _ll_register_callbacks cb 0x55e0c28867a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56788/0 (socket says 172.21.15.31:56788) 2023-10-06T21:41:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 -- 172.21.15.31:0/346132612 learned_addr learned my addr 172.21.15.31:0/346132612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b38c00 0x55e0c2b41120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b443d0 0x55e0c2b41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 -- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b443d0 msgr2=0x55e0c2b41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b443d0 0x55e0c2b41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 -- 172.21.15.31:0/346132612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b38c00 msgr2=0x55e0c2b41120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b38c00 0x55e0c2b41120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.541+0000 7f6d9cb9a700 1 -- 172.21.15.31:0/346132612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d90012cc0 con 0x55e0c2b39520 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.542+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d880116d0 tx=0x7f6d880119e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95cee447aca13a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.542+0000 7f6d80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.542+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88016070 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.542+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f6d8800f690 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.542+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0c2a85db0 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8800f850 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 10 client.27412 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e0c28a0c00 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e0c2a85db0 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6d8801aa40 con 0x55e0c2b39520 2023-10-06T21:41:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 10 client.27412 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 10 client.27412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 20 client.27412 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2b53f40 0x55e0c2b56360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 4 client.27412 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.543+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0c2a85ad0 con 0x55e0c2b53f40 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.544+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2b53f40 0x55e0c2b56360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.545+0000 7f6d80ff9700 10 client.27412 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.545+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2b53f40 0x55e0c2b56360 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ffe0132ba1cce4 server_cookie=e145e077e0154c3 in_seq=0 out_seq=0 2023-10-06T21:41:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.545+0000 7f6d80ff9700 1 -- 172.21.15.31:0/346132612 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0c2a85ad0 con 0x55e0c2b53f40 2023-10-06T21:41:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.545+0000 7f6d80ff9700 10 client.27412 handle_command_reply: tid=0 2023-10-06T21:41:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 client.27412 shutdown 2023-10-06T21:41:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 msgr2=0x55e0c2b41660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d880116d0 tx=0x7f6d880119e0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 shutdown_connections 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b443d0 0x55e0c2b41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2b53f40 0x55e0c2b56360 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b38c00 0x55e0c2b41120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/346132612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b39520 0x55e0c2b41660 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 >> 172.21.15.31:0/346132612 conn(0x55e0c2aae200 msgr2=0x55e0c2b34c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6d80ff9700 0 client.27412 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 shutdown_connections 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.546+0000 7f6db2f35b80 1 -- 172.21.15.31:0/346132612 wait complete. 2023-10-06T21:41:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.547+0000 7f6db2f35b80 20 client.27412 trim_cache size 0 max 16384 2023-10-06T21:41:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.570+0000 7f6db2f35b80 1 Processor -- start 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 -- start start 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b38c00 0x55e0c2b3baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b39520 0x55e0c2b408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c290d8c0 con 0x55e0c2b39520 2023-10-06T21:41:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2b2f2f0 con 0x55e0c2b38c00 2023-10-06T21:41:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2b39e20 con 0x55e0c2b443d0 2023-10-06T21:41:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.571+0000 7f6d9db9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56792/0 (socket says 172.21.15.31:56792) 2023-10-06T21:41:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2698131112 learned_addr learned my addr 172.21.15.31:0/2698131112 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b39520 0x55e0c2b408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2698131112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b38c00 msgr2=0x55e0c2b3baa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b38c00 0x55e0c2b3baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2698131112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b39520 msgr2=0x55e0c2b408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b39520 0x55e0c2b408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 -- 172.21.15.31:0/2698131112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0c28de1f0 con 0x55e0c2b443d0 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.572+0000 7f6d9db9c700 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d9001aba0 tx=0x7f6d900098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cae16f16012a291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.573+0000 7f6d83fff700 1 -- 172.21.15.31:0/2698131112 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90025040 con 0x55e0c2b443d0 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.573+0000 7f6d83fff700 1 -- 172.21.15.31:0/2698131112 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d90009da0 con 0x55e0c2b443d0 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.573+0000 7f6d83fff700 1 -- 172.21.15.31:0/2698131112 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000e040 con 0x55e0c2b443d0 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2698131112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 msgr2=0x55e0c2b417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d9001aba0 tx=0x7f6d900098a0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2698131112 shutdown_connections 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b443d0 0x55e0c2b417f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b38c00 0x55e0c2b3baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/2698131112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b39520 0x55e0c2b408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2698131112 >> 172.21.15.31:0/2698131112 conn(0x55e0c2aae200 msgr2=0x55e0c2b410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2698131112 shutdown_connections 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.574+0000 7f6db2f35b80 1 -- 172.21.15.31:0/2698131112 wait complete. 2023-10-06T21:41:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.575+0000 7f6db2f35b80 1 -- start start 2023-10-06T21:41:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.575+0000 7f6db2f35b80 1 Processor -- start 2023-10-06T21:41:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b4ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b48d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2b492f0 con 0x55e0c2b443d0 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2b49460 con 0x55e0c2b39520 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c2b49620 con 0x55e0c2b38c00 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6db2f35b80 10 client.0 _ll_register_callbacks cb 0x7f6d88009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6d9d39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6d9d39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:56794/0 (socket says 172.21.15.31:56794) 2023-10-06T21:41:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6d9d39b700 1 -- 172.21.15.31:0/1923167682 learned_addr learned my addr 172.21.15.31:0/1923167682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.576+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b4ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 -- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 msgr2=0x55e0c2b4ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b4ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 -- 172.21.15.31:0/1923167682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 msgr2=0x55e0c2b48d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b48d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 -- 172.21.15.31:0/1923167682 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0c28b7d80 con 0x55e0c2b38c00 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d9d39b700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d94097780 tx=0x7f6d940cb450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708aab0cfd9ed471 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9409ee60 con 0x55e0c2b38c00 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f6d940a3010 con 0x55e0c2b38c00 2023-10-06T21:41:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.577+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0c28d8f10 con 0x55e0c2b38c00 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d940cb670 con 0x55e0c2b38c00 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6db2f35b80 10 client.27418 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e0c28b6250 con 0x55e0c2b38c00 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d940b7080 con 0x55e0c2b38c00 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6d9409a200 con 0x55e0c2b38c00 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6db2f35b80 10 client.27418 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6db2f35b80 10 client.27418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.578+0000 7f6db2f35b80 20 client.27418 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.579+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2ba8be0 0x55e0c2bab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.579+0000 7f6db2f35b80 4 client.27418 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:41:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.579+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0c290df00 con 0x55e0c2ba8be0 2023-10-06T21:41:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.579+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2ba8be0 0x55e0c2bab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.580+0000 7f6d9cb9a700 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2ba8be0 0x55e0c2bab000 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77dbbb6c1695f3f server_cookie=5c97c1fcec23efb5 in_seq=0 out_seq=0 2023-10-06T21:41:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.580+0000 7f6d80ff9700 10 client.27418 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6d80ff9700 1 -- 172.21.15.31:0/1923167682 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0c290df00 con 0x55e0c2ba8be0 2023-10-06T21:41:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6d80ff9700 10 client.27418 handle_command_reply: tid=0 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 client.27418 shutdown 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 msgr2=0x55e0c2b4a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d94097780 tx=0x7f6d940cb450 comp rx=0 tx=0).stop 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 shutdown_connections 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e0c2ba8be0 0x55e0c2bab000 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c2b443d0 0x55e0c2b48d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0c2b38c00 0x55e0c2b4a860 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 --2- 172.21.15.31:0/1923167682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0c2b39520 0x55e0c2b4ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6d80ff9700 0 client.27418 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.642+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 >> 172.21.15.31:0/1923167682 conn(0x55e0c2aae200 msgr2=0x55e0c2b432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.643+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 shutdown_connections 2023-10-06T21:41:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.643+0000 7f6db2f35b80 1 -- 172.21.15.31:0/1923167682 wait complete. 2023-10-06T21:41:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.643+0000 7f6db2f35b80 20 client.27418 trim_cache size 0 max 16384 2023-10-06T21:41:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:41:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:41:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:41:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:41:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 msgr2=0x7f6d840a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 0x7f6d840a3bb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d90019c10 tx=0x7f6d9001d000 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 msgr2=0x7f6d98083330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005910 tx=0x7f6d9400f690 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 shutdown_connections 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9810d630 0x7f6d9807c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d98107cf0 0x7f6d98083330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6d840a16f0 0x7f6d840a3bb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 --2- 172.21.15.31:0/772202681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d98108690 0x7f6d980838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 >> 172.21.15.31:0/772202681 conn(0x7f6d98076c70 msgr2=0x7f6d9810b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.644+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 shutdown_connections 2023-10-06T21:41:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:10.645+0000 7f6d9f5ff700 1 -- 172.21.15.31:0/772202681 wait complete. 2023-10-06T21:41:10.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50070 6.36 MB/sec execute 299 sec latency 110.400 ms 2023-10-06T21:41:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:11 smithi146 ceph-mon[102809]: pgmap v640: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:11 smithi142 ceph-mon[102951]: pgmap v640: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:11.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:11 smithi031 ceph-mon[98539]: pgmap v640: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:11.777 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50195 6.36 MB/sec execute 300 sec latency 71.448 ms 2023-10-06T21:41:12.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50309 6.34 MB/sec execute 301 sec latency 71.540 ms 2023-10-06T21:41:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:13 smithi146 ceph-mon[102809]: pgmap v641: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:13 smithi142 ceph-mon[102951]: pgmap v641: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:13 smithi031 ceph-mon[98539]: pgmap v641: 97 pgs: 97 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2023-10-06T21:41:13.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50417 6.33 MB/sec execute 302 sec latency 70.018 ms 2023-10-06T21:41:14.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50540 6.34 MB/sec execute 303 sec latency 271.336 ms 2023-10-06T21:41:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:15 smithi031 ceph-mon[98539]: pgmap v642: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2023-10-06T21:41:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:15 smithi031 ceph-mon[98539]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:41:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:15 smithi146 ceph-mon[102809]: pgmap v642: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2023-10-06T21:41:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:15 smithi146 ceph-mon[102809]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:41:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:15 smithi142 ceph-mon[102951]: pgmap v642: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2023-10-06T21:41:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:15 smithi142 ceph-mon[102951]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:41:15.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50663 6.35 MB/sec execute 304 sec latency 74.822 ms 2023-10-06T21:41:16.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50804 6.35 MB/sec execute 305 sec latency 74.923 ms 2023-10-06T21:41:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:17 smithi031 ceph-mon[98539]: pgmap v644: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:17 smithi146 ceph-mon[102809]: pgmap v644: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:17 smithi142 ceph-mon[102951]: pgmap v644: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:17.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 50932 6.35 MB/sec execute 306 sec latency 73.557 ms 2023-10-06T21:41:18.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51084 6.39 MB/sec execute 307 sec latency 72.824 ms 2023-10-06T21:41:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:19 smithi031 ceph-mon[98539]: pgmap v645: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:19 smithi146 ceph-mon[102809]: pgmap v645: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:19.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:19 smithi142 ceph-mon[102951]: pgmap v645: 97 pgs: 97 active+clean; 877 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 229 op/s 2023-10-06T21:41:19.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51205 6.38 MB/sec execute 308 sec latency 105.297 ms 2023-10-06T21:41:20.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51373 6.40 MB/sec execute 309 sec latency 65.775 ms 2023-10-06T21:41:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:21 smithi031 ceph-mon[98539]: pgmap v646: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:21 smithi146 ceph-mon[102809]: pgmap v646: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:21 smithi142 ceph-mon[102951]: pgmap v646: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:21.778 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51474 6.40 MB/sec execute 310 sec latency 70.062 ms 2023-10-06T21:41:22.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51597 6.41 MB/sec execute 311 sec latency 63.700 ms 2023-10-06T21:41:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:23 smithi031 ceph-mon[98539]: pgmap v647: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:23 smithi146 ceph-mon[102809]: pgmap v647: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:23 smithi142 ceph-mon[102951]: pgmap v647: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 220 op/s 2023-10-06T21:41:23.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51729 6.42 MB/sec execute 312 sec latency 67.765 ms 2023-10-06T21:41:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 -- 172.21.15.31:0/3591365560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f8101a80 msgr2=0x7fe9f8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 --2- 172.21.15.31:0/3591365560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f8101a80 0x7fe9f8105ac0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80098f0 tx=0x7fe9e8009c00 comp rx=0 tx=0).stop 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 -- 172.21.15.31:0/3591365560 shutdown_connections 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 --2- 172.21.15.31:0/3591365560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 --2- 172.21.15.31:0/3591365560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f81010d0 0x7fe9f81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 --2- 172.21.15.31:0/3591365560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f8101a80 0x7fe9f8105ac0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 -- 172.21.15.31:0/3591365560 >> 172.21.15.31:0/3591365560 conn(0x7fe9f80fc950 msgr2=0x7fe9f80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.887+0000 7fea003ff700 1 -- 172.21.15.31:0/3591365560 shutdown_connections 2023-10-06T21:41:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.888+0000 7fea003ff700 1 -- 172.21.15.31:0/3591365560 wait complete. 2023-10-06T21:41:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.889+0000 7fea003ff700 1 Processor -- start 2023-10-06T21:41:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.889+0000 7fea003ff700 1 -- start start 2023-10-06T21:41:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 0x7fe9f810a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f810b070 con 0x7fe9f81010d0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f810b1b0 con 0x7fe9f810a3f0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fea003ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f806a8d0 con 0x7fe9f81061f0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fe9fd99a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53182/0 (socket says 172.21.15.31:53182) 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.890+0000 7fe9fe19b700 1 -- 172.21.15.31:0/4282642428 learned_addr learned my addr 172.21.15.31:0/4282642428 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 0x7fe9f810a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 -- 172.21.15.31:0/4282642428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 msgr2=0x7fe9f810a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 0x7fe9f810a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 -- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 msgr2=0x7fe9f810d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 -- 172.21.15.31:0/4282642428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009520 con 0x7fe9f81061f0 2023-10-06T21:41:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 0x7fe9f810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:41:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:41:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.891+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810daf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009230 tx=0x7fe9e800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.892+0000 7fe9ef7fe700 1 -- 172.21.15.31:0/4282642428 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e801c070 con 0x7fe9f81061f0 2023-10-06T21:41:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.892+0000 7fe9ef7fe700 1 -- 172.21.15.31:0/4282642428 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e800f7d0 con 0x7fe9f81061f0 2023-10-06T21:41:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.892+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f806ab50 con 0x7fe9f81061f0 2023-10-06T21:41:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.892+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f806b050 con 0x7fe9f81061f0 2023-10-06T21:41:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.892+0000 7fe9ef7fe700 1 -- 172.21.15.31:0/4282642428 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80038e0 con 0x7fe9f81061f0 2023-10-06T21:41:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.896+0000 7fe9ef7fe700 1 -- 172.21.15.31:0/4282642428 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe9e8020030 con 0x7fe9f81061f0 2023-10-06T21:41:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.898+0000 7fea13c43b80 1 Processor -- start 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.898+0000 7fea13c43b80 1 -- start start 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.898+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f819fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fea13c43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f783400 con 0x564f2f8253d0 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fea13c43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f774bc0 con 0x564f2f819c00 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fea13c43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f7724e0 con 0x564f2f81a520 2023-10-06T21:41:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9ef7fe700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 0x7fe9e40a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9fe99c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f819fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9fd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9fd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53184/0 (socket says 172.21.15.31:53184) 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.899+0000 7fe9fd99a700 1 -- 172.21.15.31:0/2325760831 learned_addr learned my addr 172.21.15.31:0/2325760831 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 0x7fe9e40a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9ef7fe700 1 -- 172.21.15.31:0/4282642428 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe9e80ce0a0 con 0x7fe9f81061f0 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 -- 172.21.15.31:0/2325760831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 msgr2=0x564f2f819fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f819fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 -- 172.21.15.31:0/2325760831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 msgr2=0x564f2f8277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 -- 172.21.15.31:0/2325760831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2f598d80 con 0x564f2f81a520 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824e90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d80 tx=0x7fe9f40116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86400fb816920870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9ecbfe700 1 -- 172.21.15.31:0/2325760831 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4011e20 con 0x564f2f81a520 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.900+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 0x7fe9e40a3b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009440 tx=0x7fe9f0010040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.901+0000 7fe9ecbfe700 1 -- 172.21.15.31:0/2325760831 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9f40044b0 con 0x564f2f81a520 2023-10-06T21:41:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.901+0000 7fe9ecbfe700 1 -- 172.21.15.31:0/2325760831 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4012450 con 0x564f2f81a520 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 -- 172.21.15.31:0/2325760831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 msgr2=0x564f2f824e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824e90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d80 tx=0x7fe9f40116c0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 -- 172.21.15.31:0/2325760831 shutdown_connections 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f819fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 --2- 172.21.15.31:0/2325760831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824e90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 -- 172.21.15.31:0/2325760831 >> 172.21.15.31:0/2325760831 conn(0x564f2f78f200 msgr2=0x564f2f816080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 -- 172.21.15.31:0/2325760831 shutdown_connections 2023-10-06T21:41:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.902+0000 7fea13c43b80 1 -- 172.21.15.31:0/2325760831 wait complete. 2023-10-06T21:41:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.903+0000 7fea13c43b80 1 -- start start 2023-10-06T21:41:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.903+0000 7fea13c43b80 1 Processor -- start 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f824dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f81a520 0x564f2f822050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f76ddd0 con 0x564f2f81a520 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f70a6b0 con 0x564f2f819c00 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f70a350 con 0x564f2f8253d0 2023-10-06T21:41:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fea13c43b80 10 client.0 _ll_register_callbacks cb 0x564f2f5677a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fe99c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f824dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f824dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39370/0 (socket says 172.21.15.31:39370) 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fe99c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39016/0 (socket says 172.21.15.31:39016) 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fe19b700 1 -- 172.21.15.31:0/3221848245 learned_addr learned my addr 172.21.15.31:0/3221848245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.904+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f81a520 0x564f2f822050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 -- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 msgr2=0x564f2f824dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f824dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 -- 172.21.15.31:0/3221848245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f81a520 msgr2=0x564f2f822050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f81a520 0x564f2f822050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 -- 172.21.15.31:0/3221848245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f40099e0 con 0x564f2f8253d0 2023-10-06T21:41:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000a660 tx=0x7fe9f0004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f035244e20f43113 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f001c600 con 0x564f2f8253d0 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe9f001c760 con 0x564f2f8253d0 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.905+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f2f80dd50 con 0x564f2f8253d0 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.906+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f001c920 con 0x564f2f8253d0 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.906+0000 7fea13c43b80 10 client.27427 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.906+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564f2f581c00 con 0x564f2f8253d0 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.906+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f2f80dd50 con 0x564f2f8253d0 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.906+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe9f0026b30 con 0x564f2f8253d0 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 10 client.27427 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 10 client.27427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 20 client.27427 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f834e80 0x564f2f8372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 4 client.27427 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f2f766db0 con 0x564f2f834e80 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.907+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f834e80 0x564f2f8372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.908+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f834e80 0x564f2f8372a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3c29615c9292fa server_cookie=cfe1625e6c80308a in_seq=0 out_seq=0 2023-10-06T21:41:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.908+0000 7fe9e1ffb700 10 client.27427 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3221848245 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f2f766db0 con 0x564f2f834e80 2023-10-06T21:41:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fe9e1ffb700 10 client.27427 handle_command_reply: tid=0 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 client.27427 shutdown 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 msgr2=0x564f2f822660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000a660 tx=0x7fe9f0004b60 comp rx=0 tx=0).stop 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 shutdown_connections 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f822660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f824dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f834e80 0x564f2f8372a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3221848245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f81a520 0x564f2f822050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fe9e1ffb700 0 client.27427 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 >> 172.21.15.31:0/3221848245 conn(0x564f2f78f200 msgr2=0x564f2f815c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 shutdown_connections 2023-10-06T21:41:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.909+0000 7fea13c43b80 1 -- 172.21.15.31:0/3221848245 wait complete. 2023-10-06T21:41:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.910+0000 7fea13c43b80 20 client.27427 trim_cache size 0 max 16384 2023-10-06T21:41:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.938+0000 7fea13c43b80 1 Processor -- start 2023-10-06T21:41:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.938+0000 7fea13c43b80 1 -- start start 2023-10-06T21:41:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f5ee8c0 con 0x564f2f8253d0 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f82c490 con 0x564f2f819c00 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fea13c43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f810200 con 0x564f2f81a520 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fe9fe19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39372/0 (socket says 172.21.15.31:39372) 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fe9fe19b700 1 -- 172.21.15.31:0/540294085 learned_addr learned my addr 172.21.15.31:0/540294085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.939+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 -- 172.21.15.31:0/540294085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 msgr2=0x564f2f824900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 -- 172.21.15.31:0/540294085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 msgr2=0x564f2f8267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 -- 172.21.15.31:0/540294085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2f5bf1f0 con 0x564f2f819c00 2023-10-06T21:41:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4011a10 tx=0x7fe9f4005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28280aea195e65c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9ed3ff700 1 -- 172.21.15.31:0/540294085 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4003680 con 0x564f2f819c00 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.940+0000 7fe9ed3ff700 1 -- 172.21.15.31:0/540294085 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9f40037e0 con 0x564f2f819c00 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.941+0000 7fe9ed3ff700 1 -- 172.21.15.31:0/540294085 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4005040 con 0x564f2f819c00 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.941+0000 7fea13c43b80 1 -- 172.21.15.31:0/540294085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 msgr2=0x564f2f82bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.941+0000 7fea13c43b80 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4011a10 tx=0x7fe9f4005990 comp rx=0 tx=0).stop 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 -- 172.21.15.31:0/540294085 shutdown_connections 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f8253d0 0x564f2f8267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f819c00 0x564f2f82bf10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 --2- 172.21.15.31:0/540294085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f81a520 0x564f2f824900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 -- 172.21.15.31:0/540294085 >> 172.21.15.31:0/540294085 conn(0x564f2f78f200 msgr2=0x564f2f81d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 -- 172.21.15.31:0/540294085 shutdown_connections 2023-10-06T21:41:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 -- 172.21.15.31:0/540294085 wait complete. 2023-10-06T21:41:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.942+0000 7fea13c43b80 1 -- start start 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 Processor -- start 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f819c00 0x564f2f8280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f887ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f8881e0 con 0x564f2f819c00 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f888320 con 0x564f2f81a520 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2f888460 con 0x564f2f8253d0 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.943+0000 7fea13c43b80 10 client.0 _ll_register_callbacks cb 0x7fe9f000e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:39374/0 (socket says 172.21.15.31:39374) 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 -- 172.21.15.31:0/3431724317 learned_addr learned my addr 172.21.15.31:0/3431724317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f819c00 0x564f2f8280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fe99c700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f887ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 -- 172.21.15.31:0/3431724317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 msgr2=0x564f2f887ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f887ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 -- 172.21.15.31:0/3431724317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f819c00 msgr2=0x564f2f8280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f819c00 0x564f2f8280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.944+0000 7fe9fd99a700 1 -- 172.21.15.31:0/3431724317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f401e040 con 0x564f2f81a520 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fe9fd99a700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e809abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb1d542a9895088 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fe9e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:41:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8021380 con 0x564f2f81a520 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe9e809b040 con 0x564f2f81a520 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9f40044b0 con 0x564f2f81a520 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e809f190 con 0x564f2f81a520 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fea13c43b80 10 client.27494 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.945+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564f2f597250 con 0x564f2f81a520 2023-10-06T21:41:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.946+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9f40044b0 con 0x564f2f81a520 2023-10-06T21:41:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.946+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe9e80a10b0 con 0x564f2f81a520 2023-10-06T21:41:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 10 client.27494 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 10 client.27494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 20 client.27494 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f889690 0x564f2f88ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 4 client.27494 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f2f766ad0 con 0x564f2f889690 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.947+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f889690 0x564f2f88ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.948+0000 7fe9fe19b700 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f889690 0x564f2f88ba80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebda39495868efcf server_cookie=665b171233249270 in_seq=0 out_seq=0 2023-10-06T21:41:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.948+0000 7fe9e1ffb700 10 client.27494 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.999+0000 7fe9e1ffb700 1 -- 172.21.15.31:0/3431724317 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f2f766ad0 con 0x564f2f889690 2023-10-06T21:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:23.999+0000 7fe9e1ffb700 10 client.27494 handle_command_reply: tid=0 2023-10-06T21:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 client.27494 shutdown 2023-10-06T21:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 msgr2=0x564f2f828620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e809abd0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 shutdown_connections 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564f2f8253d0 0x564f2f887ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x564f2f889690 0x564f2f88ba80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f2f819c00 0x564f2f8280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 --2- 172.21.15.31:0/3431724317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564f2f81a520 0x564f2f828620 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e809abd0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 >> 172.21.15.31:0/3431724317 conn(0x564f2f78f200 msgr2=0x564f2f81bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.000+0000 7fe9e1ffb700 0 client.27494 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.001+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 shutdown_connections 2023-10-06T21:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.001+0000 7fea13c43b80 1 -- 172.21.15.31:0/3431724317 wait complete. 2023-10-06T21:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.002+0000 7fea13c43b80 20 client.27494 trim_cache size 0 max 16384 2023-10-06T21:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:41:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:41:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:41:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.003+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 msgr2=0x7fe9e40a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.003+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 0x7fe9e40a3b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009440 tx=0x7fe9f0010040 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.003+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 msgr2=0x7fe9f810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.003+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810daf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009230 tx=0x7fe9e800b7e0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 shutdown_connections 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f810a3f0 0x7fe9f810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9e40a16b0 0x7fe9e40a3b70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9f81010d0 0x7fe9f810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 --2- 172.21.15.31:0/4282642428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9f81061f0 0x7fe9f810daf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 >> 172.21.15.31:0/4282642428 conn(0x7fe9f80fc950 msgr2=0x7fe9f80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 shutdown_connections 2023-10-06T21:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:24.004+0000 7fea003ff700 1 -- 172.21.15.31:0/4282642428 wait complete. 2023-10-06T21:41:24.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51848 6.43 MB/sec execute 313 sec latency 78.496 ms 2023-10-06T21:41:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:25 smithi031 ceph-mon[98539]: pgmap v648: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 194 op/s 2023-10-06T21:41:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:25 smithi146 ceph-mon[102809]: pgmap v648: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 194 op/s 2023-10-06T21:41:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:25 smithi142 ceph-mon[102951]: pgmap v648: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 194 op/s 2023-10-06T21:41:25.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 51955 6.43 MB/sec execute 314 sec latency 68.100 ms 2023-10-06T21:41:26.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52046 6.42 MB/sec execute 315 sec latency 67.687 ms 2023-10-06T21:41:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:27 smithi031 ceph-mon[98539]: pgmap v649: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 174 op/s 2023-10-06T21:41:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:27 smithi146 ceph-mon[102809]: pgmap v649: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 174 op/s 2023-10-06T21:41:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:27 smithi142 ceph-mon[102951]: pgmap v649: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 174 op/s 2023-10-06T21:41:27.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52157 6.42 MB/sec execute 316 sec latency 64.291 ms 2023-10-06T21:41:28.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52252 6.41 MB/sec execute 317 sec latency 71.558 ms 2023-10-06T21:41:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:29 smithi031 ceph-mon[98539]: pgmap v650: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 161 op/s 2023-10-06T21:41:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:29 smithi146 ceph-mon[102809]: pgmap v650: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 161 op/s 2023-10-06T21:41:29.757 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:29 smithi142 ceph-mon[102951]: pgmap v650: 97 pgs: 97 active+clean; 908 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 161 op/s 2023-10-06T21:41:29.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52369 6.41 MB/sec execute 318 sec latency 75.130 ms 2023-10-06T21:41:30.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52483 6.40 MB/sec execute 319 sec latency 86.072 ms 2023-10-06T21:41:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:31 smithi031 ceph-mon[98539]: pgmap v651: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 258 op/s 2023-10-06T21:41:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:31 smithi146 ceph-mon[102809]: pgmap v651: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 258 op/s 2023-10-06T21:41:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:31 smithi142 ceph-mon[102951]: pgmap v651: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 258 op/s 2023-10-06T21:41:31.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52602 6.39 MB/sec execute 320 sec latency 74.645 ms 2023-10-06T21:41:32.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52712 6.38 MB/sec execute 321 sec latency 73.105 ms 2023-10-06T21:41:33.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:33 smithi031 ceph-mon[98539]: pgmap v652: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 175 op/s 2023-10-06T21:41:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:33 smithi146 ceph-mon[102809]: pgmap v652: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 175 op/s 2023-10-06T21:41:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:33 smithi142 ceph-mon[102951]: pgmap v652: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 175 op/s 2023-10-06T21:41:33.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52820 6.38 MB/sec execute 322 sec latency 80.771 ms 2023-10-06T21:41:34.779 INFO:tasks.workunit.client.0.smithi031.stdout: 10 52941 6.37 MB/sec execute 323 sec latency 83.624 ms 2023-10-06T21:41:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:35 smithi031 ceph-mon[98539]: pgmap v653: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2023-10-06T21:41:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:35 smithi146 ceph-mon[102809]: pgmap v653: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2023-10-06T21:41:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:35 smithi142 ceph-mon[102951]: pgmap v653: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2023-10-06T21:41:35.780 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53077 6.38 MB/sec execute 324 sec latency 108.754 ms 2023-10-06T21:41:36.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53182 6.39 MB/sec execute 325 sec latency 175.221 ms 2023-10-06T21:41:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:37 smithi031 ceph-mon[98539]: pgmap v654: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:37 smithi146 ceph-mon[102809]: pgmap v654: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:37 smithi142 ceph-mon[102951]: pgmap v654: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:37.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53270 6.37 MB/sec execute 326 sec latency 326.933 ms 2023-10-06T21:41:38.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53391 6.38 MB/sec execute 327 sec latency 75.239 ms 2023-10-06T21:41:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:39 smithi031 ceph-mon[98539]: pgmap v655: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:39.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:39 smithi146 ceph-mon[102809]: pgmap v655: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:39 smithi142 ceph-mon[102951]: pgmap v655: 97 pgs: 97 active+clean; 915 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-10-06T21:41:39.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53477 6.37 MB/sec execute 328 sec latency 300.463 ms 2023-10-06T21:41:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.635+0000 7f1463fff700 1 -- 172.21.15.31:0/264819351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c1087d0 msgr2=0x7f145c113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.635+0000 7f1463fff700 1 --2- 172.21.15.31:0/264819351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c1087d0 0x7f145c113ac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1450009f90 tx=0x7f145000b330 comp rx=0 tx=0).stop 2023-10-06T21:41:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 -- 172.21.15.31:0/264819351 shutdown_connections 2023-10-06T21:41:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 --2- 172.21.15.31:0/264819351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c114000 0x7f145c1163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 --2- 172.21.15.31:0/264819351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 --2- 172.21.15.31:0/264819351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c1087d0 0x7f145c113ac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 -- 172.21.15.31:0/264819351 >> 172.21.15.31:0/264819351 conn(0x7f145c06d3e0 msgr2=0x7f145c06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 -- 172.21.15.31:0/264819351 shutdown_connections 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.636+0000 7f1463fff700 1 -- 172.21.15.31:0/264819351 wait complete. 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.637+0000 7f1463fff700 1 Processor -- start 2023-10-06T21:41:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.637+0000 7f1463fff700 1 -- start start 2023-10-06T21:41:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c117d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 0x7f145c1b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c118b00 con 0x7f145c107eb0 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1b2540 con 0x7f145c114000 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1463fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1b2680 con 0x7f145c1087d0 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c117d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f146159a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f146259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 0x7f145c1b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f146159a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35484/0 (socket says 172.21.15.31:35484) 2023-10-06T21:41:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.638+0000 7f146159a700 1 -- 172.21.15.31:0/505489080 learned_addr learned my addr 172.21.15.31:0/505489080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 -- 172.21.15.31:0/505489080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 msgr2=0x7f145c1182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 -- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 msgr2=0x7f145c117d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c117d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 -- 172.21.15.31:0/505489080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450009c40 con 0x7f145c114000 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146159a700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:41:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f1461d9b700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c117d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.639+0000 7f146259c700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 0x7f145c1b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1454005fd0 tx=0x7f145400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.640+0000 7f145affd700 1 -- 172.21.15.31:0/505489080 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454004220 con 0x7f145c114000 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.640+0000 7f145affd700 1 -- 172.21.15.31:0/505489080 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1454004cc0 con 0x7f145c114000 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.640+0000 7f145affd700 1 -- 172.21.15.31:0/505489080 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454005660 con 0x7f145c114000 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.640+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c1b28e0 con 0x7f145c114000 2023-10-06T21:41:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.640+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1b2da0 con 0x7f145c114000 2023-10-06T21:41:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.645+0000 7f145affd700 1 -- 172.21.15.31:0/505489080 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1454004750 con 0x7f145c114000 2023-10-06T21:41:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.647+0000 7f1477957b80 1 Processor -- start 2023-10-06T21:41:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.647+0000 7f1477957b80 1 -- start start 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b196781fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 0x55b19678ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b19678f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1966eb400 con 0x55b196782520 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1966dcbc0 con 0x55b196781c00 2023-10-06T21:41:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f1477957b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1966da4e0 con 0x55b19678d3d0 2023-10-06T21:41:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f145affd700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 0x7f14440a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.648+0000 7f146159a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b196781fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 0x55b19678ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f146159a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b196781fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42470/0 (socket says 172.21.15.31:42470) 2023-10-06T21:41:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f146159a700 1 -- 172.21.15.31:0/844656925 learned_addr learned my addr 172.21.15.31:0/844656925 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f145affd700 1 -- 172.21.15.31:0/505489080 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1454014070 con 0x7f145c114000 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f146159a700 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b19678f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 -- 172.21.15.31:0/844656925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 msgr2=0x55b19678f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b19678f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 -- 172.21.15.31:0/844656925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 msgr2=0x55b196781fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b196781fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f1461d9b700 1 -- 172.21.15.31:0/844656925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b196500d80 con 0x55b196782520 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.649+0000 7f146259c700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 0x7f14440a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.650+0000 7f1461d9b700 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 0x55b19678ce90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f144c0114f0 tx=0x7f144c011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d032daadb49d64b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.650+0000 7f1443fff700 1 -- 172.21.15.31:0/844656925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c0112f0 con 0x55b196782520 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.650+0000 7f146259c700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 0x7f14440a3c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f14540046a0 tx=0x7f1454004570 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.650+0000 7f1443fff700 1 -- 172.21.15.31:0/844656925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f144c0044c0 con 0x55b196782520 2023-10-06T21:41:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.650+0000 7f1443fff700 1 -- 172.21.15.31:0/844656925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c01a480 con 0x55b196782520 2023-10-06T21:41:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.651+0000 7f1477957b80 1 -- 172.21.15.31:0/844656925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 msgr2=0x55b19678ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.651+0000 7f1477957b80 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 0x55b19678ce90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f144c0114f0 tx=0x7f144c011800 comp rx=0 tx=0).stop 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 -- 172.21.15.31:0/844656925 shutdown_connections 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b19678f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b196781fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 --2- 172.21.15.31:0/844656925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196782520 0x55b19678ce90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 -- 172.21.15.31:0/844656925 >> 172.21.15.31:0/844656925 conn(0x55b1966f7200 msgr2=0x55b19677e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.653+0000 7f1477957b80 1 -- 172.21.15.31:0/844656925 shutdown_connections 2023-10-06T21:41:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.654+0000 7f1477957b80 1 -- 172.21.15.31:0/844656925 wait complete. 2023-10-06T21:41:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.654+0000 7f1477957b80 1 -- start start 2023-10-06T21:41:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.655+0000 7f1477957b80 1 Processor -- start 2023-10-06T21:41:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.655+0000 7f1477957b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b19678a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b19678ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1966d5dd0 con 0x55b19678d3d0 2023-10-06T21:41:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1966726b0 con 0x55b196781c00 2023-10-06T21:41:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b196672350 con 0x55b196782520 2023-10-06T21:41:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1477957b80 10 client.0 _ll_register_callbacks cb 0x55b1964cf7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35496/0 (socket says 172.21.15.31:35496) 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f1461d9b700 1 -- 172.21.15.31:0/3012571225 learned_addr learned my addr 172.21.15.31:0/3012571225 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.656+0000 7f146159a700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b19678a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b19678ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 -- 172.21.15.31:0/3012571225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 msgr2=0x55b19678a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 -- 172.21.15.31:0/3012571225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 msgr2=0x55b19678ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b19678ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 -- 172.21.15.31:0/3012571225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c009c90 con 0x55b196781c00 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:41:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.657+0000 7f146159a700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b19678a290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1450009f30 tx=0x7f1450009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdabc0b3b7449fc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.658+0000 7f14417fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.658+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000e040 con 0x55b196781c00 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.658+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1450020860 con 0x55b196781c00 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.658+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b196775d50 con 0x55b196781c00 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1450020a20 con 0x55b196781c00 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f1477957b80 10 client.27506 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b1964e9c00 con 0x55b196781c00 2023-10-06T21:41:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b196775d50 con 0x55b196781c00 2023-10-06T21:41:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1450019070 con 0x55b196781c00 2023-10-06T21:41:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f1477957b80 10 client.27506 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.659+0000 7f1477957b80 10 client.27506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.660+0000 7f1477957b80 20 client.27506 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.660+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b19679cfa0 0x55b19679f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.660+0000 7f1477957b80 4 client.27506 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.660+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1966cedb0 con 0x55b19679cfa0 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.660+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b19679cfa0 0x55b19679f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.661+0000 7f14417fa700 10 client.27506 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.661+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b19679cfa0 0x55b19679f3c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc2289160e623d server_cookie=b3b7081e1c127834 in_seq=0 out_seq=0 2023-10-06T21:41:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.662+0000 7f14417fa700 1 -- 172.21.15.31:0/3012571225 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b1966cedb0 con 0x55b19679cfa0 2023-10-06T21:41:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.662+0000 7f14417fa700 10 client.27506 handle_command_reply: tid=0 2023-10-06T21:41:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.662+0000 7f1477957b80 1 client.27506 shutdown 2023-10-06T21:41:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.662+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 msgr2=0x55b19678a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.662+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b19678a290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1450009f30 tx=0x7f1450009440 comp rx=0 tx=0).stop 2023-10-06T21:41:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 shutdown_connections 2023-10-06T21:41:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b19679cfa0 0x55b19679f3c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b19678ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f14417fa700 0 client.27506 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196781c00 0x55b19678a290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 --2- 172.21.15.31:0/3012571225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196782520 0x55b19678a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 >> 172.21.15.31:0/3012571225 conn(0x55b1966f7200 msgr2=0x55b19677dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 shutdown_connections 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.663+0000 7f1477957b80 1 -- 172.21.15.31:0/3012571225 wait complete. 2023-10-06T21:41:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.664+0000 7f1477957b80 20 client.27506 trim_cache size 0 max 16384 2023-10-06T21:41:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.687+0000 7f1477957b80 1 Processor -- start 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.687+0000 7f1477957b80 1 -- start start 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 0x55b196793f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b196787520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1965568c0 con 0x55b19678d3d0 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b196783740 con 0x55b196782520 2023-10-06T21:41:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1477957b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19677a860 con 0x55b196781c00 2023-10-06T21:41:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:42486/0 (socket says 172.21.15.31:42486) 2023-10-06T21:41:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.688+0000 7f1461d9b700 1 -- 172.21.15.31:0/3118882313 learned_addr learned my addr 172.21.15.31:0/3118882313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.689+0000 7f146159a700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 0x55b196793f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.689+0000 7f146159a700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b196787520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 -- 172.21.15.31:0/3118882313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 msgr2=0x55b196793f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 0x55b196793f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 -- 172.21.15.31:0/3118882313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 msgr2=0x55b196787520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b196787520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 -- 172.21.15.31:0/3118882313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1965271f0 con 0x55b196782520 2023-10-06T21:41:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f146159a700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 0x55b196793f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:41:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.690+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f144c005950 tx=0x7f144c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bbebf6c41194ffb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.691+0000 7f1458ff9700 1 -- 172.21.15.31:0/3118882313 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c0112f0 con 0x55b196782520 2023-10-06T21:41:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.691+0000 7f1458ff9700 1 -- 172.21.15.31:0/3118882313 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f144c020040 con 0x55b196782520 2023-10-06T21:41:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.691+0000 7f1458ff9700 1 -- 172.21.15.31:0/3118882313 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c028570 con 0x55b196782520 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- 172.21.15.31:0/3118882313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 msgr2=0x55b196784aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f144c005950 tx=0x7f144c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- 172.21.15.31:0/3118882313 shutdown_connections 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b19678d3d0 0x55b196787520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b196781c00 0x55b196793f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 --2- 172.21.15.31:0/3118882313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b196782520 0x55b196784aa0 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f144c005950 tx=0x7f144c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- 172.21.15.31:0/3118882313 >> 172.21.15.31:0/3118882313 conn(0x55b1966f7200 msgr2=0x55b196785590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- 172.21.15.31:0/3118882313 shutdown_connections 2023-10-06T21:41:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- 172.21.15.31:0/3118882313 wait complete. 2023-10-06T21:41:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.692+0000 7f1477957b80 1 -- start start 2023-10-06T21:41:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 Processor -- start 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196781c00 0x55b196792110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1967e9d60 0x55b1967ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b196792db0 con 0x55b196781c00 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1967ea750 con 0x55b1967e9d60 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1967ea910 con 0x55b19678d3d0 2023-10-06T21:41:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1477957b80 10 client.0 _ll_register_callbacks cb 0x7f1450008f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1461d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:35516/0 (socket says 172.21.15.31:35516) 2023-10-06T21:41:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.693+0000 7f1461d9b700 1 -- 172.21.15.31:0/3583453647 learned_addr learned my addr 172.21.15.31:0/3583453647 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 -- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1967e9d60 msgr2=0x55b1967ea1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:41:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1967e9d60 0x55b1967ea1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 -- 172.21.15.31:0/3583453647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196781c00 msgr2=0x55b196792110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196781c00 0x55b196792110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 -- 172.21.15.31:0/3583453647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1964e9c00 con 0x55b19678d3d0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f1461d9b700 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f144c005950 tx=0x7f144c005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638264e07d54494 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f14417fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.694+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c028c10 con 0x55b19678d3d0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f144c028d70 con 0x55b19678d3d0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f144c0044c0 con 0x55b19678d3d0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f144c028070 con 0x55b19678d3d0 2023-10-06T21:41:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f1477957b80 10 client.27451 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b196500d80 con 0x55b19678d3d0 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f144c0044c0 con 0x55b19678d3d0 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.695+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f144c005560 con 0x55b19678d3d0 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 10 client.27451 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 10 client.27451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 20 client.27451 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b1967f1d30 0x55b1967f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 4 client.27451 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b1966cead0 con 0x55b1967f1d30 2023-10-06T21:41:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.696+0000 7f146159a700 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b1967f1d30 0x55b1967f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.697+0000 7f14417fa700 10 client.27451 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.697+0000 7f146159a700 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b1967f1d30 0x55b1967f4150 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd4f7b5a6553f01 server_cookie=e7b5e5418ee8893f in_seq=0 out_seq=0 2023-10-06T21:41:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f14417fa700 1 -- 172.21.15.31:0/3583453647 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b1966cead0 con 0x55b1967f1d30 2023-10-06T21:41:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f14417fa700 10 client.27451 handle_command_reply: tid=0 2023-10-06T21:41:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 client.27451 shutdown 2023-10-06T21:41:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 msgr2=0x55b196792650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f144c005950 tx=0x7f144c005e20 comp rx=0 tx=0).stop 2023-10-06T21:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 shutdown_connections 2023-10-06T21:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1967e9d60 0x55b1967ea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b1967f1d30 0x55b1967f4150 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b196781c00 0x55b196792110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 --2- 172.21.15.31:0/3583453647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b19678d3d0 0x55b196792650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f14417fa700 0 client.27451 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.759+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 >> 172.21.15.31:0/3583453647 conn(0x55b1966f7200 msgr2=0x55b19677d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.760+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 shutdown_connections 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.760+0000 7f1477957b80 1 -- 172.21.15.31:0/3583453647 wait complete. 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.760+0000 7f1477957b80 20 client.27451 trim_cache size 0 max 16384 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:41:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:41:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:41:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 msgr2=0x7f14440a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 0x7f14440a3c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f14540046a0 tx=0x7f1454004570 comp rx=0 tx=0).stop 2023-10-06T21:41:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 msgr2=0x7f145c1b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 0x7f145c1b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1454005fd0 tx=0x7f145400ec20 comp rx=0 tx=0).stop 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 shutdown_connections 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c114000 0x7f145c1b2000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f14440a17c0 0x7f14440a3c80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c107eb0 0x7f145c117d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 --2- 172.21.15.31:0/505489080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f145c1087d0 0x7f145c1182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 >> 172.21.15.31:0/505489080 conn(0x7f145c06d3e0 msgr2=0x7f145c06f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 shutdown_connections 2023-10-06T21:41:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:40.761+0000 7f1463fff700 1 -- 172.21.15.31:0/505489080 wait complete. 2023-10-06T21:41:40.859 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53597 6.37 MB/sec execute 329 sec latency 111.100 ms 2023-10-06T21:41:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:41 smithi031 ceph-mon[98539]: pgmap v656: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 298 op/s 2023-10-06T21:41:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:41 smithi146 ceph-mon[102809]: pgmap v656: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 298 op/s 2023-10-06T21:41:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:41 smithi142 ceph-mon[102951]: pgmap v656: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 298 op/s 2023-10-06T21:41:41.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53709 6.35 MB/sec execute 330 sec latency 138.938 ms 2023-10-06T21:41:42.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53819 6.34 MB/sec execute 331 sec latency 247.758 ms 2023-10-06T21:41:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:43 smithi031 ceph-mon[98539]: pgmap v657: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 201 op/s 2023-10-06T21:41:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:43 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:41:43.568 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:43 smithi146 ceph-mon[102809]: pgmap v657: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 201 op/s 2023-10-06T21:41:43.568 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:43 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:41:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:43 smithi142 ceph-mon[102951]: pgmap v657: 97 pgs: 97 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 201 op/s 2023-10-06T21:41:43.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:43 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:41:43.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 53932 6.34 MB/sec execute 332 sec latency 91.521 ms 2023-10-06T21:41:44.782 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54038 6.33 MB/sec execute 333 sec latency 178.769 ms 2023-10-06T21:41:45.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:45 smithi031 ceph-mon[98539]: pgmap v658: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2023-10-06T21:41:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:45 smithi146 ceph-mon[102809]: pgmap v658: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2023-10-06T21:41:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:45 smithi142 ceph-mon[102951]: pgmap v658: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2023-10-06T21:41:45.781 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54118 6.32 MB/sec execute 334 sec latency 365.227 ms 2023-10-06T21:41:46.782 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54266 6.33 MB/sec execute 335 sec latency 74.932 ms 2023-10-06T21:41:47.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:47 smithi031 ceph-mon[98539]: pgmap v659: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:47.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:47 smithi146 ceph-mon[102809]: pgmap v659: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:47 smithi142 ceph-mon[102951]: pgmap v659: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:47.782 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54388 6.34 MB/sec execute 336 sec latency 81.717 ms 2023-10-06T21:41:48.782 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54507 6.35 MB/sec execute 337 sec latency 77.997 ms 2023-10-06T21:41:49.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:49 smithi031 ceph-mon[98539]: pgmap v660: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:49.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:49 smithi146 ceph-mon[102809]: pgmap v660: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:49 smithi142 ceph-mon[102951]: pgmap v660: 97 pgs: 97 active+clean; 922 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-10-06T21:41:49.782 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54645 6.36 MB/sec execute 338 sec latency 83.972 ms 2023-10-06T21:41:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:50 smithi142 ceph-mon[102951]: pgmap v661: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2023-10-06T21:41:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:50 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:50 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:50 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:50 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:50 smithi031 ceph-mon[98539]: pgmap v661: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2023-10-06T21:41:50.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:50 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:50 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:50 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:50 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54780 6.38 MB/sec execute 339 sec latency 90.669 ms 2023-10-06T21:41:50.801 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:50 smithi146 ceph-mon[102809]: pgmap v661: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2023-10-06T21:41:50.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:50 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:50 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:50 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:50.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:50 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:41:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:51 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:41:51.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 54902 6.38 MB/sec execute 340 sec latency 68.019 ms 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:41:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:51 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:41:51.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:51.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi031 to 222.0M 2023-10-06T21:41:51.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:51 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi031 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:41:52.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:52 smithi031 ceph-mon[98539]: pgmap v662: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2023-10-06T21:41:52.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55046 6.39 MB/sec execute 341 sec latency 66.780 ms 2023-10-06T21:41:52.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:52 smithi146 ceph-mon[102809]: pgmap v662: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2023-10-06T21:41:52.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:52 smithi142 ceph-mon[102951]: pgmap v662: 97 pgs: 97 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2023-10-06T21:41:53.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55156 6.40 MB/sec execute 342 sec latency 278.031 ms 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.992+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/4228393754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 msgr2=0x7f0f6810dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.992+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/4228393754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6810dcc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c008570 tx=0x7f0f5c00acb0 comp rx=0 tx=0).stop 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/4228393754 shutdown_connections 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/4228393754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68108870 0x7f0f68114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/4228393754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6810dcc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/4228393754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68107eb0 0x7f0f68108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/4228393754 >> 172.21.15.31:0/4228393754 conn(0x7f0f6806d3e0 msgr2=0x7f0f6806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/4228393754 shutdown_connections 2023-10-06T21:41:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.993+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/4228393754 wait complete. 2023-10-06T21:41:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.994+0000 7f0f6f1ff700 1 Processor -- start 2023-10-06T21:41:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 -- start start 2023-10-06T21:41:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 0x7f0f681a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6819d9f0 con 0x7f0f68107eb0 2023-10-06T21:41:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6819db60 con 0x7f0f68108870 2023-10-06T21:41:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.995+0000 7f0f6f1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6819dd20 con 0x7f0f6810d8e0 2023-10-06T21:41:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f67fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 0x7f0f681a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48390/0 (socket says 172.21.15.31:48390) 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 -- 172.21.15.31:0/3943241213 learned_addr learned my addr 172.21.15.31:0/3943241213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f67fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 0x7f0f681a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:58208/0 (socket says 172.21.15.31:58208) 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 -- 172.21.15.31:0/3943241213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 msgr2=0x7f0f6819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 -- 172.21.15.31:0/3943241213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 msgr2=0x7f0f681a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 0x7f0f681a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.996+0000 7f0f6cf9b700 1 -- 172.21.15.31:0/3943241213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6000be50 con 0x7f0f68107eb0 2023-10-06T21:41:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f6cf9b700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0084c0 tx=0x7f0f5c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f65ffb700 1 -- 172.21.15.31:0/3943241213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c003a00 con 0x7f0f68107eb0 2023-10-06T21:41:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c0081d0 con 0x7f0f68107eb0 2023-10-06T21:41:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f65ffb700 1 -- 172.21.15.31:0/3943241213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f5c003b60 con 0x7f0f68107eb0 2023-10-06T21:41:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f65ffb700 1 -- 172.21.15.31:0/3943241213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c01e6b0 con 0x7f0f68107eb0 2023-10-06T21:41:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:53.997+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f681aa330 con 0x7f0f68107eb0 2023-10-06T21:41:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.002+0000 7f0f65ffb700 1 -- 172.21.15.31:0/3943241213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0f5c00a0d0 con 0x7f0f68107eb0 2023-10-06T21:41:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.003+0000 7f0f82a37b80 1 Processor -- start 2023-10-06T21:41:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.004+0000 7f0f82a37b80 1 -- start start 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accefe57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55acceff06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55acceff2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accef46400 con 0x55accefe5d30 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accef37bc0 con 0x55acceff0be0 2023-10-06T21:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.005+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accef354e0 con 0x55accefe5410 2023-10-06T21:41:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55acceff06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55acceff06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48406/0 (socket says 172.21.15.31:48406) 2023-10-06T21:41:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6cf9b700 1 -- 172.21.15.31:0/3585473410 learned_addr learned my addr 172.21.15.31:0/3585473410 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accefe57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f67fff700 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55acceff2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 -- 172.21.15.31:0/3585473410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 msgr2=0x55acceff2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55acceff2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 -- 172.21.15.31:0/3585473410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 msgr2=0x55acceff06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55acceff06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f6d79c700 1 -- 172.21.15.31:0/3585473410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acced5bd80 con 0x55accefe5410 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.006+0000 7f0f65ffb700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 0x7f0f580a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f65ffb700 1 -- 172.21.15.31:0/3943241213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0f5c0cdc30 con 0x7f0f68107eb0 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accefe57f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f681a4220 tx=0x7f0f6000b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c3945333715dbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f52ffd700 1 -- 172.21.15.31:0/3585473410 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60010040 con 0x55accefe5410 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f6cf9b700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 0x7f0f580a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f52ffd700 1 -- 172.21.15.31:0/3585473410 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f60009d80 con 0x55accefe5410 2023-10-06T21:41:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.007+0000 7f0f52ffd700 1 -- 172.21.15.31:0/3585473410 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60018800 con 0x55accefe5410 2023-10-06T21:41:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3585473410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 msgr2=0x55accefe57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accefe57f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f681a4220 tx=0x7f0f6000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3585473410 shutdown_connections 2023-10-06T21:41:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55acceff2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accefe57f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3585473410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55acceff06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3585473410 >> 172.21.15.31:0/3585473410 conn(0x55accef52200 msgr2=0x55accefd8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f6cf9b700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 0x7f0f580a3b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0a29a0 tx=0x7f0f5c00a040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3585473410 shutdown_connections 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.008+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3585473410 wait complete. 2023-10-06T21:41:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.009+0000 7f0f82a37b80 1 -- start start 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.010+0000 7f0f82a37b80 1 Processor -- start 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.010+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.010+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55accefed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.010+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55accefedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accef30dd0 con 0x55accefe5d30 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acceecd6b0 con 0x55acceff0be0 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acceecd350 con 0x55accefe5410 2023-10-06T21:41:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f82a37b80 10 client.0 _ll_register_callbacks cb 0x55acced2a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57802/0 (socket says 172.21.15.31:57802) 2023-10-06T21:41:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 -- 172.21.15.31:0/285223580 learned_addr learned my addr 172.21.15.31:0/285223580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55accefedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f67fff700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55accefed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 -- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 msgr2=0x55accefedf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55accefedf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.011+0000 7f0f6d79c700 1 -- 172.21.15.31:0/285223580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 msgr2=0x55accefed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55accefed890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f6d79c700 1 -- 172.21.15.31:0/285223580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f60003b90 con 0x55accefe5410 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f600095a0 tx=0x7f0f60006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a226374f6d5c1d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f47fff700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60010070 con 0x55accefe5410 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0f6002c030 con 0x55accefe5410 2023-10-06T21:41:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.012+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55accefd0d50 con 0x55accefe5410 2023-10-06T21:41:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f60018480 con 0x55accefe5410 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f82a37b80 10 client.27463 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55acced44c00 con 0x55accefe5410 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55accefd0d50 con 0x55accefe5410 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f600085b0 con 0x55accefe5410 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f82a37b80 10 client.27463 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.013+0000 7f0f82a37b80 10 client.27463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.014+0000 7f0f82a37b80 20 client.27463 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.014+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0006e0 0x55accf002b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.014+0000 7f0f82a37b80 4 client.27463 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.014+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55accef29db0 con 0x55accf0006e0 2023-10-06T21:41:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.014+0000 7f0f67fff700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0006e0 0x55accf002b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.015+0000 7f0f47fff700 10 client.27463 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.015+0000 7f0f67fff700 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0006e0 0x55accf002b00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c25e79dc87b8eab server_cookie=5410f815fbdc5530 in_seq=0 out_seq=0 2023-10-06T21:41:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.016+0000 7f0f47fff700 1 -- 172.21.15.31:0/285223580 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55accef29db0 con 0x55accf0006e0 2023-10-06T21:41:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.016+0000 7f0f47fff700 10 client.27463 handle_command_reply: tid=0 2023-10-06T21:41:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.016+0000 7f0f82a37b80 1 client.27463 shutdown 2023-10-06T21:41:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 msgr2=0x55acceff05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f600095a0 tx=0x7f0f60006040 comp rx=0 tx=0).stop 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 shutdown_connections 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acceff0be0 0x55accefedf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55acceff05a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0006e0 0x55accf002b00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/285223580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55accefe5d30 0x55accefed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 >> 172.21.15.31:0/285223580 conn(0x55accef52200 msgr2=0x55accefd8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.017+0000 7f0f47fff700 0 client.27463 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.018+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 shutdown_connections 2023-10-06T21:41:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.018+0000 7f0f82a37b80 1 -- 172.21.15.31:0/285223580 wait complete. 2023-10-06T21:41:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.018+0000 7f0f82a37b80 20 client.27463 trim_cache size 0 max 16384 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.037+0000 7f0f82a37b80 1 Processor -- start 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.037+0000 7f0f82a37b80 1 -- start start 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5410 0x55acceff0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55accefed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accedb18c0 con 0x55acceff0be0 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accefda430 con 0x55accefe5410 2023-10-06T21:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accefd0bb0 con 0x55accefe5d30 2023-10-06T21:41:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57808/0 (socket says 172.21.15.31:57808) 2023-10-06T21:41:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f67fff700 1 -- 172.21.15.31:0/3285411401 learned_addr learned my addr 172.21.15.31:0/3285411401 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.038+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5410 0x55acceff0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55accefed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 -- 172.21.15.31:0/3285411401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5410 msgr2=0x55acceff0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5410 0x55acceff0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 -- 172.21.15.31:0/3285411401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 msgr2=0x55accefed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55accefed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 -- 172.21.15.31:0/3285411401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acced821f0 con 0x55accefe5d30 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f67fff700 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f54012010 tx=0x7f0f54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bfdd7503ca9162 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f537fe700 1 -- 172.21.15.31:0/3285411401 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54015040 con 0x55accefe5d30 2023-10-06T21:41:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f537fe700 1 -- 172.21.15.31:0/3285411401 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f54016040 con 0x55accefe5d30 2023-10-06T21:41:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.039+0000 7f0f537fe700 1 -- 172.21.15.31:0/3285411401 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5401f5b0 con 0x55accefe5d30 2023-10-06T21:41:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3285411401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 msgr2=0x55accefed050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f54012010 tx=0x7f0f54009260 comp rx=0 tx=0).stop 2023-10-06T21:41:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3285411401 shutdown_connections 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55accefed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5410 0x55acceff0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/3285411401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5d30 0x55accefed050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.040+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3285411401 >> 172.21.15.31:0/3285411401 conn(0x55accef52200 msgr2=0x55accefe8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.041+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3285411401 shutdown_connections 2023-10-06T21:41:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.041+0000 7f0f82a37b80 1 -- 172.21.15.31:0/3285411401 wait complete. 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.041+0000 7f0f82a37b80 1 -- start start 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 Processor -- start 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accf04d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 0x55accf04dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55acceff3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acceff3e20 con 0x55acceff0be0 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acceff3f90 con 0x55accefe5d30 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.042+0000 7f0f82a37b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acceff4150 con 0x55accefe5410 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f82a37b80 10 client.0 _ll_register_callbacks cb 0x7f0f60004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f6d79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accf04d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 0x55accf04dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f6d79c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accf04d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57820/0 (socket says 172.21.15.31:57820) 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f6d79c700 1 -- 172.21.15.31:0/1470231447 learned_addr learned my addr 172.21.15.31:0/1470231447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:41:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55acceff3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 -- 172.21.15.31:0/1470231447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 msgr2=0x55accf04d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accf04d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 -- 172.21.15.31:0/1470231447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 msgr2=0x55acceff3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55acceff3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.043+0000 7f0f67fff700 1 -- 172.21.15.31:0/1470231447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f54010a00 con 0x55accefe5d30 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.044+0000 7f0f67fff700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 0x55accf04dd80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55accefe6750 tx=0x7f0f54009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a921184652ff90e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.044+0000 7f0f47fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.044+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54015050 con 0x55accefe5d30 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f60009d80 con 0x55accefe5d30 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0f54004050 con 0x55accefe5d30 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f54004210 con 0x55accefe5d30 2023-10-06T21:41:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f82a37b80 10 client.27527 fetch_fsmap learned FSMap version 49 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55acced5a250 con 0x55accefe5d30 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f5403c020 con 0x55accefe5d30 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.045+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f54003830 con 0x55accefe5d30 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 10 client.27527 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 10 client.27527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 20 client.27527 populate_metadata read hostname 'smithi031' 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0555b0 0x55accf0579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 4 client.27527 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:41:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55accef29ad0 con 0x55accf0555b0 2023-10-06T21:41:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.046+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0555b0 0x55accf0579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:41:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.047+0000 7f0f6d79c700 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0555b0 0x55accf0579d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817755157d0a65d server_cookie=c1bfc82c24e1b7f2 in_seq=0 out_seq=0 2023-10-06T21:41:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.047+0000 7f0f47fff700 10 client.27527 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.098+0000 7f0f47fff700 1 -- 172.21.15.31:0/1470231447 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55accef29ad0 con 0x55accf0555b0 2023-10-06T21:41:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.098+0000 7f0f47fff700 10 client.27527 handle_command_reply: tid=0 2023-10-06T21:41:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.098+0000 7f0f82a37b80 1 client.27527 shutdown 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 msgr2=0x55accf04dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 0x55accf04dd80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55accefe6750 tx=0x7f0f54009ed0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 shutdown_connections 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55accf0555b0 0x55accf0579d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55acceff0be0 0x55acceff3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55accefe5410 0x55accf04d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f47fff700 0 client.27527 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 --2- 172.21.15.31:0/1470231447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55accefe5d30 0x55accf04dd80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 >> 172.21.15.31:0/1470231447 conn(0x55accef52200 msgr2=0x55accefe72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 shutdown_connections 2023-10-06T21:41:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.099+0000 7f0f82a37b80 1 -- 172.21.15.31:0/1470231447 wait complete. 2023-10-06T21:41:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f82a37b80 20 client.27527 trim_cache size 0 max 16384 2023-10-06T21:41:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:41:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:41:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:41:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 msgr2=0x7f0f580a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 0x7f0f580a3b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0a29a0 tx=0x7f0f5c00a040 comp rx=0 tx=0).stop 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 msgr2=0x7f0f681a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0084c0 tx=0x7f0f5c004510 comp rx=0 tx=0).stop 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 shutdown_connections 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6810d8e0 0x7f0f6819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0f580a1670 0x7f0f580a3b30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f68107eb0 0x7f0f681a3520 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 --2- 172.21.15.31:0/3943241213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68108870 0x7f0f681a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:41:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.100+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 >> 172.21.15.31:0/3943241213 conn(0x7f0f6806d3e0 msgr2=0x7f0f680717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:41:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.101+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 shutdown_connections 2023-10-06T21:41:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:41:54.101+0000 7f0f6f1ff700 1 -- 172.21.15.31:0/3943241213 wait complete. 2023-10-06T21:41:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: pgmap v663: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 240 op/s 2023-10-06T21:41:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:41:54.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:41:54.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:41:54.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:41:54.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:54 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:54.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55264 6.39 MB/sec execute 343 sec latency 84.063 ms 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: pgmap v663: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 240 op/s 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:41:54.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:41:54.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:41:54.800 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:54 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: pgmap v663: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 240 op/s 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi146 to 220.9M 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi146 to 231676211: error parsing value: Value '231676211' is below minimum 939524096 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:41:54.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:54 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:41:55.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55346 6.38 MB/sec execute 344 sec latency 66.360 ms 2023-10-06T21:41:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:56 smithi031 ceph-mon[98539]: pgmap v664: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:56.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55463 6.40 MB/sec execute 345 sec latency 74.338 ms 2023-10-06T21:41:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:56 smithi146 ceph-mon[102809]: pgmap v664: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:56.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:56 smithi142 ceph-mon[102951]: pgmap v664: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:57.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55567 6.39 MB/sec execute 346 sec latency 73.875 ms 2023-10-06T21:41:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:41:58 smithi031 ceph-mon[98539]: pgmap v665: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:58.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55690 6.40 MB/sec execute 347 sec latency 78.074 ms 2023-10-06T21:41:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:41:58 smithi146 ceph-mon[102809]: pgmap v665: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:58.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:41:58 smithi142 ceph-mon[102951]: pgmap v665: 97 pgs: 97 active+clean; 949 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 150 op/s 2023-10-06T21:41:59.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55785 6.39 MB/sec execute 348 sec latency 83.069 ms 2023-10-06T21:42:00.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55893 6.39 MB/sec execute 349 sec latency 104.982 ms 2023-10-06T21:42:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:01 smithi031 ceph-mon[98539]: pgmap v666: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 242 op/s 2023-10-06T21:42:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:01 smithi146 ceph-mon[102809]: pgmap v666: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 242 op/s 2023-10-06T21:42:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:01 smithi142 ceph-mon[102951]: pgmap v666: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 242 op/s 2023-10-06T21:42:01.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 55999 6.38 MB/sec execute 350 sec latency 78.670 ms 2023-10-06T21:42:02.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56128 6.37 MB/sec execute 351 sec latency 72.312 ms 2023-10-06T21:42:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:03 smithi031 ceph-mon[98539]: pgmap v667: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 166 op/s 2023-10-06T21:42:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:03 smithi146 ceph-mon[102809]: pgmap v667: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 166 op/s 2023-10-06T21:42:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:03 smithi142 ceph-mon[102951]: pgmap v667: 97 pgs: 97 active+clean; 956 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 166 op/s 2023-10-06T21:42:03.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56222 6.36 MB/sec execute 352 sec latency 81.445 ms 2023-10-06T21:42:04.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56320 6.35 MB/sec execute 353 sec latency 177.420 ms 2023-10-06T21:42:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:05 smithi031 ceph-mon[98539]: pgmap v668: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 276 op/s 2023-10-06T21:42:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:05 smithi031 ceph-mon[98539]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:42:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:05 smithi146 ceph-mon[102809]: pgmap v668: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 276 op/s 2023-10-06T21:42:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:05 smithi146 ceph-mon[102809]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:42:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:05 smithi142 ceph-mon[102951]: pgmap v668: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 276 op/s 2023-10-06T21:42:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:05 smithi142 ceph-mon[102951]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:42:05.784 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56461 6.36 MB/sec execute 354 sec latency 81.564 ms 2023-10-06T21:42:06.783 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56593 6.36 MB/sec execute 355 sec latency 82.665 ms 2023-10-06T21:42:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:07 smithi031 ceph-mon[98539]: pgmap v670: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:07 smithi146 ceph-mon[102809]: pgmap v670: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:07 smithi142 ceph-mon[102951]: pgmap v670: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 -- 172.21.15.31:0/1706184550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f641038b0 msgr2=0x7f3f64103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/1706184550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f641038b0 0x7f3f64103d30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098f0 tx=0x7f3f54009c00 comp rx=0 tx=0).stop 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 -- 172.21.15.31:0/1706184550 shutdown_connections 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/1706184550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f641038b0 0x7f3f64103d30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/1706184550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f64105e20 0x7f3f640682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/1706184550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 0x7f3f64068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 -- 172.21.15.31:0/1706184550 >> 172.21.15.31:0/1706184550 conn(0x7f3f64076b70 msgr2=0x7f3f64076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 -- 172.21.15.31:0/1706184550 shutdown_connections 2023-10-06T21:42:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.780+0000 7f3f6bdff700 1 -- 172.21.15.31:0/1706184550 wait complete. 2023-10-06T21:42:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.782+0000 7f3f6bdff700 1 Processor -- start 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.782+0000 7f3f6bdff700 1 -- start start 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 0x7f3f64072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 0x7f3f64072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6406baf0 con 0x7f3f64068810 2023-10-06T21:42:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6406bc60 con 0x7f3f641038b0 2023-10-06T21:42:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.783+0000 7f3f6bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6406be20 con 0x7f3f64105e20 2023-10-06T21:42:07.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56701 6.35 MB/sec execute 356 sec latency 82.547 ms 2023-10-06T21:42:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.784+0000 7f3f6939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 0x7f3f64072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.784+0000 7f3f6a39c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.784+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 0x7f3f64072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.784+0000 7f3f6a39c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39926/0 (socket says 172.21.15.31:39926) 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.784+0000 7f3f6a39c700 1 -- 172.21.15.31:0/2195204360 learned_addr learned my addr 172.21.15.31:0/2195204360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 -- 172.21.15.31:0/2195204360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 msgr2=0x7f3f6406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 -- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 msgr2=0x7f3f64072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 0x7f3f64072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 -- 172.21.15.31:0/2195204360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54009520 con 0x7f3f641038b0 2023-10-06T21:42:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:42:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.785+0000 7f3f6939a700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 0x7f3f64072a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007ec0 tx=0x7f3f5800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.786+0000 7f3f62ffd700 1 -- 172.21.15.31:0/2195204360 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004db0 con 0x7f3f641038b0 2023-10-06T21:42:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.786+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6406c100 con 0x7f3f641038b0 2023-10-06T21:42:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.786+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f640703e0 con 0x7f3f641038b0 2023-10-06T21:42:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.787+0000 7f3f62ffd700 1 -- 172.21.15.31:0/2195204360 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f5800f040 con 0x7f3f641038b0 2023-10-06T21:42:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.787+0000 7f3f62ffd700 1 -- 172.21.15.31:0/2195204360 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58005870 con 0x7f3f641038b0 2023-10-06T21:42:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.792+0000 7f3f7f70bb80 1 Processor -- start 2023-10-06T21:42:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.792+0000 7f3f7f70bb80 1 -- start start 2023-10-06T21:42:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 0x55e6bc313fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc27d400 con 0x55e6bc313c00 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc26ebc0 con 0x55e6bc314520 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc26c4e0 con 0x55e6bc31f3d0 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f62ffd700 1 -- 172.21.15.31:0/2195204360 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3f58015400 con 0x7f3f641038b0 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.793+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40716/0 (socket says 172.21.15.31:40716) 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 -- 172.21.15.31:0/438085425 learned_addr learned my addr 172.21.15.31:0/438085425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f6939a700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 -- 172.21.15.31:0/438085425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 msgr2=0x55e6bc3217c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3217c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 -- 172.21.15.31:0/438085425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 msgr2=0x55e6bc313fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 0x55e6bc313fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 0x55e6bc313fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 -- 172.21.15.31:0/438085425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6bc092d80 con 0x55e6bc314520 2023-10-06T21:42:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f6939a700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:42:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 0x55e6bc313fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:42:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.794+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004660 tx=0x7f3f5c00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44877ac522fd98d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.795+0000 7f3f4bfff700 1 -- 172.21.15.31:0/438085425 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0039a0 con 0x55e6bc314520 2023-10-06T21:42:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.795+0000 7f3f4bfff700 1 -- 172.21.15.31:0/438085425 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f5c01a470 con 0x55e6bc314520 2023-10-06T21:42:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.795+0000 7f3f4bfff700 1 -- 172.21.15.31:0/438085425 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0116a0 con 0x55e6bc314520 2023-10-06T21:42:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.795+0000 7f3f62ffd700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 0x7f3f4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f62ffd700 1 -- 172.21.15.31:0/2195204360 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f3f580cc9a0 con 0x7f3f641038b0 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/438085425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 msgr2=0x55e6bc31ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004660 tx=0x7f3f5c00ba10 comp rx=0 tx=0).stop 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/438085425 shutdown_connections 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc313c00 0x55e6bc313fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/438085425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31ee90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/438085425 >> 172.21.15.31:0/438085425 conn(0x55e6bc289200 msgr2=0x55e6bc310080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.796+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 0x7f3f4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.797+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/438085425 shutdown_connections 2023-10-06T21:42:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.797+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/438085425 wait complete. 2023-10-06T21:42:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.798+0000 7f3f7f70bb80 1 -- start start 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.798+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 0x7f3f4c0a3b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098c0 tx=0x7f3f54005dc0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.798+0000 7f3f7f70bb80 1 Processor -- start 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc31ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc31c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc267dd0 con 0x55e6bc31f3d0 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc2046b0 con 0x55e6bc314520 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc204350 con 0x55e6bc313c00 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f7f70bb80 10 client.0 _ll_register_callbacks cb 0x55e6bc0617a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f6a39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc31c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.799+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc31ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f6a39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc31c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42882/0 (socket says 172.21.15.31:42882) 2023-10-06T21:42:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f6a39c700 1 -- 172.21.15.31:0/3042631744 learned_addr learned my addr 172.21.15.31:0/3042631744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f6939a700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f69b9b700 1 -- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 msgr2=0x55e6bc31bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f69b9b700 1 -- 172.21.15.31:0/3042631744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 msgr2=0x55e6bc31c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc31c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.800+0000 7f3f69b9b700 1 -- 172.21.15.31:0/3042631744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c004290 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc31ed50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004260 tx=0x7f3f5c01a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c10afca26c9dbaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f497fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01e040 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6bc307d50 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f3f5c003bb0 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f5c003d70 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f7f70bb80 10 client.27481 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.801+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e6bc07bc00 con 0x55e6bc313c00 2023-10-06T21:42:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6bc307d50 con 0x55e6bc313c00 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f5c011440 con 0x55e6bc313c00 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 10 client.27481 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 10 client.27481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 20 client.27481 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc32ed80 0x55e6bc3311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 4 client.27481 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:42:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.802+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6bc260db0 con 0x55e6bc32ed80 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.803+0000 7f3f6939a700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc32ed80 0x55e6bc3311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.804+0000 7f3f6939a700 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc32ed80 0x55e6bc3311a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e74dff6dc64e5c3 server_cookie=a37e797bfabc46bd in_seq=0 out_seq=0 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.804+0000 7f3f497fa700 10 client.27481 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.805+0000 7f3f497fa700 1 -- 172.21.15.31:0/3042631744 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e6bc260db0 con 0x55e6bc32ed80 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.805+0000 7f3f497fa700 10 client.27481 handle_command_reply: tid=0 2023-10-06T21:42:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.805+0000 7f3f7f70bb80 1 client.27481 shutdown 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.805+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 msgr2=0x55e6bc31ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.805+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc31ed50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004260 tx=0x7f3f5c01a810 comp rx=0 tx=0).stop 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 shutdown_connections 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc32ed80 0x55e6bc3311a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc31c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc31ed50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/3042631744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc31bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 >> 172.21.15.31:0/3042631744 conn(0x55e6bc289200 msgr2=0x55e6bc30fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.806+0000 7f3f497fa700 0 client.27481 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.807+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 shutdown_connections 2023-10-06T21:42:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.807+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/3042631744 wait complete. 2023-10-06T21:42:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.807+0000 7f3f7f70bb80 20 client.27481 trim_cache size 0 max 16384 2023-10-06T21:42:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.831+0000 7f3f7f70bb80 1 Processor -- start 2023-10-06T21:42:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 -- start start 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc314520 0x55e6bc330d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc31f3d0 0x55e6bc32a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc0e88c0 con 0x55e6bc314520 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc317e40 con 0x55e6bc31f3d0 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc31d320 con 0x55e6bc313c00 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39962/0 (socket says 172.21.15.31:39962) 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1433405280 learned_addr learned my addr 172.21.15.31:0/1433405280 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.832+0000 7f3f6939a700 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc314520 0x55e6bc330d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc31f3d0 0x55e6bc32a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1433405280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc31f3d0 msgr2=0x55e6bc32a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc31f3d0 0x55e6bc32a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1433405280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc314520 msgr2=0x55e6bc330d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc314520 0x55e6bc330d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1433405280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6bc0b91f0 con 0x55e6bc313c00 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.833+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004580 tx=0x7f3f5c005950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abbec6e8c4f13610 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.834+0000 7f3f60bff700 1 -- 172.21.15.31:0/1433405280 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01e040 con 0x55e6bc313c00 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.834+0000 7f3f60bff700 1 -- 172.21.15.31:0/1433405280 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f5c00bec0 con 0x55e6bc313c00 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.834+0000 7f3f60bff700 1 -- 172.21.15.31:0/1433405280 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c023440 con 0x55e6bc313c00 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1433405280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 msgr2=0x55e6bc325f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c004580 tx=0x7f3f5c005950 comp rx=0 tx=0).stop 2023-10-06T21:42:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1433405280 shutdown_connections 2023-10-06T21:42:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc31f3d0 0x55e6bc32a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc325f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1433405280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc314520 0x55e6bc330d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.835+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1433405280 >> 172.21.15.31:0/1433405280 conn(0x55e6bc289200 msgr2=0x55e6bc30f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.836+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1433405280 shutdown_connections 2023-10-06T21:42:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.836+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1433405280 wait complete. 2023-10-06T21:42:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.836+0000 7f3f7f70bb80 1 -- start start 2023-10-06T21:42:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.837+0000 7f3f7f70bb80 1 Processor -- start 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.837+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.837+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc322490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f7f70bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc381b60 con 0x55e6bc31f3d0 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc381ca0 con 0x55e6bc314520 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f7f70bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bc381de0 con 0x55e6bc313c00 2023-10-06T21:42:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f7f70bb80 10 client.0 _ll_register_callbacks cb 0x7f3f5c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:39978/0 (socket says 172.21.15.31:39978) 2023-10-06T21:42:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1925927518 learned_addr learned my addr 172.21.15.31:0/1925927518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f6939a700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc322490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f6a39c700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 msgr2=0x55e6bc322490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc322490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1925927518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 msgr2=0x55e6bc3229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.838+0000 7f3f69b9b700 1 -- 172.21.15.31:0/1925927518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6bc07bc00 con 0x55e6bc313c00 2023-10-06T21:42:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.839+0000 7f3f69b9b700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0118e0 tx=0x7f3f5c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f96bbc423a8eae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.839+0000 7f3f497fa700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.839+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01e070 con 0x55e6bc313c00 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.839+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f3f5c000e40 con 0x55e6bc313c00 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.839+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6bc260ad0 con 0x55e6bc313c00 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.840+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f5c01ab30 con 0x55e6bc313c00 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.840+0000 7f3f7f70bb80 10 client.27487 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.840+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e6bc091250 con 0x55e6bc313c00 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.840+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6bc260ad0 con 0x55e6bc313c00 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f5c03d9f0 con 0x55e6bc313c00 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 10 client.27487 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 10 client.27487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 20 client.27487 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc3830c0 0x55e6bc3854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 4 client.27487 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:42:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6bc0b3f10 con 0x55e6bc3830c0 2023-10-06T21:42:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.841+0000 7f3f6939a700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc3830c0 0x55e6bc3854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.842+0000 7f3f6939a700 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc3830c0 0x55e6bc3854b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08a554cb1a68612 server_cookie=c7aee455b7691e87 in_seq=0 out_seq=0 2023-10-06T21:42:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.842+0000 7f3f497fa700 10 client.27487 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.915+0000 7f3f497fa700 1 -- 172.21.15.31:0/1925927518 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6bc0b3f10 con 0x55e6bc3830c0 2023-10-06T21:42:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.915+0000 7f3f497fa700 10 client.27487 handle_command_reply: tid=0 2023-10-06T21:42:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.915+0000 7f3f7f70bb80 1 client.27487 shutdown 2023-10-06T21:42:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.915+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 msgr2=0x55e6bc321f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.915+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0118e0 tx=0x7f3f5c005c70 comp rx=0 tx=0).stop 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 shutdown_connections 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e6bc3830c0 0x55e6bc3854b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6bc31f3d0 0x55e6bc3229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e6bc313c00 0x55e6bc321f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 --2- 172.21.15.31:0/1925927518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6bc314520 0x55e6bc322490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f497fa700 0 client.27487 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 >> 172.21.15.31:0/1925927518 conn(0x55e6bc289200 msgr2=0x55e6bc330d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 shutdown_connections 2023-10-06T21:42:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 1 -- 172.21.15.31:0/1925927518 wait complete. 2023-10-06T21:42:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.916+0000 7f3f7f70bb80 20 client.27487 trim_cache size 0 max 16384 2023-10-06T21:42:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:42:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:42:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:42:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.917+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 msgr2=0x7f3f4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.917+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 0x7f3f4c0a3b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098c0 tx=0x7f3f54005dc0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.917+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 msgr2=0x7f3f64072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.917+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 0x7f3f64072a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007ec0 tx=0x7f3f5800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 shutdown_connections 2023-10-06T21:42:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f64105e20 0x7f3f6406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3f4c0a1670 0x7f3f4c0a3b30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64068810 0x7f3f64072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 --2- 172.21.15.31:0/2195204360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f641038b0 0x7f3f64072a00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.918+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 >> 172.21.15.31:0/2195204360 conn(0x7f3f64076b70 msgr2=0x7f3f64102890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.919+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 shutdown_connections 2023-10-06T21:42:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:07.919+0000 7f3f6bdff700 1 -- 172.21.15.31:0/2195204360 wait complete. 2023-10-06T21:42:08.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56798 6.35 MB/sec execute 357 sec latency 119.999 ms 2023-10-06T21:42:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:09 smithi031 ceph-mon[98539]: pgmap v671: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:09 smithi146 ceph-mon[102809]: pgmap v671: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:09.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:09 smithi142 ceph-mon[102951]: pgmap v671: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 242 op/s 2023-10-06T21:42:09.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 56909 6.35 MB/sec execute 358 sec latency 248.770 ms 2023-10-06T21:42:10.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57039 6.35 MB/sec execute 359 sec latency 66.591 ms 2023-10-06T21:42:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:11 smithi031 ceph-mon[98539]: pgmap v672: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:11 smithi146 ceph-mon[102809]: pgmap v672: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:11 smithi142 ceph-mon[102951]: pgmap v672: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:11.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57160 6.35 MB/sec execute 360 sec latency 71.874 ms 2023-10-06T21:42:12.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57289 6.34 MB/sec execute 361 sec latency 66.385 ms 2023-10-06T21:42:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:13 smithi031 ceph-mon[98539]: pgmap v673: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:13 smithi146 ceph-mon[102809]: pgmap v673: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:13 smithi142 ceph-mon[102951]: pgmap v673: 97 pgs: 97 active+clean; 950 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-10-06T21:42:13.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57430 6.35 MB/sec execute 362 sec latency 68.164 ms 2023-10-06T21:42:14.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57556 6.34 MB/sec execute 363 sec latency 72.379 ms 2023-10-06T21:42:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:15 smithi031 ceph-mon[98539]: pgmap v674: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:15 smithi031 ceph-mon[98539]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:42:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:15 smithi146 ceph-mon[102809]: pgmap v674: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:15 smithi146 ceph-mon[102809]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:42:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:15 smithi142 ceph-mon[102951]: pgmap v674: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:15.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:15 smithi142 ceph-mon[102951]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:42:15.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57668 6.33 MB/sec execute 364 sec latency 69.844 ms 2023-10-06T21:42:16.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57797 6.34 MB/sec execute 365 sec latency 67.496 ms 2023-10-06T21:42:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:17 smithi031 ceph-mon[98539]: pgmap v676: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:17 smithi146 ceph-mon[102809]: pgmap v676: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:17 smithi142 ceph-mon[102951]: pgmap v676: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:17.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 57940 6.35 MB/sec execute 366 sec latency 105.609 ms 2023-10-06T21:42:18.785 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58090 6.37 MB/sec execute 367 sec latency 84.251 ms 2023-10-06T21:42:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:19 smithi031 ceph-mon[98539]: pgmap v677: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:19 smithi146 ceph-mon[102809]: pgmap v677: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:19.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:19 smithi142 ceph-mon[102951]: pgmap v677: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2023-10-06T21:42:19.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58229 6.37 MB/sec execute 368 sec latency 70.743 ms 2023-10-06T21:42:20.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58310 6.37 MB/sec execute 369 sec latency 419.334 ms 2023-10-06T21:42:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:21 smithi031 ceph-mon[98539]: pgmap v678: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:21 smithi146 ceph-mon[102809]: pgmap v678: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:21 smithi142 ceph-mon[102951]: pgmap v678: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.694+0000 7ff42bfff700 1 -- 172.21.15.31:0/2226318587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 msgr2=0x7ff424103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.694+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2226318587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424103700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff420009990 tx=0x7ff420009ca0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 -- 172.21.15.31:0/2226318587 shutdown_connections 2023-10-06T21:42:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2226318587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff424108440 0x7ff42410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2226318587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424103700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2226318587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424103cd0 0x7ff424107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 -- 172.21.15.31:0/2226318587 >> 172.21.15.31:0/2226318587 conn(0x7ff4240feb60 msgr2=0x7ff424100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 -- 172.21.15.31:0/2226318587 shutdown_connections 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.695+0000 7ff42bfff700 1 -- 172.21.15.31:0/2226318587 wait complete. 2023-10-06T21:42:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.696+0000 7ff42bfff700 1 Processor -- start 2023-10-06T21:42:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 -- start start 2023-10-06T21:42:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 0x7ff42419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419e760 con 0x7ff424108440 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419e8d0 con 0x7ff42419dce0 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419ea90 con 0x7ff424103320 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42959a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32920/0 (socket says 172.21.15.31:32920) 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42959a700 1 -- 172.21.15.31:0/2087590644 learned_addr learned my addr 172.21.15.31:0/2087590644 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.697+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 0x7ff42419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff429d9b700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42959a700 1 -- 172.21.15.31:0/2087590644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 msgr2=0x7ff424199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42959a700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42959a700 1 -- 172.21.15.31:0/2087590644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 msgr2=0x7ff42419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42959a700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 0x7ff42419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42959a700 1 -- 172.21.15.31:0/2087590644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200095f0 con 0x7ff424108440 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.698+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 0x7ff42419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:42:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff42959a700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e970 tx=0x7ff41400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff41affd700 1 -- 172.21.15.31:0/2087590644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414009790 con 0x7ff424108440 2023-10-06T21:42:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff41affd700 1 -- 172.21.15.31:0/2087590644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff41400c7e0 con 0x7ff424108440 2023-10-06T21:42:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4241a5ff0 con 0x7ff424108440 2023-10-06T21:42:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff41affd700 1 -- 172.21.15.31:0/2087590644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414005440 con 0x7ff424108440 2023-10-06T21:42:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.699+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a64c0 con 0x7ff424108440 2023-10-06T21:42:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.704+0000 7ff41affd700 1 -- 172.21.15.31:0/2087590644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff41400c950 con 0x7ff424108440 2023-10-06T21:42:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.705+0000 7ff43f947b80 1 Processor -- start 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.705+0000 7ff43f947b80 1 -- start start 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe39577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe39626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe38b8400 con 0x559fe3957410 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe38a9bc0 con 0x559fe3962be0 2023-10-06T21:42:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff43f947b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe38a74e0 con 0x559fe3957d30 2023-10-06T21:42:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff41affd700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 0x7ff4100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff42959a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff42959a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41000/0 (socket says 172.21.15.31:41000) 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.706+0000 7ff42959a700 1 -- 172.21.15.31:0/3860979454 learned_addr learned my addr 172.21.15.31:0/3860979454 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff429d9b700 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe39626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff41affd700 1 -- 172.21.15.31:0/2087590644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff414014070 con 0x7ff424108440 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff42959a700 1 -- 172.21.15.31:0/3860979454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 msgr2=0x559fe39626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff42959a700 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe39626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff42959a700 1 -- 172.21.15.31:0/3860979454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 msgr2=0x559fe39577f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff42959a700 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe39577f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.707+0000 7ff42959a700 1 -- 172.21.15.31:0/3860979454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fe36cdd80 con 0x559fe3962be0 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.708+0000 7ff42959a700 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ca8f0 tx=0x7ff41400cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365330ffc61fe506 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.708+0000 7ff40ffff700 1 -- 172.21.15.31:0/3860979454 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41409a210 con 0x559fe3962be0 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.708+0000 7ff40ffff700 1 -- 172.21.15.31:0/3860979454 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff41409e360 con 0x559fe3962be0 2023-10-06T21:42:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.708+0000 7ff40ffff700 1 -- 172.21.15.31:0/3860979454 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414016330 con 0x559fe3962be0 2023-10-06T21:42:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.708+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 0x7ff4100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 -- 172.21.15.31:0/3860979454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 msgr2=0x559fe3964fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ca8f0 tx=0x7ff41400cf50 comp rx=0 tx=0).stop 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 0x7ff4100a3b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007550 tx=0x7ff41c009130 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 -- 172.21.15.31:0/3860979454 shutdown_connections 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe3964fd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe39577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3860979454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe39626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 -- 172.21.15.31:0/3860979454 >> 172.21.15.31:0/3860979454 conn(0x559fe38c4200 msgr2=0x559fe394aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.709+0000 7ff43f947b80 1 -- 172.21.15.31:0/3860979454 shutdown_connections 2023-10-06T21:42:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.710+0000 7ff43f947b80 1 -- 172.21.15.31:0/3860979454 wait complete. 2023-10-06T21:42:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.710+0000 7ff43f947b80 1 -- start start 2023-10-06T21:42:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.710+0000 7ff43f947b80 1 Processor -- start 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 0x559fe39625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe395f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe38a2dd0 con 0x559fe3957d30 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe383f6b0 con 0x559fe3962be0 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe383f350 con 0x559fe3957410 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff43f947b80 10 client.0 _ll_register_callbacks cb 0x559fe369c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.711+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:41006/0 (socket says 172.21.15.31:41006) 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 -- 172.21.15.31:0/2134585098 learned_addr learned my addr 172.21.15.31:0/2134585098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42959a700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe395f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 -- 172.21.15.31:0/2134585098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 msgr2=0x559fe39625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff429d9b700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 0x559fe39625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 0x559fe39625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 -- 172.21.15.31:0/2134585098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 msgr2=0x559fe395f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe395f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff42a59c700 1 -- 172.21.15.31:0/2134585098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a64c0 con 0x559fe3962be0 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.712+0000 7ff429d9b700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 0x559fe39625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff42a59c700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb26ebbc441582e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff40d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c021640 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff41c016e20 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fe3942d50 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff41c0217a0 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff43f947b80 10 client.27557 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.713+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559fe36b6c00 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559fe3942d50 con 0x559fe3962be0 2023-10-06T21:42:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff41c02a890 con 0x559fe3962be0 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 10 client.27557 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 10 client.27557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 20 client.27557 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39726b0 0x559fe3974aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 4 client.27557 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.714+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fe389bdb0 con 0x559fe39726b0 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.715+0000 7ff429d9b700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39726b0 0x559fe3974aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.715+0000 7ff429d9b700 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39726b0 0x559fe3974aa0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8da771d3a417a5 server_cookie=83f93aaeaf17a8b in_seq=0 out_seq=0 2023-10-06T21:42:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.715+0000 7ff40d7fa700 10 client.27557 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff40d7fa700 1 -- 172.21.15.31:0/2134585098 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fe389bdb0 con 0x559fe39726b0 2023-10-06T21:42:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff40d7fa700 10 client.27557 handle_command_reply: tid=0 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 client.27557 shutdown 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 msgr2=0x559fe395fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c004b40 comp rx=0 tx=0).stop 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 shutdown_connections 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395fed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957410 0x559fe39625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39726b0 0x559fe3974aa0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 --2- 172.21.15.31:0/2134585098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe395f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff40d7fa700 0 client.27557 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.717+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 >> 172.21.15.31:0/2134585098 conn(0x559fe38c4200 msgr2=0x559fe394abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.718+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 shutdown_connections 2023-10-06T21:42:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.718+0000 7ff43f947b80 1 -- 172.21.15.31:0/2134585098 wait complete. 2023-10-06T21:42:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.718+0000 7ff43f947b80 20 client.27557 trim_cache size 0 max 16384 2023-10-06T21:42:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.745+0000 7ff43f947b80 1 Processor -- start 2023-10-06T21:42:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 -- start start 2023-10-06T21:42:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe3964fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe395a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe37238c0 con 0x559fe3957410 2023-10-06T21:42:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe395b750 con 0x559fe3962be0 2023-10-06T21:42:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff43f947b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe396ef80 con 0x559fe3957d30 2023-10-06T21:42:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff429d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe3964fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff429d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe3964fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32942/0 (socket says 172.21.15.31:32942) 2023-10-06T21:42:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff429d9b700 1 -- 172.21.15.31:0/3631686793 learned_addr learned my addr 172.21.15.31:0/3631686793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.746+0000 7ff42a59c700 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42959a700 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe395a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 -- 172.21.15.31:0/3631686793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 msgr2=0x559fe395a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe395a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 -- 172.21.15.31:0/3631686793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 msgr2=0x559fe3964fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe3964fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 -- 172.21.15.31:0/3631686793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fe36f41f0 con 0x559fe3962be0 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.747+0000 7ff42a59c700 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395f0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02acc0 tx=0x7ff41c009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39437f8f97ca0c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.748+0000 7ff418ff9700 1 -- 172.21.15.31:0/3631686793 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c003c00 con 0x559fe3962be0 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.748+0000 7ff418ff9700 1 -- 172.21.15.31:0/3631686793 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff41c01a070 con 0x559fe3962be0 2023-10-06T21:42:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.748+0000 7ff418ff9700 1 -- 172.21.15.31:0/3631686793 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0042a0 con 0x559fe3962be0 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 -- 172.21.15.31:0/3631686793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 msgr2=0x559fe395f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395f0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02acc0 tx=0x7ff41c009da0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 -- 172.21.15.31:0/3631686793 shutdown_connections 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3962be0 0x559fe395f0a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957410 0x559fe3964fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 --2- 172.21.15.31:0/3631686793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3957d30 0x559fe395a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 -- 172.21.15.31:0/3631686793 >> 172.21.15.31:0/3631686793 conn(0x559fe38c4200 msgr2=0x559fe395abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 -- 172.21.15.31:0/3631686793 shutdown_connections 2023-10-06T21:42:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.749+0000 7ff43f947b80 1 -- 172.21.15.31:0/3631686793 wait complete. 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.750+0000 7ff43f947b80 1 -- start start 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.750+0000 7ff43f947b80 1 Processor -- start 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 0x559fe3967fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe39684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 0x559fe3968a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe39c6640 con 0x559fe3957d30 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe39c67b0 con 0x559fe3957410 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fe39c6970 con 0x559fe3962be0 2023-10-06T21:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff43f947b80 10 client.0 _ll_register_callbacks cb 0x7ff41c015880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 0x559fe3968a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 0x559fe3968a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46322/0 (socket says 172.21.15.31:46322) 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff42a59c700 1 -- 172.21.15.31:0/901398684 learned_addr learned my addr 172.21.15.31:0/901398684 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.751+0000 7ff429d9b700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 0x559fe3967fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff42959a700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe39684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 -- 172.21.15.31:0/901398684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 msgr2=0x559fe3968a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 0x559fe3968a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 -- 172.21.15.31:0/901398684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 msgr2=0x559fe39684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe39684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 -- 172.21.15.31:0/901398684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fe36b6c00 con 0x559fe3957410 2023-10-06T21:42:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff42959a700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe39684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff429d9b700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 0x559fe3967fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff420005910 tx=0x7ff420005500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5426353460469b97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.752+0000 7ff40d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42001d040 con 0x559fe3957410 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff41409e360 con 0x559fe3957410 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff420027c70 con 0x559fe3957410 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4200275b0 con 0x559fe3957410 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff43f947b80 10 client.27563 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.753+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559fe36cc250 con 0x559fe3957410 2023-10-06T21:42:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff41409e360 con 0x559fe3957410 2023-10-06T21:42:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff420030470 con 0x559fe3957410 2023-10-06T21:42:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 10 client.27563 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 10 client.27563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 20 client.27563 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39c8180 0x559fe39ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 4 client.27563 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559fe389bad0 con 0x559fe39c8180 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.754+0000 7ff42959a700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39c8180 0x559fe39ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.755+0000 7ff42959a700 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39c8180 0x559fe39ca5a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9f1d2871ecf5e0 server_cookie=5aef6b25cd0acedc in_seq=0 out_seq=0 2023-10-06T21:42:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.755+0000 7ff40d7fa700 10 client.27563 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:21.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58395 6.36 MB/sec execute 370 sec latency 78.782 ms 2023-10-06T21:42:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.816+0000 7ff40d7fa700 1 -- 172.21.15.31:0/901398684 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559fe389bad0 con 0x559fe39c8180 2023-10-06T21:42:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.816+0000 7ff40d7fa700 10 client.27563 handle_command_reply: tid=0 2023-10-06T21:42:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.816+0000 7ff43f947b80 1 client.27563 shutdown 2023-10-06T21:42:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 msgr2=0x559fe3967fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 0x559fe3967fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff420005910 tx=0x7ff420005500 comp rx=0 tx=0).stop 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 shutdown_connections 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fe3962be0 0x559fe3968a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fe3957410 0x559fe3967fb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559fe39c8180 0x559fe39ca5a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 --2- 172.21.15.31:0/901398684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe3957d30 0x559fe39684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 >> 172.21.15.31:0/901398684 conn(0x559fe38c4200 msgr2=0x559fe3961160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff40d7fa700 0 client.27563 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 shutdown_connections 2023-10-06T21:42:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.817+0000 7ff43f947b80 1 -- 172.21.15.31:0/901398684 wait complete. 2023-10-06T21:42:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.818+0000 7ff43f947b80 20 client.27563 trim_cache size 0 max 16384 2023-10-06T21:42:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:42:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:42:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:42:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:42:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 msgr2=0x7ff4100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 0x7ff4100a3b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007550 tx=0x7ff41c009130 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 msgr2=0x7ff424199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e970 tx=0x7ff41400ec80 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 shutdown_connections 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42419dce0 0x7ff42419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff424103320 0x7ff424199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff4100a1670 0x7ff4100a3b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 --2- 172.21.15.31:0/2087590644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff424108440 0x7ff424199670 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 >> 172.21.15.31:0/2087590644 conn(0x7ff4240feb60 msgr2=0x7ff424106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.819+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 shutdown_connections 2023-10-06T21:42:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:21.820+0000 7ff42bfff700 1 -- 172.21.15.31:0/2087590644 wait complete. 2023-10-06T21:42:22.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58542 6.38 MB/sec execute 371 sec latency 73.574 ms 2023-10-06T21:42:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:23 smithi031 ceph-mon[98539]: pgmap v679: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:23 smithi146 ceph-mon[102809]: pgmap v679: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:23 smithi142 ceph-mon[102951]: pgmap v679: 97 pgs: 97 active+clean; 975 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 207 op/s 2023-10-06T21:42:23.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58682 6.39 MB/sec execute 372 sec latency 66.084 ms 2023-10-06T21:42:24.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58810 6.39 MB/sec execute 373 sec latency 70.231 ms 2023-10-06T21:42:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:25 smithi031 ceph-mon[98539]: pgmap v680: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 207 op/s 2023-10-06T21:42:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:25 smithi146 ceph-mon[102809]: pgmap v680: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 207 op/s 2023-10-06T21:42:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:25 smithi142 ceph-mon[102951]: pgmap v680: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 207 op/s 2023-10-06T21:42:25.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 58925 6.39 MB/sec execute 374 sec latency 179.613 ms 2023-10-06T21:42:26.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59015 6.39 MB/sec execute 375 sec latency 273.910 ms 2023-10-06T21:42:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:27 smithi031 ceph-mon[98539]: pgmap v681: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 186 op/s 2023-10-06T21:42:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:27 smithi146 ceph-mon[102809]: pgmap v681: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 186 op/s 2023-10-06T21:42:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:27 smithi142 ceph-mon[102951]: pgmap v681: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 186 op/s 2023-10-06T21:42:27.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59125 6.39 MB/sec execute 376 sec latency 77.100 ms 2023-10-06T21:42:28.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59218 6.38 MB/sec execute 377 sec latency 73.421 ms 2023-10-06T21:42:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:29 smithi031 ceph-mon[98539]: pgmap v682: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 173 op/s 2023-10-06T21:42:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:29 smithi146 ceph-mon[102809]: pgmap v682: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 173 op/s 2023-10-06T21:42:29.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:29 smithi142 ceph-mon[102951]: pgmap v682: 97 pgs: 97 active+clean; 994 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 173 op/s 2023-10-06T21:42:29.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59330 6.39 MB/sec execute 378 sec latency 64.260 ms 2023-10-06T21:42:30.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59458 6.39 MB/sec execute 379 sec latency 71.873 ms 2023-10-06T21:42:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:31 smithi031 ceph-mon[98539]: pgmap v683: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2023-10-06T21:42:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:31 smithi146 ceph-mon[102809]: pgmap v683: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2023-10-06T21:42:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:31 smithi142 ceph-mon[102951]: pgmap v683: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2023-10-06T21:42:31.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59536 6.38 MB/sec execute 380 sec latency 388.225 ms 2023-10-06T21:42:32.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59659 6.37 MB/sec execute 381 sec latency 72.372 ms 2023-10-06T21:42:33.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:33 smithi031 ceph-mon[98539]: pgmap v684: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2023-10-06T21:42:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:33 smithi146 ceph-mon[102809]: pgmap v684: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2023-10-06T21:42:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:33 smithi142 ceph-mon[102951]: pgmap v684: 97 pgs: 97 active+clean; 1003 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2023-10-06T21:42:33.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59764 6.37 MB/sec execute 382 sec latency 59.872 ms 2023-10-06T21:42:34.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59876 6.36 MB/sec execute 383 sec latency 65.747 ms 2023-10-06T21:42:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:35 smithi031 ceph-mon[98539]: pgmap v685: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 270 op/s 2023-10-06T21:42:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:35 smithi146 ceph-mon[102809]: pgmap v685: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 270 op/s 2023-10-06T21:42:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:35 smithi142 ceph-mon[102951]: pgmap v685: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 270 op/s 2023-10-06T21:42:35.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 59981 6.36 MB/sec execute 384 sec latency 229.347 ms 2023-10-06T21:42:36.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60109 6.36 MB/sec execute 385 sec latency 64.647 ms 2023-10-06T21:42:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 -- 172.21.15.31:0/1954426848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0106f60 msgr2=0x7f0ff0107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/1954426848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0106f60 0x7f0ff0107340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009990 tx=0x7f0fe0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 -- 172.21.15.31:0/1954426848 shutdown_connections 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/1954426848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0068c90 0x7f0ff010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/1954426848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0106f60 0x7f0ff0107340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/1954426848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff00682d0 0x7f0ff0068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 -- 172.21.15.31:0/1954426848 >> 172.21.15.31:0/1954426848 conn(0x7f0ff0076b70 msgr2=0x7f0ff0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.825+0000 7f0ff61ff700 1 -- 172.21.15.31:0/1954426848 shutdown_connections 2023-10-06T21:42:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.826+0000 7f0ff61ff700 1 -- 172.21.15.31:0/1954426848 wait complete. 2023-10-06T21:42:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.827+0000 7f0ff61ff700 1 Processor -- start 2023-10-06T21:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.827+0000 7f0ff61ff700 1 -- start start 2023-10-06T21:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 0x7f0ff0109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 0x7f0ff010a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 0x7f0ff01a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff010ad40 con 0x7f0ff0106f60 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0198d10 con 0x7f0ff00682d0 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0ff61ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0198ed0 con 0x7f0ff0068c90 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 0x7f0ff010a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 0x7f0ff010a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49354/0 (socket says 172.21.15.31:49354) 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.828+0000 7f0feeffd700 1 -- 172.21.15.31:0/3460001444 learned_addr learned my addr 172.21.15.31:0/3460001444 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 0x7f0ff01a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 0x7f0ff0109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 -- 172.21.15.31:0/3460001444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 msgr2=0x7f0ff010a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 0x7f0ff010a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 -- 172.21.15.31:0/3460001444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 msgr2=0x7f0ff0109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 0x7f0ff0109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0feffff700 1 -- 172.21.15.31:0/3460001444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe00095f0 con 0x7f0ff0106f60 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.829+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 0x7f0ff0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 0x7f0ff01a9570 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800efc0 tx=0x7f0fe800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0fecff9700 1 -- 172.21.15.31:0/3460001444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8018070 con 0x7f0ff0106f60 2023-10-06T21:42:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0fecff9700 1 -- 172.21.15.31:0/3460001444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe800f040 con 0x7f0ff0106f60 2023-10-06T21:42:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0fecff9700 1 -- 172.21.15.31:0/3460001444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800cbd0 con 0x7f0ff0106f60 2023-10-06T21:42:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff0199150 con 0x7f0ff0106f60 2023-10-06T21:42:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.830+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff0199610 con 0x7f0ff0106f60 2023-10-06T21:42:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.836+0000 7f0fecff9700 1 -- 172.21.15.31:0/3460001444 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0fe8007500 con 0x7f0ff0106f60 2023-10-06T21:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.837+0000 7f1009b80b80 1 Processor -- start 2023-10-06T21:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 -- start start 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 0x55c946e47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 0x55c946e52e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946db1400 con 0x55c946e47c00 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946da2bc0 con 0x55c946e533d0 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.838+0000 7f1009b80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946da04e0 con 0x55c946e48520 2023-10-06T21:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fecff9700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 0x7f0fd80a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 0x55c946e47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 0x55c946e47fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50714/0 (socket says 172.21.15.31:50714) 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0feeffd700 1 -- 172.21.15.31:0/2912470949 learned_addr learned my addr 172.21.15.31:0/2912470949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fecff9700 1 -- 172.21.15.31:0/3460001444 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0fe80ccc70 con 0x7f0ff0106f60 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0feeffd700 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 0x55c946e52e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 -- 172.21.15.31:0/2912470949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 msgr2=0x55c946e557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 -- 172.21.15.31:0/2912470949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 msgr2=0x55c946e47fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 0x55c946e47fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.839+0000 7f0fef7fe700 1 -- 172.21.15.31:0/2912470949 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c946bc6d80 con 0x55c946e48520 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.840+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 0x7f0fd80a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.840+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 0x55c946e52e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0005910 tx=0x7f0fe0005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f620ba27e4bc162a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.840+0000 7f0fd5ffb700 1 -- 172.21.15.31:0/2912470949 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00056a0 con 0x55c946e48520 2023-10-06T21:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.840+0000 7f0fd5ffb700 1 -- 172.21.15.31:0/2912470949 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe001e040 con 0x55c946e48520 2023-10-06T21:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.840+0000 7f0fd5ffb700 1 -- 172.21.15.31:0/2912470949 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0004920 con 0x55c946e48520 2023-10-06T21:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.841+0000 7f1009b80b80 1 -- 172.21.15.31:0/2912470949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 msgr2=0x55c946e52e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.841+0000 7f0feffff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 0x7f0fd80a3c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c7a0 tx=0x7f0fe8004c50 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.841+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 0x55c946e52e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0005910 tx=0x7f0fe0005c00 comp rx=0 tx=0).stop 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 -- 172.21.15.31:0/2912470949 shutdown_connections 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e47c00 0x55c946e47fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2912470949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e48520 0x55c946e52e90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 -- 172.21.15.31:0/2912470949 >> 172.21.15.31:0/2912470949 conn(0x55c946dbd200 msgr2=0x55c946e44080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 -- 172.21.15.31:0/2912470949 shutdown_connections 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 -- 172.21.15.31:0/2912470949 wait complete. 2023-10-06T21:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.842+0000 7f1009b80b80 1 -- start start 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.843+0000 7f1009b80b80 1 Processor -- start 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e48520 0x55c946e50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e533d0 0x55c946e50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946d9bdd0 con 0x55c946e533d0 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946d386b0 con 0x55c946e48520 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946d38350 con 0x55c946e47c00 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f1009b80b80 10 client.0 _ll_register_callbacks cb 0x55c946b957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49358/0 (socket says 172.21.15.31:49358) 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f0feeffd700 1 -- 172.21.15.31:0/1006610098 learned_addr learned my addr 172.21.15.31:0/1006610098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e533d0 0x55c946e50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.844+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e48520 0x55c946e50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 -- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e48520 msgr2=0x55c946e50670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e48520 0x55c946e50670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 -- 172.21.15.31:0/1006610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e533d0 msgr2=0x55c946e50bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e533d0 0x55c946e50bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 -- 172.21.15.31:0/1006610098 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff010e5c0 con 0x55c946e47c00 2023-10-06T21:42:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4010440 tx=0x7f0fe400b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812ed97f50d92bc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0fceffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4010de0 con 0x55c946e47c00 2023-10-06T21:42:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.845+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c946e3bd50 con 0x55c946e47c00 2023-10-06T21:42:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0fe4022400 con 0x55c946e47c00 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe40125e0 con 0x55c946e47c00 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f1009b80b80 10 client.27514 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c946bafc00 con 0x55c946e47c00 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c946e3bd50 con 0x55c946e47c00 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.846+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fe4003bb0 con 0x55c946e47c00 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 10 client.27514 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 10 client.27514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 20 client.27514 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946e62e70 0x55c946e65290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 4 client.27514 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:42:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c946d94db0 con 0x55c946e62e70 2023-10-06T21:42:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.847+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946e62e70 0x55c946e65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.848+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946e62e70 0x55c946e65290 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8b8de773f4db60 server_cookie=a648853e2d27f5ad in_seq=0 out_seq=0 2023-10-06T21:42:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.848+0000 7f0fceffd700 10 client.27514 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f0fceffd700 1 -- 172.21.15.31:0/1006610098 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c946d94db0 con 0x55c946e62e70 2023-10-06T21:42:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f0fceffd700 10 client.27514 handle_command_reply: tid=0 2023-10-06T21:42:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 client.27514 shutdown 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 msgr2=0x55c946e50130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4010440 tx=0x7f0fe400b500 comp rx=0 tx=0).stop 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 shutdown_connections 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946e62e70 0x55c946e65290 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e533d0 0x55c946e50bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e50130 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f0fceffd700 0 client.27514 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1006610098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e48520 0x55c946e50670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.850+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 >> 172.21.15.31:0/1006610098 conn(0x55c946dbd200 msgr2=0x55c946e43c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.851+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 shutdown_connections 2023-10-06T21:42:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.851+0000 7f1009b80b80 1 -- 172.21.15.31:0/1006610098 wait complete. 2023-10-06T21:42:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.851+0000 7f1009b80b80 20 client.27514 trim_cache size 0 max 16384 2023-10-06T21:42:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.877+0000 7f1009b80b80 1 Processor -- start 2023-10-06T21:42:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.877+0000 7f1009b80b80 1 -- start start 2023-10-06T21:42:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e43bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e533d0 0x55c946e440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946c1c8c0 con 0x55c946e48520 2023-10-06T21:42:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946e48900 con 0x55c946e47c00 2023-10-06T21:42:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f1009b80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946e40860 con 0x55c946e533d0 2023-10-06T21:42:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f0fef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e43bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.878+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:37518/0 (socket says 172.21.15.31:37518) 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0fef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e43bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50740/0 (socket says 172.21.15.31:50740) 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 -- 172.21.15.31:0/2141295574 learned_addr learned my addr 172.21.15.31:0/2141295574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e533d0 0x55c946e440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 -- 172.21.15.31:0/2141295574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e533d0 msgr2=0x55c946e440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e533d0 0x55c946e440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 -- 172.21.15.31:0/2141295574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 msgr2=0x55c946e43bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e43bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 -- 172.21.15.31:0/2141295574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c946e4ff40 con 0x55c946e47c00 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.879+0000 7f0feeffd700 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40127b0 tx=0x7f0fe4001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed24b6ce563ebf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.880+0000 7f0fd67fc700 1 -- 172.21.15.31:0/2141295574 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40125e0 con 0x55c946e47c00 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.880+0000 7f0fd67fc700 1 -- 172.21.15.31:0/2141295574 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe400eec0 con 0x55c946e47c00 2023-10-06T21:42:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.880+0000 7f0fd67fc700 1 -- 172.21.15.31:0/2141295574 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4012970 con 0x55c946e47c00 2023-10-06T21:42:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- 172.21.15.31:0/2141295574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 msgr2=0x55c946e4ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40127b0 tx=0x7f0fe4001e70 comp rx=0 tx=0).stop 2023-10-06T21:42:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- 172.21.15.31:0/2141295574 shutdown_connections 2023-10-06T21:42:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e533d0 0x55c946e440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e47c00 0x55c946e4ac10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 --2- 172.21.15.31:0/2141295574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- 172.21.15.31:0/2141295574 >> 172.21.15.31:0/2141295574 conn(0x55c946dbd200 msgr2=0x55c946e4b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- 172.21.15.31:0/2141295574 shutdown_connections 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- 172.21.15.31:0/2141295574 wait complete. 2023-10-06T21:42:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.881+0000 7f1009b80b80 1 -- start start 2023-10-06T21:42:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 Processor -- start 2023-10-06T21:42:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e5a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e57f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946e58650 con 0x55c946e48520 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.882+0000 7f1009b80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946e587c0 con 0x55c946e533d0 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f1009b80b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946e58980 con 0x55c946e47c00 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f1009b80b80 10 client.0 _ll_register_callbacks cb 0x7f0fe4012b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:49378/0 (socket says 172.21.15.31:49378) 2023-10-06T21:42:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 -- 172.21.15.31:0/1756279472 learned_addr learned my addr 172.21.15.31:0/1756279472 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e57f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e5a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 -- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 msgr2=0x55c946e57f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e57f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 -- 172.21.15.31:0/1756279472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 msgr2=0x55c946e5a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e5a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.883+0000 7f0feeffd700 1 -- 172.21.15.31:0/1756279472 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c946bafc00 con 0x55c946e47c00 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.884+0000 7f0feeffd700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4012db0 tx=0x7f0fe400e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56cb2a0f0d4134f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.884+0000 7f0fceffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.884+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4010de0 con 0x55c946e47c00 2023-10-06T21:42:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.884+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f0fe4022400 con 0x55c946e47c00 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.884+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c946d94ad0 con 0x55c946e47c00 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.885+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe400ea30 con 0x55c946e47c00 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.885+0000 7f1009b80b80 10 client.27520 fetch_fsmap learned FSMap version 49 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.885+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c946bed1f0 con 0x55c946e47c00 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.885+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c946d94ad0 con 0x55c946e47c00 2023-10-06T21:42:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.885+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fe4042030 con 0x55c946e47c00 2023-10-06T21:42:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 10 client.27520 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:42:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 10 client.27520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 20 client.27520 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946eb7c30 0x55c946eba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 4 client.27520 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:42:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c946be7f10 con 0x55c946eb7c30 2023-10-06T21:42:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.886+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946eb7c30 0x55c946eba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.887+0000 7f0fef7fe700 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946eb7c30 0x55c946eba020 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c6895003aa585c server_cookie=6cd44d1b18d4d2ae in_seq=0 out_seq=0 2023-10-06T21:42:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.887+0000 7f0fceffd700 10 client.27520 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f0fceffd700 1 -- 172.21.15.31:0/1756279472 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c946be7f10 con 0x55c946eb7c30 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f0fceffd700 10 client.27520 handle_command_reply: tid=0 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 client.27520 shutdown 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 msgr2=0x55c946e59b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4012db0 tx=0x7f0fe400e350 comp rx=0 tx=0).stop 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 shutdown_connections 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c946e533d0 0x55c946e57f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c946e47c00 0x55c946e59b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c946eb7c30 0x55c946eba020 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 --2- 172.21.15.31:0/1756279472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c946e48520 0x55c946e5a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 >> 172.21.15.31:0/1756279472 conn(0x55c946dbd200 msgr2=0x55c946e4b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f0fceffd700 0 client.27520 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 shutdown_connections 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.969+0000 7f1009b80b80 1 -- 172.21.15.31:0/1756279472 wait complete. 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f1009b80b80 20 client.27520 trim_cache size 0 max 16384 2023-10-06T21:42:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 msgr2=0x7f0fd80a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 0x7f0fd80a3c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c7a0 tx=0x7f0fe8004c50 comp rx=0 tx=0).stop 2023-10-06T21:42:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 msgr2=0x7f0ff01a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 0x7f0ff01a9570 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800efc0 tx=0x7f0fe800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 shutdown_connections 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f0fd80a17c0 0x7f0fd80a3c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff0106f60 0x7f0ff01a9570 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00682d0 0x7f0ff0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.970+0000 7f0ff61ff700 1 --2- 172.21.15.31:0/3460001444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ff0068c90 0x7f0ff010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.971+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 >> 172.21.15.31:0/3460001444 conn(0x7f0ff0076b70 msgr2=0x7f0ff00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.971+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 shutdown_connections 2023-10-06T21:42:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:36.971+0000 7f0ff61ff700 1 -- 172.21.15.31:0/3460001444 wait complete. 2023-10-06T21:42:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:37 smithi031 ceph-mon[98539]: pgmap v686: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:37 smithi146 ceph-mon[102809]: pgmap v686: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:37 smithi142 ceph-mon[102951]: pgmap v686: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:37.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60223 6.36 MB/sec execute 386 sec latency 115.356 ms 2023-10-06T21:42:38.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60339 6.35 MB/sec execute 387 sec latency 71.455 ms 2023-10-06T21:42:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:39 smithi031 ceph-mon[98539]: pgmap v687: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:39 smithi146 ceph-mon[102809]: pgmap v687: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:39 smithi142 ceph-mon[102951]: pgmap v687: 97 pgs: 97 active+clean; 1009 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 181 op/s 2023-10-06T21:42:39.786 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60462 6.35 MB/sec execute 388 sec latency 66.167 ms 2023-10-06T21:42:40.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:40 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:42:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:40 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:42:40.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:40 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:42:40.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60581 6.34 MB/sec execute 389 sec latency 61.043 ms 2023-10-06T21:42:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:41 smithi031 ceph-mon[98539]: pgmap v688: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 292 op/s 2023-10-06T21:42:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:41 smithi146 ceph-mon[102809]: pgmap v688: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 292 op/s 2023-10-06T21:42:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:41 smithi142 ceph-mon[102951]: pgmap v688: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 292 op/s 2023-10-06T21:42:41.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60730 6.35 MB/sec execute 390 sec latency 68.290 ms 2023-10-06T21:42:42.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 60854 6.35 MB/sec execute 391 sec latency 85.055 ms 2023-10-06T21:42:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:43 smithi031 ceph-mon[98539]: pgmap v689: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2023-10-06T21:42:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:43 smithi146 ceph-mon[102809]: pgmap v689: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2023-10-06T21:42:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:43 smithi142 ceph-mon[102951]: pgmap v689: 97 pgs: 97 active+clean; 1006 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2023-10-06T21:42:43.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61002 6.35 MB/sec execute 392 sec latency 77.312 ms 2023-10-06T21:42:44.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61105 6.34 MB/sec execute 393 sec latency 85.565 ms 2023-10-06T21:42:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:45 smithi146 ceph-mon[102809]: pgmap v690: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2023-10-06T21:42:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:45 smithi142 ceph-mon[102951]: pgmap v690: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2023-10-06T21:42:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:45 smithi031 ceph-mon[98539]: pgmap v690: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2023-10-06T21:42:45.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61225 6.35 MB/sec execute 394 sec latency 88.413 ms 2023-10-06T21:42:46.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61356 6.35 MB/sec execute 395 sec latency 71.899 ms 2023-10-06T21:42:47.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:47 smithi146 ceph-mon[102809]: pgmap v691: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:47 smithi142 ceph-mon[102951]: pgmap v691: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:47 smithi031 ceph-mon[98539]: pgmap v691: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:47.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61510 6.37 MB/sec execute 396 sec latency 81.201 ms 2023-10-06T21:42:48.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61654 6.37 MB/sec execute 397 sec latency 80.061 ms 2023-10-06T21:42:49.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:49 smithi146 ceph-mon[102809]: pgmap v692: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:49 smithi142 ceph-mon[102951]: pgmap v692: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:49 smithi031 ceph-mon[98539]: pgmap v692: 97 pgs: 97 active+clean; 1016 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2023-10-06T21:42:49.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61786 6.38 MB/sec execute 398 sec latency 73.814 ms 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 -- 172.21.15.31:0/997929954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee30101a80 msgr2=0x7fee30105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 --2- 172.21.15.31:0/997929954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee30101a80 0x7fee30105ac0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee20009990 tx=0x7fee20009ca0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 -- 172.21.15.31:0/997929954 shutdown_connections 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 --2- 172.21.15.31:0/997929954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301061f0 0x7fee3010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 --2- 172.21.15.31:0/997929954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee301010d0 0x7fee301014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 --2- 172.21.15.31:0/997929954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee30101a80 0x7fee30105ac0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.229+0000 7fee379ff700 1 -- 172.21.15.31:0/997929954 >> 172.21.15.31:0/997929954 conn(0x7fee300fc950 msgr2=0x7fee300fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.230+0000 7fee379ff700 1 -- 172.21.15.31:0/997929954 shutdown_connections 2023-10-06T21:42:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.230+0000 7fee379ff700 1 -- 172.21.15.31:0/997929954 wait complete. 2023-10-06T21:42:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.231+0000 7fee379ff700 1 Processor -- start 2023-10-06T21:42:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.231+0000 7fee379ff700 1 -- start start 2023-10-06T21:42:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee301061f0 0x7fee30199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee3019ddd0 0x7fee3019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019e850 con 0x7fee3019ddd0 2023-10-06T21:42:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019e9c0 con 0x7fee301061f0 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee379ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019eb80 con 0x7fee301010d0 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee3579b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee3579b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:38060/0 (socket says 172.21.15.31:38060) 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.232+0000 7fee3579b700 1 -- 172.21.15.31:0/329485679 learned_addr learned my addr 172.21.15.31:0/329485679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee34f9a700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee301061f0 0x7fee30199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee35f9c700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee3019ddd0 0x7fee3019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 -- 172.21.15.31:0/329485679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee301061f0 msgr2=0x7fee30199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee301061f0 0x7fee30199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 -- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee3019ddd0 msgr2=0x7fee3019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee3019ddd0 0x7fee3019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 -- 172.21.15.31:0/329485679 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200095f0 con 0x7fee301010d0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.233+0000 7fee3579b700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00ef10 tx=0x7fee2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.234+0000 7fee267fc700 1 -- 172.21.15.31:0/329485679 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00cad0 con 0x7fee301010d0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.234+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee301a6080 con 0x7fee301010d0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.234+0000 7fee267fc700 1 -- 172.21.15.31:0/329485679 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee2c00f040 con 0x7fee301010d0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.234+0000 7fee267fc700 1 -- 172.21.15.31:0/329485679 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0146e0 con 0x7fee301010d0 2023-10-06T21:42:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.234+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee301a65b0 con 0x7fee301010d0 2023-10-06T21:42:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.239+0000 7fee267fc700 1 -- 172.21.15.31:0/329485679 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fee2c007500 con 0x7fee301010d0 2023-10-06T21:42:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.240+0000 7fee4b360b80 1 Processor -- start 2023-10-06T21:42:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 -- start start 2023-10-06T21:42:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050cfbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 0x565050d0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050c65400 con 0x565050cfc520 2023-10-06T21:42:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050c56bc0 con 0x565050d0fbe0 2023-10-06T21:42:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee4b360b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050c544e0 con 0x565050cfbc00 2023-10-06T21:42:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.241+0000 7fee267fc700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 0x7fee1c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43388/0 (socket says 172.21.15.31:43388) 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 -- 172.21.15.31:0/2968578057 learned_addr learned my addr 172.21.15.31:0/2968578057 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee267fc700 1 -- 172.21.15.31:0/329485679 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee2c0cd620 con 0x7fee301010d0 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee3579b700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 0x565050d0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee35f9c700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050cfbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 -- 172.21.15.31:0/2968578057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 msgr2=0x565050cfbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050cfbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 -- 172.21.15.31:0/2968578057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 msgr2=0x565050d0f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 0x565050d0f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee34f9a700 1 -- 172.21.15.31:0/2968578057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565050a7ad80 con 0x565050d0fbe0 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee3579b700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 0x565050d0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.242+0000 7fee35f9c700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 0x7fee1c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.243+0000 7fee34f9a700 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee3006a540 tx=0x7fee20009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183d0793bc85474c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.243+0000 7fee1b7fe700 1 -- 172.21.15.31:0/2968578057 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200056a0 con 0x565050d0fbe0 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.243+0000 7fee1b7fe700 1 -- 172.21.15.31:0/2968578057 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee2001e040 con 0x565050d0fbe0 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.243+0000 7fee1b7fe700 1 -- 172.21.15.31:0/2968578057 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20004990 con 0x565050d0fbe0 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 -- 172.21.15.31:0/2968578057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 msgr2=0x565050d11fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee35f9c700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 0x7fee1c0a3b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee28011520 tx=0x7fee28009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee3006a540 tx=0x7fee20009570 comp rx=0 tx=0).stop 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 -- 172.21.15.31:0/2968578057 shutdown_connections 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d11fd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050cfbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 --2- 172.21.15.31:0/2968578057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfc520 0x565050d0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.244+0000 7fee4b360b80 1 -- 172.21.15.31:0/2968578057 >> 172.21.15.31:0/2968578057 conn(0x565050c71200 msgr2=0x565050cf8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.245+0000 7fee4b360b80 1 -- 172.21.15.31:0/2968578057 shutdown_connections 2023-10-06T21:42:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.245+0000 7fee4b360b80 1 -- 172.21.15.31:0/2968578057 wait complete. 2023-10-06T21:42:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.245+0000 7fee4b360b80 1 -- start start 2023-10-06T21:42:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 Processor -- start 2023-10-06T21:42:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d0cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050c4fdd0 con 0x565050d0fbe0 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050bec6b0 con 0x565050cfc520 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050bec350 con 0x565050cfbc00 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee4b360b80 10 client.0 _ll_register_callbacks cb 0x565050a497a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee3579b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee3579b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:38086/0 (socket says 172.21.15.31:38086) 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.246+0000 7fee3579b700 1 -- 172.21.15.31:0/3574567044 learned_addr learned my addr 172.21.15.31:0/3574567044 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee35f9c700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee34f9a700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d0cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee3579b700 1 -- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 msgr2=0x565050d0cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee3579b700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d0cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee3579b700 1 -- 172.21.15.31:0/3574567044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 msgr2=0x565050d0d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee3579b700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d0d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee3579b700 1 -- 172.21.15.31:0/3574567044 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee300732a0 con 0x565050cfbc00 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.247+0000 7fee34f9a700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d0cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee3579b700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0cad00 tx=0x7fee2c09a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e731a1853a0c787 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c09a990 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee2c09a530 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565050cefd50 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee2c09e680 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee4b360b80 10 client.27532 fetch_fsmap learned FSMap version 50 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.248+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x565050a63c00 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565050cefd50 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fee2c0a2430 con 0x565050cfbc00 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 10 client.27532 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:42:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 10 client.27532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 20 client.27532 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d1f750 0x565050d21b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 4 client.27532 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:42:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565050c48db0 con 0x565050d1f750 2023-10-06T21:42:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.249+0000 7fee34f9a700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d1f750 0x565050d21b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.250+0000 7fee18ff9700 10 client.27532 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.250+0000 7fee34f9a700 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d1f750 0x565050d21b70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4b408a5bc846a1 server_cookie=e617144d52130368 in_seq=0 out_seq=0 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee18ff9700 1 -- 172.21.15.31:0/3574567044 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565050c48db0 con 0x565050d1f750 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee18ff9700 10 client.27532 handle_command_reply: tid=0 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 client.27532 shutdown 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 msgr2=0x565050d0ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0cad00 tx=0x7fee2c09a120 comp rx=0 tx=0).stop 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 shutdown_connections 2023-10-06T21:42:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d1f750 0x565050d21b70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d0ca00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee18ff9700 0 client.27532 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 --2- 172.21.15.31:0/3574567044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d0cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.251+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 >> 172.21.15.31:0/3574567044 conn(0x565050c71200 msgr2=0x565050cf7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.252+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 shutdown_connections 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.252+0000 7fee4b360b80 1 -- 172.21.15.31:0/3574567044 wait complete. 2023-10-06T21:42:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.252+0000 7fee4b360b80 20 client.27532 trim_cache size 0 max 16384 2023-10-06T21:42:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 Processor -- start 2023-10-06T21:42:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 -- start start 2023-10-06T21:42:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfbc00 0x565050d0d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfc520 0x565050d1b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050ad08c0 con 0x565050cfbc00 2023-10-06T21:42:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050d088e0 con 0x565050d0fbe0 2023-10-06T21:42:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.276+0000 7fee4b360b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050d12220 con 0x565050cfc520 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee3579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfbc00 0x565050d0d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43410/0 (socket says 172.21.15.31:43410) 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 -- 172.21.15.31:0/1551667643 learned_addr learned my addr 172.21.15.31:0/1551667643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee34f9a700 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfc520 0x565050d1b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 -- 172.21.15.31:0/1551667643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfc520 msgr2=0x565050d1b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfc520 0x565050d1b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 -- 172.21.15.31:0/1551667643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfbc00 msgr2=0x565050d0d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfbc00 0x565050d0d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.277+0000 7fee35f9c700 1 -- 172.21.15.31:0/1551667643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565050aa11f0 con 0x565050d0fbe0 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.278+0000 7fee35f9c700 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee2800f6b0 tx=0x7fee28009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31516076e60a77e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.278+0000 7fee1bfff700 1 -- 172.21.15.31:0/1551667643 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2800fd70 con 0x565050d0fbe0 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.278+0000 7fee1bfff700 1 -- 172.21.15.31:0/1551667643 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee28022890 con 0x565050d0fbe0 2023-10-06T21:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.278+0000 7fee1bfff700 1 -- 172.21.15.31:0/1551667643 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2802a730 con 0x565050d0fbe0 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 -- 172.21.15.31:0/1551667643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 msgr2=0x565050d07d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee2800f6b0 tx=0x7fee28009710 comp rx=0 tx=0).stop 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 -- 172.21.15.31:0/1551667643 shutdown_connections 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050d0fbe0 0x565050d07d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050cfbc00 0x565050d0d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1551667643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfc520 0x565050d1b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 -- 172.21.15.31:0/1551667643 >> 172.21.15.31:0/1551667643 conn(0x565050c71200 msgr2=0x565050cf67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 -- 172.21.15.31:0/1551667643 shutdown_connections 2023-10-06T21:42:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.279+0000 7fee4b360b80 1 -- 172.21.15.31:0/1551667643 wait complete. 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.280+0000 7fee4b360b80 1 -- start start 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.280+0000 7fee4b360b80 1 Processor -- start 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d14530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d6c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050d6ca90 con 0x565050d0fbe0 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050d6cc00 con 0x565050cfc520 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565050d6cdc0 con 0x565050cfbc00 2023-10-06T21:42:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee4b360b80 10 client.0 _ll_register_callbacks cb 0x565050a497a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee34f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee34f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d14a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43420/0 (socket says 172.21.15.31:43420) 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee34f9a700 1 -- 172.21.15.31:0/1659080529 learned_addr learned my addr 172.21.15.31:0/1659080529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee3579b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d14530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.281+0000 7fee35f9c700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d6c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 -- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 msgr2=0x565050d14a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d14a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 -- 172.21.15.31:0/1659080529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 msgr2=0x565050d6c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d6c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 -- 172.21.15.31:0/1659080529 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2800f420 con 0x565050cfbc00 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee3579b700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d14530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee2c096a30 tx=0x7fee2c0cd530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d57e64976408d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c09a990 con 0x565050cfbc00 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee2c09a530 con 0x565050cfbc00 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.282+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee2001e040 con 0x565050cfbc00 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.283+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee2c09e680 con 0x565050cfbc00 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.283+0000 7fee4b360b80 10 client.27538 fetch_fsmap learned FSMap version 50 2023-10-06T21:42:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.283+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x565050a79250 con 0x565050cfbc00 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.283+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee2001e040 con 0x565050cfbc00 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fee2c0a2c10 con 0x565050cfbc00 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 10 client.27538 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 10 client.27538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 20 client.27538 populate_metadata read hostname 'smithi031' 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d74150 0x565050d76570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 4 client.27538 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:42:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565050c48ad0 con 0x565050d74150 2023-10-06T21:42:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.284+0000 7fee34f9a700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d74150 0x565050d76570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:42:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.285+0000 7fee18ff9700 10 client.27538 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.285+0000 7fee34f9a700 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d74150 0x565050d76570 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d93ff22a880422 server_cookie=700ca2de5b00aaff in_seq=0 out_seq=0 2023-10-06T21:42:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.431+0000 7fee18ff9700 1 -- 172.21.15.31:0/1659080529 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565050c48ad0 con 0x565050d74150 2023-10-06T21:42:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.431+0000 7fee18ff9700 10 client.27538 handle_command_reply: tid=0 2023-10-06T21:42:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.431+0000 7fee4b360b80 1 client.27538 shutdown 2023-10-06T21:42:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 msgr2=0x565050d14530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d14530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee2c096a30 tx=0x7fee2c0cd530 comp rx=0 tx=0).stop 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 shutdown_connections 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x565050d74150 0x565050d76570 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565050d0fbe0 0x565050d6c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565050cfbc00 0x565050d14530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 --2- 172.21.15.31:0/1659080529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565050cfc520 0x565050d14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee18ff9700 0 client.27538 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 >> 172.21.15.31:0/1659080529 conn(0x565050c71200 msgr2=0x565050d07d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.432+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 shutdown_connections 2023-10-06T21:42:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.433+0000 7fee4b360b80 1 -- 172.21.15.31:0/1659080529 wait complete. 2023-10-06T21:42:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.433+0000 7fee4b360b80 20 client.27538 trim_cache size 0 max 16384 2023-10-06T21:42:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:42:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:42:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:42:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 msgr2=0x7fee1c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 0x7fee1c0a3b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee28011520 tx=0x7fee28009260 comp rx=0 tx=0).stop 2023-10-06T21:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 msgr2=0x7fee30199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00ef10 tx=0x7fee2c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 shutdown_connections 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee1c0a1670 0x7fee1c0a3b30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee3019ddd0 0x7fee3019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee301010d0 0x7fee30199220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 --2- 172.21.15.31:0/329485679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee301061f0 0x7fee30199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.434+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 >> 172.21.15.31:0/329485679 conn(0x7fee300fc950 msgr2=0x7fee300fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.435+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 shutdown_connections 2023-10-06T21:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:42:50.435+0000 7fee379ff700 1 -- 172.21.15.31:0/329485679 wait complete. 2023-10-06T21:42:50.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 61872 6.38 MB/sec execute 399 sec latency 87.434 ms 2023-10-06T21:42:51.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:51 smithi146 ceph-mon[102809]: pgmap v693: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2023-10-06T21:42:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:51 smithi142 ceph-mon[102951]: pgmap v693: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2023-10-06T21:42:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:51 smithi031 ceph-mon[98539]: pgmap v693: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2023-10-06T21:42:51.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62001 6.38 MB/sec execute 400 sec latency 80.641 ms 2023-10-06T21:42:52.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62115 6.38 MB/sec execute 401 sec latency 83.797 ms 2023-10-06T21:42:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:53 smithi146 ceph-mon[102809]: pgmap v694: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 178 op/s 2023-10-06T21:42:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:53 smithi142 ceph-mon[102951]: pgmap v694: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 178 op/s 2023-10-06T21:42:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:53 smithi031 ceph-mon[98539]: pgmap v694: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 178 op/s 2023-10-06T21:42:53.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62234 6.39 MB/sec execute 402 sec latency 77.540 ms 2023-10-06T21:42:54.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62335 6.38 MB/sec execute 403 sec latency 82.463 ms 2023-10-06T21:42:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:55 smithi146 ceph-mon[102809]: pgmap v695: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2023-10-06T21:42:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:55 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:42:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:55 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:55 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:55 smithi142 ceph-mon[102951]: pgmap v695: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2023-10-06T21:42:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:55 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:42:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:55 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:55 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:55 smithi031 ceph-mon[98539]: pgmap v695: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2023-10-06T21:42:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:55 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:42:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:55 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:55 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:55.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62451 6.38 MB/sec execute 404 sec latency 86.642 ms 2023-10-06T21:42:56.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:56 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:56 smithi142 ceph-mon[102951]: pgmap v696: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:56.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:56 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:56 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:56 smithi031 ceph-mon[98539]: pgmap v696: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:56 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:56.787 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62557 6.39 MB/sec execute 405 sec latency 72.390 ms 2023-10-06T21:42:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:56 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:56 smithi146 ceph-mon[102809]: pgmap v696: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:56.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:56 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:57.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62652 6.38 MB/sec execute 406 sec latency 144.709 ms 2023-10-06T21:42:57.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:57 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:57.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:57.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:57 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:57 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:42:58.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62760 6.38 MB/sec execute 407 sec latency 76.589 ms 2023-10-06T21:42:58.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:42:58 smithi146 ceph-mon[102809]: pgmap v697: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:58.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:42:58 smithi142 ceph-mon[102951]: pgmap v697: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:59.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:42:58 smithi031 ceph-mon[98539]: pgmap v697: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 170 op/s 2023-10-06T21:42:59.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62890 6.38 MB/sec execute 408 sec latency 76.699 ms 2023-10-06T21:43:00.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 62983 6.38 MB/sec execute 409 sec latency 309.224 ms 2023-10-06T21:43:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:01 smithi031 ceph-mon[98539]: pgmap v698: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 253 op/s 2023-10-06T21:43:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:01 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:01 smithi146 ceph-mon[102809]: pgmap v698: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 253 op/s 2023-10-06T21:43:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:01 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:01 smithi142 ceph-mon[102951]: pgmap v698: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 253 op/s 2023-10-06T21:43:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:01 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:01.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63093 6.37 MB/sec execute 410 sec latency 88.104 ms 2023-10-06T21:43:02.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63175 6.37 MB/sec execute 411 sec latency 298.899 ms 2023-10-06T21:43:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:03 smithi031 ceph-mon[98539]: pgmap v699: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 168 op/s 2023-10-06T21:43:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:03 smithi146 ceph-mon[102809]: pgmap v699: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 168 op/s 2023-10-06T21:43:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:03 smithi142 ceph-mon[102951]: pgmap v699: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 168 op/s 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 -- 172.21.15.31:0/3613886235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c0753e0 msgr2=0x7fe93c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 --2- 172.21.15.31:0/3613886235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c0753e0 0x7fe93c0757c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe938009990 tx=0x7fe938009ca0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 -- 172.21.15.31:0/3613886235 shutdown_connections 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 --2- 172.21.15.31:0/3613886235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 --2- 172.21.15.31:0/3613886235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c0753e0 0x7fe93c0757c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 --2- 172.21.15.31:0/3613886235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c075d00 0x7fe93c1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 -- 172.21.15.31:0/3613886235 >> 172.21.15.31:0/3613886235 conn(0x7fe93c0feb20 msgr2=0x7fe93c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 -- 172.21.15.31:0/3613886235 shutdown_connections 2023-10-06T21:43:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.683+0000 7fe942dff700 1 -- 172.21.15.31:0/3613886235 wait complete. 2023-10-06T21:43:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.685+0000 7fe942dff700 1 Processor -- start 2023-10-06T21:43:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.685+0000 7fe942dff700 1 -- start start 2023-10-06T21:43:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.685+0000 7fe942dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c0753e0 0x7fe93c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe942dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c075d00 0x7fe93c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe942dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe942dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1a15f0 con 0x7fe93c0753e0 2023-10-06T21:43:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe942dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1a1760 con 0x7fe93c111d10 2023-10-06T21:43:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe942dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1a1920 con 0x7fe93c075d00 2023-10-06T21:43:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43490/0 (socket says 172.21.15.31:43490) 2023-10-06T21:43:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 -- 172.21.15.31:0/2711670512 learned_addr learned my addr 172.21.15.31:0/2711670512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe940b9b700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c0753e0 0x7fe93c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe933fff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c075d00 0x7fe93c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 -- 172.21.15.31:0/2711670512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c075d00 msgr2=0x7fe93c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c075d00 0x7fe93c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 -- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c0753e0 msgr2=0x7fe93c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.686+0000 7fe94139c700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c0753e0 0x7fe93c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe94139c700 1 -- 172.21.15.31:0/2711670512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9380095f0 con 0x7fe93c111d10 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe94139c700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe9340094f0 tx=0x7fe93400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe931ffb700 1 -- 172.21.15.31:0/2711670512 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400ddd0 con 0x7fe93c111d10 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe931ffb700 1 -- 172.21.15.31:0/2711670512 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe93400f040 con 0x7fe93c111d10 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe931ffb700 1 -- 172.21.15.31:0/2711670512 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934005a00 con 0x7fe93c111d10 2023-10-06T21:43:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c1a1c00 con 0x7fe93c111d10 2023-10-06T21:43:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.687+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c1aa560 con 0x7fe93c111d10 2023-10-06T21:43:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.693+0000 7fe931ffb700 1 -- 172.21.15.31:0/2711670512 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe934005020 con 0x7fe93c111d10 2023-10-06T21:43:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.693+0000 7fe956821b80 1 Processor -- start 2023-10-06T21:43:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 -- start start 2023-10-06T21:43:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362998c00 0x55a362998fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362902400 con 0x55a3629a43d0 2023-10-06T21:43:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3628f3bc0 con 0x55a362999520 2023-10-06T21:43:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.694+0000 7fe956821b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3628f14e0 con 0x55a362998c00 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43502/0 (socket says 172.21.15.31:43502) 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 -- 172.21.15.31:0/3247947572 learned_addr learned my addr 172.21.15.31:0/3247947572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe933fff700 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362998c00 0x55a362998fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe931ffb700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 0x7fe9240a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe933fff700 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 -- 172.21.15.31:0/3247947572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362998c00 msgr2=0x55a362998fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362998c00 0x55a362998fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 -- 172.21.15.31:0/3247947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 msgr2=0x55a3629a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.695+0000 7fe940b9b700 1 -- 172.21.15.31:0/3247947572 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a362717d80 con 0x55a362999520 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe931ffb700 1 -- 172.21.15.31:0/2711670512 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe9340cca90 con 0x7fe93c111d10 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe94139c700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 0x7fe9240a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe938015000 tx=0x7fe938005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e45ec3c4851e99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe922ffd700 1 -- 172.21.15.31:0/3247947572 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93801d070 con 0x55a362999520 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe922ffd700 1 -- 172.21.15.31:0/3247947572 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe93801e040 con 0x55a362999520 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.696+0000 7fe922ffd700 1 -- 172.21.15.31:0/3247947572 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938004900 con 0x55a362999520 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.697+0000 7fe956821b80 1 -- 172.21.15.31:0/3247947572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 msgr2=0x55a3629a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.697+0000 7fe956821b80 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe938015000 tx=0x7fe938005f00 comp rx=0 tx=0).stop 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.697+0000 7fe94139c700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 0x7fe9240a3b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ca170 tx=0x7fe93400df30 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.697+0000 7fe956821b80 1 -- 172.21.15.31:0/3247947572 shutdown_connections 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362998c00 0x55a362998fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 --2- 172.21.15.31:0/3247947572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362999520 0x55a3629a3e90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 -- 172.21.15.31:0/3247947572 >> 172.21.15.31:0/3247947572 conn(0x55a36290e200 msgr2=0x55a362995080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 -- 172.21.15.31:0/3247947572 shutdown_connections 2023-10-06T21:43:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 -- 172.21.15.31:0/3247947572 wait complete. 2023-10-06T21:43:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.698+0000 7fe956821b80 1 -- start start 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 Processor -- start 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a3629a1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a3629a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3628ecdd0 con 0x55a362999520 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3628896b0 con 0x55a362998c00 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362889350 con 0x55a3629a43d0 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.699+0000 7fe956821b80 10 client.0 _ll_register_callbacks cb 0x55a3626e67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43506/0 (socket says 172.21.15.31:43506) 2023-10-06T21:43:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 -- 172.21.15.31:0/600342056 learned_addr learned my addr 172.21.15.31:0/600342056 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a3629a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe940b9b700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a3629a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 -- 172.21.15.31:0/600342056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 msgr2=0x55a3629a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a3629a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 -- 172.21.15.31:0/600342056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 msgr2=0x55a3629a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.700+0000 7fe933fff700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a3629a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe933fff700 1 -- 172.21.15.31:0/600342056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c114bf0 con 0x55a362998c00 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe933fff700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011e20 tx=0x7fe92c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2471828a1e9d0292 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe91bfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0109c0 con 0x55a362998c00 2023-10-06T21:43:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a36298cd50 con 0x55a362998c00 2023-10-06T21:43:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.701+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe92c010b20 con 0x55a362998c00 2023-10-06T21:43:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.702+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c010ce0 con 0x55a362998c00 2023-10-06T21:43:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.702+0000 7fe956821b80 10 client.27611 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.702+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a362700c00 con 0x55a362998c00 2023-10-06T21:43:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.702+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a36298cd50 con 0x55a362998c00 2023-10-06T21:43:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.702+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe92c003a00 con 0x55a362998c00 2023-10-06T21:43:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 10 client.27611 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 10 client.27611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 20 client.27611 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a3629b3dd0 0x55a3629b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 4 client.27611 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3628e5db0 con 0x55a3629b3dd0 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.703+0000 7fe940b9b700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a3629b3dd0 0x55a3629b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.704+0000 7fe940b9b700 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a3629b3dd0 0x55a3629b61f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871cbb76c23e29e3 server_cookie=9ba6803d31c2f57b in_seq=0 out_seq=0 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.704+0000 7fe91bfff700 10 client.27611 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe91bfff700 1 -- 172.21.15.31:0/600342056 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3628e5db0 con 0x55a3629b3dd0 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe91bfff700 10 client.27611 handle_command_reply: tid=0 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 client.27611 shutdown 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 msgr2=0x55a3629a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011e20 tx=0x7fe92c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 shutdown_connections 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a3629a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a3d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a3629b3dd0 0x55a3629b61f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 --2- 172.21.15.31:0/600342056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a3629a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe91bfff700 0 client.27611 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 >> 172.21.15.31:0/600342056 conn(0x55a36290e200 msgr2=0x55a362994c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.705+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 shutdown_connections 2023-10-06T21:43:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.706+0000 7fe956821b80 1 -- 172.21.15.31:0/600342056 wait complete. 2023-10-06T21:43:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.706+0000 7fe956821b80 20 client.27611 trim_cache size 0 max 16384 2023-10-06T21:43:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.730+0000 7fe956821b80 1 Processor -- start 2023-10-06T21:43:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.730+0000 7fe956821b80 1 -- start start 2023-10-06T21:43:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362999520 0x55a3629aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36276d8c0 con 0x55a3629a43d0 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3629ab450 con 0x55a362998c00 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe956821b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36299f590 con 0x55a362999520 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe933fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe933fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43512/0 (socket says 172.21.15.31:43512) 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe933fff700 1 -- 172.21.15.31:0/3620556998 learned_addr learned my addr 172.21.15.31:0/3620556998 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.731+0000 7fe933fff700 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362999520 0x55a3629aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 -- 172.21.15.31:0/3620556998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362999520 msgr2=0x55a3629aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362999520 0x55a3629aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 -- 172.21.15.31:0/3620556998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 msgr2=0x55a3629a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 -- 172.21.15.31:0/3620556998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a36273e1f0 con 0x55a362998c00 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.732+0000 7fe933fff700 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe92c001eb0 tx=0x7fe92c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1522bd58e5c202 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.733+0000 7fe9237fe700 1 -- 172.21.15.31:0/3620556998 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0095c0 con 0x55a362998c00 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.733+0000 7fe9237fe700 1 -- 172.21.15.31:0/3620556998 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe92c018070 con 0x55a362998c00 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.733+0000 7fe9237fe700 1 -- 172.21.15.31:0/3620556998 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c003f80 con 0x55a362998c00 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 -- 172.21.15.31:0/3620556998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 msgr2=0x55a3629a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe92c001eb0 tx=0x7fe92c004750 comp rx=0 tx=0).stop 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 -- 172.21.15.31:0/3620556998 shutdown_connections 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3629a43d0 0x55a3629a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a3629a15f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 --2- 172.21.15.31:0/3620556998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a362999520 0x55a3629aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 -- 172.21.15.31:0/3620556998 >> 172.21.15.31:0/3620556998 conn(0x55a36290e200 msgr2=0x55a3629bab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 -- 172.21.15.31:0/3620556998 shutdown_connections 2023-10-06T21:43:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.734+0000 7fe956821b80 1 -- 172.21.15.31:0/3620556998 wait complete. 2023-10-06T21:43:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.735+0000 7fe956821b80 1 -- start start 2023-10-06T21:43:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.735+0000 7fe956821b80 1 Processor -- start 2023-10-06T21:43:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.735+0000 7fe956821b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a36299d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.735+0000 7fe956821b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe956821b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a36299dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe956821b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36299e470 con 0x55a362999520 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe956821b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36299e5e0 con 0x55a362998c00 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe956821b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36299e7a0 con 0x55a3629a43d0 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe956821b80 10 client.0 _ll_register_callbacks cb 0x7fe92c011b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe940b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe940b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41066/0 (socket says 172.21.15.31:41066) 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe940b9b700 1 -- 172.21.15.31:0/3624122892 learned_addr learned my addr 172.21.15.31:0/3624122892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe933fff700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a36299dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.736+0000 7fe933fff700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a36299d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.737+0000 7fe940b9b700 1 -- 172.21.15.31:0/3624122892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 msgr2=0x55a36299dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.737+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a36299dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.737+0000 7fe940b9b700 1 -- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 msgr2=0x55a36299d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.737+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a36299d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.737+0000 7fe940b9b700 1 -- 172.21.15.31:0/3624122892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a362700c00 con 0x55a362999520 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.738+0000 7fe933fff700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a36299d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.738+0000 7fe940b9b700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe9380056f0 tx=0x7fe938004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87d24685dd0ca23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.738+0000 7fe91bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.738+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938032070 con 0x55a362999520 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.738+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe9380344e0 con 0x55a362999520 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.739+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe93801e040 con 0x55a362999520 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.739+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe93801d040 con 0x55a362999520 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.739+0000 7fe956821b80 10 client.17880 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.739+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a362716250 con 0x55a362999520 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe93801e040 con 0x55a362999520 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe93800ba00 con 0x55a362999520 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 10 client.17880 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 20 client.17880 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a362a084f0 0x55a362a0a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 4 client.17880 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.740+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a3628e5ad0 con 0x55a362a084f0 2023-10-06T21:43:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.741+0000 7fe933fff700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a362a084f0 0x55a362a0a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.741+0000 7fe91bfff700 10 client.17880 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.741+0000 7fe933fff700 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a362a084f0 0x55a362a0a910 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4952e2089ba101c4 server_cookie=92619d80c3d2ebfc in_seq=0 out_seq=0 2023-10-06T21:43:03.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63282 6.36 MB/sec execute 412 sec latency 112.674 ms 2023-10-06T21:43:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe91bfff700 1 -- 172.21.15.31:0/3624122892 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a3628e5ad0 con 0x55a362a084f0 2023-10-06T21:43:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe91bfff700 10 client.17880 handle_command_reply: tid=0 2023-10-06T21:43:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 client.17880 shutdown 2023-10-06T21:43:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 msgr2=0x55a36299d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe9380056f0 tx=0x7fe938004500 comp rx=0 tx=0).stop 2023-10-06T21:43:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 shutdown_connections 2023-10-06T21:43:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3629a43d0 0x55a36299dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a362998c00 0x55a36299d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a362a084f0 0x55a362a0a910 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 --2- 172.21.15.31:0/3624122892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a362999520 0x55a36299d7c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 >> 172.21.15.31:0/3624122892 conn(0x55a36290e200 msgr2=0x55a3629a1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe91bfff700 0 client.17880 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.812+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 shutdown_connections 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.813+0000 7fe956821b80 1 -- 172.21.15.31:0/3624122892 wait complete. 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.813+0000 7fe956821b80 20 client.17880 trim_cache size 0 max 16384 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:43:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:43:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:43:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:43:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.814+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 msgr2=0x7fe9240a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.814+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 0x7fe9240a3b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ca170 tx=0x7fe93400df30 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.814+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 msgr2=0x7fe93c1a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.814+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe9340094f0 tx=0x7fe93400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 shutdown_connections 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe93c111d10 0x7fe93c1a0ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe9240a1670 0x7fe9240a3b30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c0753e0 0x7fe93c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 --2- 172.21.15.31:0/2711670512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe93c075d00 0x7fe93c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 >> 172.21.15.31:0/2711670512 conn(0x7fe93c0feb20 msgr2=0x7fe93c112980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 shutdown_connections 2023-10-06T21:43:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:03.815+0000 7fe942dff700 1 -- 172.21.15.31:0/2711670512 wait complete. 2023-10-06T21:43:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:43:04.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:43:04.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:04 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:43:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:04 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:43:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:04 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:43:04.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63367 6.35 MB/sec execute 413 sec latency 177.049 ms 2023-10-06T21:43:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:05 smithi031 ceph-mon[98539]: pgmap v700: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 273 op/s 2023-10-06T21:43:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:05 smithi031 ceph-mon[98539]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:43:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:05 smithi031 ceph-mon[98539]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:43:05.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:05 smithi031 ceph-mon[98539]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:43:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:05 smithi146 ceph-mon[102809]: pgmap v700: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 273 op/s 2023-10-06T21:43:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:05 smithi146 ceph-mon[102809]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:43:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:05 smithi146 ceph-mon[102809]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:43:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:05 smithi146 ceph-mon[102809]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:43:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:05 smithi142 ceph-mon[102951]: pgmap v700: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 273 op/s 2023-10-06T21:43:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:05 smithi142 ceph-mon[102951]: Adjusting osd_memory_target on smithi142 to 220.9M 2023-10-06T21:43:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:05 smithi142 ceph-mon[102951]: Unable to set osd_memory_target on smithi142 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-10-06T21:43:05.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:05 smithi142 ceph-mon[102951]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:43:05.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63470 6.35 MB/sec execute 414 sec latency 237.972 ms 2023-10-06T21:43:06.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63584 6.35 MB/sec execute 415 sec latency 153.760 ms 2023-10-06T21:43:07.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:07 smithi031 ceph-mon[98539]: pgmap v702: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:07 smithi146 ceph-mon[102809]: pgmap v702: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:07 smithi142 ceph-mon[102951]: pgmap v702: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:07.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63704 6.35 MB/sec execute 416 sec latency 66.506 ms 2023-10-06T21:43:08.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63820 6.35 MB/sec execute 417 sec latency 82.991 ms 2023-10-06T21:43:09.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:09 smithi031 ceph-mon[98539]: pgmap v703: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:09 smithi146 ceph-mon[102809]: pgmap v703: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:09 smithi142 ceph-mon[102951]: pgmap v703: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:43:09.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 63925 6.34 MB/sec execute 418 sec latency 73.515 ms 2023-10-06T21:43:10.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64039 6.34 MB/sec execute 419 sec latency 69.960 ms 2023-10-06T21:43:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:11 smithi031 ceph-mon[98539]: pgmap v704: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:11.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:11 smithi146 ceph-mon[102809]: pgmap v704: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:11.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:11 smithi142 ceph-mon[102951]: pgmap v704: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:11.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64137 6.33 MB/sec execute 420 sec latency 303.185 ms 2023-10-06T21:43:12.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64262 6.33 MB/sec execute 421 sec latency 75.771 ms 2023-10-06T21:43:13.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:13 smithi031 ceph-mon[98539]: pgmap v705: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:13 smithi146 ceph-mon[102809]: pgmap v705: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:13 smithi142 ceph-mon[102951]: pgmap v705: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-06T21:43:13.788 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64409 6.34 MB/sec execute 422 sec latency 76.718 ms 2023-10-06T21:43:14.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64514 6.33 MB/sec execute 423 sec latency 88.651 ms 2023-10-06T21:43:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:15 smithi031 ceph-mon[98539]: pgmap v706: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:15 smithi031 ceph-mon[98539]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:43:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:15 smithi146 ceph-mon[102809]: pgmap v706: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:15 smithi146 ceph-mon[102809]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:43:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:15 smithi142 ceph-mon[102951]: pgmap v706: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:15.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:15 smithi142 ceph-mon[102951]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:43:15.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64638 6.33 MB/sec execute 424 sec latency 78.555 ms 2023-10-06T21:43:16.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64779 6.34 MB/sec execute 425 sec latency 106.950 ms 2023-10-06T21:43:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 -- 172.21.15.31:0/2910635769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd010fa90 msgr2=0x7f2dd01083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/2910635769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd010fa90 0x7f2dd01083f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2db80098f0 tx=0x7f2db8009c00 comp rx=0 tx=0).stop 2023-10-06T21:43:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 -- 172.21.15.31:0/2910635769 shutdown_connections 2023-10-06T21:43:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/2910635769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd011a690 0x7f2dd011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/2910635769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 0x7f2dd0107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/2910635769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd010fa90 0x7f2dd01083f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 -- 172.21.15.31:0/2910635769 >> 172.21.15.31:0/2910635769 conn(0x7f2dd006cfa0 msgr2=0x7f2dd006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.063+0000 7f2dd4fff700 1 -- 172.21.15.31:0/2910635769 shutdown_connections 2023-10-06T21:43:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.064+0000 7f2dd4fff700 1 -- 172.21.15.31:0/2910635769 wait complete. 2023-10-06T21:43:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.065+0000 7f2dd4fff700 1 Processor -- start 2023-10-06T21:43:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.065+0000 7f2dd4fff700 1 -- start start 2023-10-06T21:43:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 0x7f2dd01a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd010fa90 0x7f2dd01a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01aafb0 con 0x7f2dd010f620 2023-10-06T21:43:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01ab120 con 0x7f2dd011a690 2023-10-06T21:43:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dd4fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01ab2e0 con 0x7f2dd010fa90 2023-10-06T21:43:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.066+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:47462/0 (socket says 172.21.15.31:47462) 2023-10-06T21:43:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 -- 172.21.15.31:0/3112044821 learned_addr learned my addr 172.21.15.31:0/3112044821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dce59c700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 0x7f2dd01a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd010fa90 0x7f2dd01a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 -- 172.21.15.31:0/3112044821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd010fa90 msgr2=0x7f2dd01a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd010fa90 0x7f2dd01a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 -- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 msgr2=0x7f2dd01a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 0x7f2dd01a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.067+0000 7f2dced9d700 1 -- 172.21.15.31:0/3112044821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009520 con 0x7f2dd011a690 2023-10-06T21:43:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dced9d700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b7b0 tx=0x7f2dc000bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dc77fe700 1 -- 172.21.15.31:0/3112044821 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000f840 con 0x7f2dd011a690 2023-10-06T21:43:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dc77fe700 1 -- 172.21.15.31:0/3112044821 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dc000fe30 con 0x7f2dd011a690 2023-10-06T21:43:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dc77fe700 1 -- 172.21.15.31:0/3112044821 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000e3b0 con 0x7f2dd011a690 2023-10-06T21:43:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd01ab5c0 con 0x7f2dd011a690 2023-10-06T21:43:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.068+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd01b2e00 con 0x7f2dd011a690 2023-10-06T21:43:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.073+0000 7f2dc77fe700 1 -- 172.21.15.31:0/3112044821 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2dc000f9a0 con 0x7f2dd011a690 2023-10-06T21:43:17.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.075+0000 7f2de89d7b80 1 Processor -- start 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 -- start start 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2dc77fe700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 0x7f2dbc0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2dc77fe700 1 -- 172.21.15.31:0/3112044821 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2dc00cce70 con 0x7f2dd011a690 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b54967c00 0x556b54967fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54972e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b548d1400 con 0x556b54967c00 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b548c2bc0 con 0x556b54968520 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.076+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b548c04e0 con 0x556b549733d0 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 0x7f2dbc0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52656/0 (socket says 172.21.15.31:52656) 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dced9d700 1 -- 172.21.15.31:0/1702215104 learned_addr learned my addr 172.21.15.31:0/1702215104 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54972e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.077+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b54967c00 0x556b54967fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 -- 172.21.15.31:0/1702215104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 msgr2=0x556b54972e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54972e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 -- 172.21.15.31:0/1702215104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b54967c00 msgr2=0x556b54967fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b54967c00 0x556b54967fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 -- 172.21.15.31:0/1702215104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b546e6d80 con 0x556b549733d0 2023-10-06T21:43:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 0x7f2dbc0a3c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2dd01a75e0 tx=0x7f2db8010c30 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dced9d700 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00a1be0 tx=0x7f2dc0099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4658960babb24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.078+0000 7f2dc4ff9700 1 -- 172.21.15.31:0/1702215104 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000fc50 con 0x556b549733d0 2023-10-06T21:43:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.079+0000 7f2dc4ff9700 1 -- 172.21.15.31:0/1702215104 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dc009a480 con 0x556b549733d0 2023-10-06T21:43:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.079+0000 7f2dc4ff9700 1 -- 172.21.15.31:0/1702215104 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00ca770 con 0x556b549733d0 2023-10-06T21:43:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1702215104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 msgr2=0x556b549757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00a1be0 tx=0x7f2dc0099990 comp rx=0 tx=0).stop 2023-10-06T21:43:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1702215104 shutdown_connections 2023-10-06T21:43:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b549733d0 0x556b549757c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b54967c00 0x556b54967fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1702215104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54972e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1702215104 >> 172.21.15.31:0/1702215104 conn(0x556b548dd200 msgr2=0x556b54964080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1702215104 shutdown_connections 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.080+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1702215104 wait complete. 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.081+0000 7f2de89d7b80 1 -- start start 2023-10-06T21:43:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.081+0000 7f2de89d7b80 1 Processor -- start 2023-10-06T21:43:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54970090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54970710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b548bbdd0 con 0x556b549733d0 2023-10-06T21:43:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b548586b0 con 0x556b54968520 2023-10-06T21:43:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b54858350 con 0x556b54967c00 2023-10-06T21:43:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.082+0000 7f2de89d7b80 10 client.0 _ll_register_callbacks cb 0x556b546b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dce59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dce59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52664/0 (socket says 172.21.15.31:52664) 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dce59c700 1 -- 172.21.15.31:0/1271227326 learned_addr learned my addr 172.21.15.31:0/1271227326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dced9d700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54970710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54970090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/1271227326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 msgr2=0x556b54972da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/1271227326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 msgr2=0x556b54970710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54970710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/1271227326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd01b2e00 con 0x556b54968520 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dced9d700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54970710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.083+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54970090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009810 tx=0x7f2db8009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be670139bf8dc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8005450 con 0x556b54968520 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2db800bb70 con 0x556b54968520 2023-10-06T21:43:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b5495bd50 con 0x556b54968520 2023-10-06T21:43:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db80055b0 con 0x556b54968520 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2de89d7b80 10 client.27629 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.084+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556b546cfc00 con 0x556b54968520 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556b5495bd50 con 0x556b54968520 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2db8026570 con 0x556b54968520 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 10 client.27629 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 10 client.27629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 20 client.27629 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b54982b70 0x556b54984f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 4 client.27629 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.085+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b548b4db0 con 0x556b54982b70 2023-10-06T21:43:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.086+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b54982b70 0x556b54984f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.086+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b54982b70 0x556b54984f90 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27123dc8a0e420d5 server_cookie=1d35d558fdbc9657 in_seq=0 out_seq=0 2023-10-06T21:43:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.086+0000 7f2db5ffb700 10 client.27629 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.087+0000 7f2db5ffb700 1 -- 172.21.15.31:0/1271227326 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b548b4db0 con 0x556b54982b70 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.087+0000 7f2db5ffb700 10 client.27629 handle_command_reply: tid=0 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.087+0000 7f2de89d7b80 1 client.27629 shutdown 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 msgr2=0x556b54970090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54970090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009810 tx=0x7f2db8009f50 comp rx=0 tx=0).stop 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 shutdown_connections 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b54982b70 0x556b54984f90 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54970710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b54972da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1271227326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54970090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 >> 172.21.15.31:0/1271227326 conn(0x556b548dd200 msgr2=0x556b54963c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2db5ffb700 0 client.27629 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 shutdown_connections 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1271227326 wait complete. 2023-10-06T21:43:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.088+0000 7f2de89d7b80 20 client.27629 trim_cache size 0 max 16384 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.112+0000 7f2de89d7b80 1 Processor -- start 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.112+0000 7f2de89d7b80 1 -- start start 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 0x556b54970690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 0x556b54979de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b5496b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b5473c8c0 con 0x556b549733d0 2023-10-06T21:43:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b5497a350 con 0x556b54967c00 2023-10-06T21:43:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b5497ec50 con 0x556b54968520 2023-10-06T21:43:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b5496b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 0x556b54979de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dced9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b5496b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39422/0 (socket says 172.21.15.31:39422) 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dced9d700 1 -- 172.21.15.31:0/1480276203 learned_addr learned my addr 172.21.15.31:0/1480276203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 0x556b54979de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52680/0 (socket says 172.21.15.31:52680) 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.113+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 0x556b54970690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 -- 172.21.15.31:0/1480276203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 msgr2=0x556b54979de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 0x556b54979de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 -- 172.21.15.31:0/1480276203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 msgr2=0x556b5496b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b5496b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 -- 172.21.15.31:0/1480276203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b5470d1f0 con 0x556b54967c00 2023-10-06T21:43:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.114+0000 7f2dce59c700 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 0x556b54970690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x556b54985570 tx=0x7f2dc8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5424f1903a4c222e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.115+0000 7f2dc57fa700 1 -- 172.21.15.31:0/1480276203 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017040 con 0x556b54967c00 2023-10-06T21:43:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.115+0000 7f2dc57fa700 1 -- 172.21.15.31:0/1480276203 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dc8025070 con 0x556b54967c00 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.115+0000 7f2dc57fa700 1 -- 172.21.15.31:0/1480276203 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8004290 con 0x556b54967c00 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1480276203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 msgr2=0x556b54970690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 0x556b54970690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x556b54985570 tx=0x7f2dc8009310 comp rx=0 tx=0).stop 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1480276203 shutdown_connections 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b5496b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54967c00 0x556b54970690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/1480276203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54968520 0x556b54979de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1480276203 >> 172.21.15.31:0/1480276203 conn(0x556b548dd200 msgr2=0x556b5497e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.116+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1480276203 shutdown_connections 2023-10-06T21:43:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.117+0000 7f2de89d7b80 1 -- 172.21.15.31:0/1480276203 wait complete. 2023-10-06T21:43:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.117+0000 7f2de89d7b80 1 -- start start 2023-10-06T21:43:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.117+0000 7f2de89d7b80 1 Processor -- start 2023-10-06T21:43:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b549795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54975dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b54976430 con 0x556b549733d0 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b549765a0 con 0x556b54968520 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b54976760 con 0x556b54967c00 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.118+0000 7f2de89d7b80 10 client.0 _ll_register_callbacks cb 0x556b546b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.119+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.119+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:47502/0 (socket says 172.21.15.31:47502) 2023-10-06T21:43:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.119+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/2043573425 learned_addr learned my addr 172.21.15.31:0/2043573425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.119+0000 7f2dced9d700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54975dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.119+0000 7f2dce59c700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b549795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/2043573425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 msgr2=0x556b549795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b549795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/2043573425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 msgr2=0x556b54975dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54975dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 -- 172.21.15.31:0/2043573425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc8009e20 con 0x556b54968520 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2dcdd9b700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556b549715d0 tx=0x7f2db8026700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7607df9125af943a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.120+0000 7f2db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8005660 con 0x556b54968520 2023-10-06T21:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dc009a480 con 0x556b54968520 2023-10-06T21:43:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2db800bb70 con 0x556b54968520 2023-10-06T21:43:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db800b930 con 0x556b54968520 2023-10-06T21:43:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2de89d7b80 10 client.27635 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.121+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556b546e5250 con 0x556b54968520 2023-10-06T21:43:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.122+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2db804e020 con 0x556b54968520 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.122+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2db8038540 con 0x556b54968520 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 10 client.27635 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 10 client.27635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 20 client.27635 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b549d7670 0x556b549d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 4 client.27635 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b548b4ad0 con 0x556b549d7670 2023-10-06T21:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.123+0000 7f2dce59c700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b549d7670 0x556b549d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.124+0000 7f2dce59c700 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b549d7670 0x556b549d9a90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d151b8ccd333b975 server_cookie=d706c5431ee9bb6b in_seq=0 out_seq=0 2023-10-06T21:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.124+0000 7f2db5ffb700 10 client.27635 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.343+0000 7f2db5ffb700 1 -- 172.21.15.31:0/2043573425 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b548b4ad0 con 0x556b549d7670 2023-10-06T21:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.343+0000 7f2db5ffb700 10 client.27635 handle_command_reply: tid=0 2023-10-06T21:43:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.343+0000 7f2de89d7b80 1 client.27635 shutdown 2023-10-06T21:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.344+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 msgr2=0x556b54979ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.344+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556b549715d0 tx=0x7f2db8026700 comp rx=0 tx=0).stop 2023-10-06T21:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 shutdown_connections 2023-10-06T21:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556b549d7670 0x556b549d9a90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b549733d0 0x556b54975dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b54967c00 0x556b549795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 --2- 172.21.15.31:0/2043573425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b54968520 0x556b54979ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 >> 172.21.15.31:0/2043573425 conn(0x556b548dd200 msgr2=0x556b54969c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2db5ffb700 0 client.27635 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 shutdown_connections 2023-10-06T21:43:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.345+0000 7f2de89d7b80 1 -- 172.21.15.31:0/2043573425 wait complete. 2023-10-06T21:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.349+0000 7f2de89d7b80 20 client.27635 trim_cache size 0 max 16384 2023-10-06T21:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:43:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 msgr2=0x7f2dbc0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 0x7f2dbc0a3c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2dd01a75e0 tx=0x7f2db8010c30 comp rx=0 tx=0).stop 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 msgr2=0x7f2dd01aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b7b0 tx=0x7f2dc000bb70 comp rx=0 tx=0).stop 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 shutdown_connections 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd011a690 0x7f2dd01aa970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2dbc0a17c0 0x7f2dbc0a3c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd010f620 0x7f2dd01a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 --2- 172.21.15.31:0/3112044821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dd010fa90 0x7f2dd01a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 >> 172.21.15.31:0/3112044821 conn(0x7f2dd006cfa0 msgr2=0x7f2dd0118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 shutdown_connections 2023-10-06T21:43:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:17.350+0000 7f2dd4fff700 1 -- 172.21.15.31:0/3112044821 wait complete. 2023-10-06T21:43:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:17 smithi031 ceph-mon[98539]: pgmap v708: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:17.550 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:17 smithi146 ceph-mon[102809]: pgmap v708: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:17 smithi142 ceph-mon[102951]: pgmap v708: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:17.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 64885 6.34 MB/sec execute 426 sec latency 137.322 ms 2023-10-06T21:43:18.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65031 6.34 MB/sec execute 427 sec latency 71.204 ms 2023-10-06T21:43:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:19 smithi031 ceph-mon[98539]: pgmap v709: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:19.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:19 smithi146 ceph-mon[102809]: pgmap v709: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:19.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:19 smithi142 ceph-mon[102951]: pgmap v709: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2023-10-06T21:43:19.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65154 6.36 MB/sec execute 428 sec latency 78.735 ms 2023-10-06T21:43:20.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65242 6.36 MB/sec execute 429 sec latency 303.811 ms 2023-10-06T21:43:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:21 smithi031 ceph-mon[98539]: pgmap v710: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:21 smithi146 ceph-mon[102809]: pgmap v710: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:21 smithi142 ceph-mon[102951]: pgmap v710: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:21.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65357 6.36 MB/sec execute 430 sec latency 86.902 ms 2023-10-06T21:43:22.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65474 6.37 MB/sec execute 431 sec latency 80.592 ms 2023-10-06T21:43:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:23 smithi031 ceph-mon[98539]: pgmap v711: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:23 smithi146 ceph-mon[102809]: pgmap v711: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:23 smithi142 ceph-mon[102951]: pgmap v711: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2023-10-06T21:43:23.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65591 6.36 MB/sec execute 432 sec latency 73.665 ms 2023-10-06T21:43:24.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65693 6.36 MB/sec execute 433 sec latency 72.331 ms 2023-10-06T21:43:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:25 smithi031 ceph-mon[98539]: pgmap v712: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2023-10-06T21:43:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:25 smithi146 ceph-mon[102809]: pgmap v712: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2023-10-06T21:43:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:25 smithi142 ceph-mon[102951]: pgmap v712: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2023-10-06T21:43:25.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65805 6.36 MB/sec execute 434 sec latency 91.321 ms 2023-10-06T21:43:26.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 65899 6.35 MB/sec execute 435 sec latency 314.358 ms 2023-10-06T21:43:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:27 smithi031 ceph-mon[98539]: pgmap v713: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 192 op/s 2023-10-06T21:43:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:27 smithi146 ceph-mon[102809]: pgmap v713: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 192 op/s 2023-10-06T21:43:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:27 smithi142 ceph-mon[102951]: pgmap v713: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 192 op/s 2023-10-06T21:43:27.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66029 6.36 MB/sec execute 436 sec latency 72.938 ms 2023-10-06T21:43:28.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66130 6.35 MB/sec execute 437 sec latency 66.968 ms 2023-10-06T21:43:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:29 smithi031 ceph-mon[98539]: pgmap v714: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:29.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:29 smithi146 ceph-mon[102809]: pgmap v714: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:29 smithi142 ceph-mon[102951]: pgmap v714: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:29.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66226 6.34 MB/sec execute 438 sec latency 71.725 ms 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.614+0000 7fbec0b9d700 1 -- 172.21.15.31:0/660720775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc073250 msgr2=0x7fbebc073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.614+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/660720775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc073250 0x7fbebc073630 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009990 tx=0x7fbeac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 -- 172.21.15.31:0/660720775 shutdown_connections 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/660720775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc111cb0 0x7fbebc1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/660720775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc073250 0x7fbebc073630 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/660720775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc111770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 -- 172.21.15.31:0/660720775 >> 172.21.15.31:0/660720775 conn(0x7fbebc0fc730 msgr2=0x7fbebc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 -- 172.21.15.31:0/660720775 shutdown_connections 2023-10-06T21:43:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.615+0000 7fbec0b9d700 1 -- 172.21.15.31:0/660720775 wait complete. 2023-10-06T21:43:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.617+0000 7fbec0b9d700 1 Processor -- start 2023-10-06T21:43:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 -- start start 2023-10-06T21:43:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 0x7fbebc19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc111cb0 0x7fbebc1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1a23e0 con 0x7fbebc111cb0 2023-10-06T21:43:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1a2550 con 0x7fbebc073250 2023-10-06T21:43:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.618+0000 7fbec0b9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1a2710 con 0x7fbebc073b70 2023-10-06T21:43:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57332/0 (socket says 172.21.15.31:57332) 2023-10-06T21:43:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebaffd700 1 -- 172.21.15.31:0/637374115 learned_addr learned my addr 172.21.15.31:0/637374115 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 0x7fbebc19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebbfff700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc111cb0 0x7fbebc1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 -- 172.21.15.31:0/637374115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 msgr2=0x7fbebc19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 -- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc111cb0 msgr2=0x7fbebc1a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc111cb0 0x7fbebc1a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.619+0000 7fbebb7fe700 1 -- 172.21.15.31:0/637374115 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0095f0 con 0x7fbebc073250 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.620+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 0x7fbebc19d400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0098e0 tx=0x7fbeac003850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.621+0000 7fbeb8ff9700 1 -- 172.21.15.31:0/637374115 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac01d070 con 0x7fbebc073250 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.621+0000 7fbeb8ff9700 1 -- 172.21.15.31:0/637374115 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbeac0041d0 con 0x7fbebc073250 2023-10-06T21:43:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.621+0000 7fbeb8ff9700 1 -- 172.21.15.31:0/637374115 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0173f0 con 0x7fbebc073250 2023-10-06T21:43:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.621+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc1a2990 con 0x7fbebc073250 2023-10-06T21:43:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.621+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc1a2c10 con 0x7fbebc073250 2023-10-06T21:43:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.628+0000 7fbeb8ff9700 1 -- 172.21.15.31:0/637374115 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbeac017550 con 0x7fbebc073250 2023-10-06T21:43:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.628+0000 7fbed56c6b80 1 Processor -- start 2023-10-06T21:43:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.629+0000 7fbed56c6b80 1 -- start start 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a27680 0x55e909a31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a32530 0x55e909a34920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909990400 con 0x55e909a26d60 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909981bc0 con 0x55e909a27680 2023-10-06T21:43:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.630+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e90997f4e0 con 0x55e909a32530 2023-10-06T21:43:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49686/0 (socket says 172.21.15.31:49686) 2023-10-06T21:43:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 -- 172.21.15.31:0/736949910 learned_addr learned my addr 172.21.15.31:0/736949910 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebaffd700 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a32530 0x55e909a34920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a27680 0x55e909a31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 -- 172.21.15.31:0/736949910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a32530 msgr2=0x55e909a34920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a32530 0x55e909a34920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 -- 172.21.15.31:0/736949910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a27680 msgr2=0x55e909a31ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a27680 0x55e909a31ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.631+0000 7fbebbfff700 1 -- 172.21.15.31:0/736949910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9097a5d80 con 0x55e909a26d60 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.632+0000 7fbebbfff700 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4008520 tx=0x7fbeb4008830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28880dead673b228 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.632+0000 7fbeb8ff9700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 0x7fbea80a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.632+0000 7fbeb1ffb700 1 -- 172.21.15.31:0/736949910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4003c00 con 0x55e909a26d60 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.633+0000 7fbeb1ffb700 1 -- 172.21.15.31:0/736949910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbeb4010040 con 0x55e909a26d60 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.633+0000 7fbeb1ffb700 1 -- 172.21.15.31:0/736949910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4012760 con 0x55e909a26d60 2023-10-06T21:43:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.633+0000 7fbeb8ff9700 1 -- 172.21.15.31:0/637374115 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbeac0ce3e0 con 0x7fbebc073250 2023-10-06T21:43:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.633+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 0x7fbea80a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 -- 172.21.15.31:0/736949910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 msgr2=0x55e909a27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4008520 tx=0x7fbeb4008830 comp rx=0 tx=0).stop 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 -- 172.21.15.31:0/736949910 shutdown_connections 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a32530 0x55e909a34920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a27140 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/736949910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a27680 0x55e909a31ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 -- 172.21.15.31:0/736949910 >> 172.21.15.31:0/736949910 conn(0x55e90999c200 msgr2=0x55e909a231b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 -- 172.21.15.31:0/736949910 shutdown_connections 2023-10-06T21:43:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbebb7fe700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 0x7fbea80a3c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbeac097df0 tx=0x7fbeac097040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.634+0000 7fbed56c6b80 1 -- 172.21.15.31:0/736949910 wait complete. 2023-10-06T21:43:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.635+0000 7fbed56c6b80 1 -- start start 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.635+0000 7fbed56c6b80 1 Processor -- start 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a26d60 0x55e909a31f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 0x55e909a2f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e90997add0 con 0x55e909a32530 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9099176b0 con 0x55e909a26d60 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909917350 con 0x55e909a27680 2023-10-06T21:43:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.636+0000 7fbed56c6b80 10 client.0 _ll_register_callbacks cb 0x55e9097747a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 0x55e909a2f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 0x55e909a2f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49690/0 (socket says 172.21.15.31:49690) 2023-10-06T21:43:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebbfff700 1 -- 172.21.15.31:0/1307122261 learned_addr learned my addr 172.21.15.31:0/1307122261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebbfff700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a26d60 0x55e909a31f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 -- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a26d60 msgr2=0x55e909a31f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a26d60 0x55e909a31f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 -- 172.21.15.31:0/1307122261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 msgr2=0x55e909a2f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 0x55e909a2f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.637+0000 7fbebaffd700 1 -- 172.21.15.31:0/1307122261 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb4007d60 con 0x55e909a27680 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.638+0000 7fbebaffd700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2f1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbea400bf60 tx=0x7fbea400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ecbe4f7dd2fcba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.638+0000 7fbe9affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.638+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400fe80 con 0x55e909a27680 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.638+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbea4014460 con 0x55e909a27680 2023-10-06T21:43:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.638+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e909a1aeb0 con 0x55e909a27680 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4014620 con 0x55e909a27680 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 10 client.27580 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e90978ec00 con 0x55e909a27680 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e909a1aeb0 con 0x55e909a27680 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbea40138c0 con 0x55e909a27680 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 10 client.27580 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 10 client.27580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 20 client.27580 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.639+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a421d0 0x55e909a445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.640+0000 7fbed56c6b80 4 client.27580 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.640+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e909973db0 con 0x55e909a421d0 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.640+0000 7fbebbfff700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a421d0 0x55e909a445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.641+0000 7fbebbfff700 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a421d0 0x55e909a445f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53ccbb81fd565fe server_cookie=1dcdc43be8e5cd91 in_seq=0 out_seq=0 2023-10-06T21:43:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.641+0000 7fbe9affd700 10 client.27580 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.642+0000 7fbe9affd700 1 -- 172.21.15.31:0/1307122261 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e909973db0 con 0x55e909a421d0 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.642+0000 7fbe9affd700 10 client.27580 handle_command_reply: tid=0 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.642+0000 7fbed56c6b80 1 client.27580 shutdown 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 msgr2=0x55e909a2f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2f1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbea400bf60 tx=0x7fbea400bf90 comp rx=0 tx=0).stop 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 shutdown_connections 2023-10-06T21:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a421d0 0x55e909a445f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a32530 0x55e909a2f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a26d60 0x55e909a31f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/1307122261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2f1f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 >> 172.21.15.31:0/1307122261 conn(0x55e90999c200 msgr2=0x55e909a22da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbe9affd700 0 client.27580 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.643+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 shutdown_connections 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.644+0000 7fbed56c6b80 1 -- 172.21.15.31:0/1307122261 wait complete. 2023-10-06T21:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.644+0000 7fbed56c6b80 20 client.27580 trim_cache size 0 max 16384 2023-10-06T21:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.670+0000 7fbed56c6b80 1 Processor -- start 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.670+0000 7fbed56c6b80 1 -- start start 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9097fb8c0 con 0x55e909a26d60 2023-10-06T21:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909a3f700 con 0x55e909a32530 2023-10-06T21:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909a39610 con 0x55e909a27680 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49706/0 (socket says 172.21.15.31:49706) 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbebbfff700 1 -- 172.21.15.31:0/3420363371 learned_addr learned my addr 172.21.15.31:0/3420363371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.671+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebaffd700 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 -- 172.21.15.31:0/3420363371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 msgr2=0x55e909a31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 -- 172.21.15.31:0/3420363371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 msgr2=0x55e909a2ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 -- 172.21.15.31:0/3420363371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9097cc1f0 con 0x55e909a26d60 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.672+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4000f80 tx=0x7fbeb4008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff347c6da105632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.673+0000 7fbeb27fc700 1 -- 172.21.15.31:0/3420363371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4004a90 con 0x55e909a26d60 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.673+0000 7fbeb27fc700 1 -- 172.21.15.31:0/3420363371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbeb4012ec0 con 0x55e909a26d60 2023-10-06T21:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.673+0000 7fbeb27fc700 1 -- 172.21.15.31:0/3420363371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4023db0 con 0x55e909a26d60 2023-10-06T21:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3420363371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 msgr2=0x55e909a390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4000f80 tx=0x7fbeb4008f60 comp rx=0 tx=0).stop 2023-10-06T21:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3420363371 shutdown_connections 2023-10-06T21:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a390d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3420363371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3420363371 >> 172.21.15.31:0/3420363371 conn(0x55e90999c200 msgr2=0x55e909a2a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3420363371 shutdown_connections 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.674+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3420363371 wait complete. 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.675+0000 7fbed56c6b80 1 -- start start 2023-10-06T21:43:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.675+0000 7fbed56c6b80 1 Processor -- start 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909a2d1b0 con 0x55e909a26d60 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909a2d320 con 0x55e909a32530 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e909a2bf20 con 0x55e909a27680 2023-10-06T21:43:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbed56c6b80 10 client.0 _ll_register_callbacks cb 0x7fbea400b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57370/0 (socket says 172.21.15.31:57370) 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.676+0000 7fbebaffd700 1 -- 172.21.15.31:0/3509987161 learned_addr learned my addr 172.21.15.31:0/3509987161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 -- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 msgr2=0x55e909a2b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 -- 172.21.15.31:0/3509987161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 msgr2=0x55e909a2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a2ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 -- 172.21.15.31:0/3509987161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb402d040 con 0x55e909a27680 2023-10-06T21:43:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbebaffd700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55e909a30830 tx=0x7fbea400fe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b97bd97ffecea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.677+0000 7fbe9affd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.678+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4004240 con 0x55e909a27680 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.678+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fbea4013e90 con 0x55e909a27680 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.678+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb4010040 con 0x55e909a27680 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.679+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea40043a0 con 0x55e909a27680 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.679+0000 7fbed56c6b80 10 client.27586 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.679+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e9097a4250 con 0x55e909a27680 2023-10-06T21:43:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.679+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea403d080 con 0x55e909a27680 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbea4003840 con 0x55e909a27680 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 10 client.27586 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 10 client.27586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 20 client.27586 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a97180 0x55e909a995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 4 client.27586 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.680+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e909973ad0 con 0x55e909a97180 2023-10-06T21:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.681+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a97180 0x55e909a995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.682+0000 7fbebbfff700 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a97180 0x55e909a995a0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7b987006bc0f40 server_cookie=110894e2d0ab85be in_seq=0 out_seq=0 2023-10-06T21:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.682+0000 7fbe9affd700 10 client.27586 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.740+0000 7fbe9affd700 1 -- 172.21.15.31:0/3509987161 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e909973ad0 con 0x55e909a97180 2023-10-06T21:43:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.740+0000 7fbe9affd700 10 client.27586 handle_command_reply: tid=0 2023-10-06T21:43:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.740+0000 7fbed56c6b80 1 client.27586 shutdown 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 msgr2=0x55e909a2b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55e909a30830 tx=0x7fbea400fe90 comp rx=0 tx=0).stop 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 shutdown_connections 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e909a32530 0x55e909a2b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55e909a97180 0x55e909a995a0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e909a26d60 0x55e909a2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbe9affd700 0 client.27586 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 --2- 172.21.15.31:0/3509987161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e909a27680 0x55e909a2b470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 >> 172.21.15.31:0/3509987161 conn(0x55e90999c200 msgr2=0x55e909a2ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 shutdown_connections 2023-10-06T21:43:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.741+0000 7fbed56c6b80 1 -- 172.21.15.31:0/3509987161 wait complete. 2023-10-06T21:43:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbed56c6b80 20 client.27586 trim_cache size 0 max 16384 2023-10-06T21:43:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:43:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:43:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:43:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:43:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 msgr2=0x7fbea80a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 0x7fbea80a3c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbeac097df0 tx=0x7fbeac097040 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 msgr2=0x7fbebc19d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 0x7fbebc19d400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0098e0 tx=0x7fbeac003850 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 shutdown_connections 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fbea80a1780 0x7fbea80a3c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbebc111cb0 0x7fbebc1a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc073250 0x7fbebc19d400 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 --2- 172.21.15.31:0/637374115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbebc073b70 0x7fbebc19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 >> 172.21.15.31:0/637374115 conn(0x7fbebc0fc730 msgr2=0x7fbebc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.742+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 shutdown_connections 2023-10-06T21:43:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:30.743+0000 7fbec0b9d700 1 -- 172.21.15.31:0/637374115 wait complete. 2023-10-06T21:43:30.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66333 6.35 MB/sec execute 439 sec latency 91.960 ms 2023-10-06T21:43:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:31 smithi031 ceph-mon[98539]: pgmap v715: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 258 op/s 2023-10-06T21:43:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:31 smithi146 ceph-mon[102809]: pgmap v715: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 258 op/s 2023-10-06T21:43:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:31 smithi142 ceph-mon[102951]: pgmap v715: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 258 op/s 2023-10-06T21:43:31.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66457 6.35 MB/sec execute 440 sec latency 81.461 ms 2023-10-06T21:43:32.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66592 6.35 MB/sec execute 441 sec latency 88.088 ms 2023-10-06T21:43:33.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:33 smithi031 ceph-mon[98539]: pgmap v716: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 152 op/s 2023-10-06T21:43:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:33 smithi146 ceph-mon[102809]: pgmap v716: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 152 op/s 2023-10-06T21:43:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:33 smithi142 ceph-mon[102951]: pgmap v716: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 152 op/s 2023-10-06T21:43:33.789 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66707 6.35 MB/sec execute 442 sec latency 68.448 ms 2023-10-06T21:43:34.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66819 6.34 MB/sec execute 443 sec latency 86.859 ms 2023-10-06T21:43:35.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:35 smithi031 ceph-mon[98539]: pgmap v717: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 257 op/s 2023-10-06T21:43:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:35 smithi146 ceph-mon[102809]: pgmap v717: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 257 op/s 2023-10-06T21:43:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:35 smithi142 ceph-mon[102951]: pgmap v717: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 257 op/s 2023-10-06T21:43:35.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 66946 6.35 MB/sec execute 444 sec latency 80.793 ms 2023-10-06T21:43:36.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67061 6.34 MB/sec execute 445 sec latency 69.091 ms 2023-10-06T21:43:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:37 smithi031 ceph-mon[98539]: pgmap v718: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:37 smithi146 ceph-mon[102809]: pgmap v718: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:37 smithi142 ceph-mon[102951]: pgmap v718: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:37.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67173 6.34 MB/sec execute 446 sec latency 70.183 ms 2023-10-06T21:43:38.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67298 6.35 MB/sec execute 447 sec latency 88.545 ms 2023-10-06T21:43:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:39 smithi031 ceph-mon[98539]: pgmap v719: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:39 smithi146 ceph-mon[102809]: pgmap v719: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:39 smithi142 ceph-mon[102951]: pgmap v719: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2023-10-06T21:43:39.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67408 6.34 MB/sec execute 448 sec latency 81.450 ms 2023-10-06T21:43:40.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67543 6.34 MB/sec execute 449 sec latency 99.988 ms 2023-10-06T21:43:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:41 smithi146 ceph-mon[102809]: pgmap v720: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 276 op/s 2023-10-06T21:43:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:41 smithi142 ceph-mon[102951]: pgmap v720: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 276 op/s 2023-10-06T21:43:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:41 smithi031 ceph-mon[98539]: pgmap v720: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 276 op/s 2023-10-06T21:43:41.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67660 6.34 MB/sec execute 450 sec latency 177.420 ms 2023-10-06T21:43:42.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67798 6.34 MB/sec execute 451 sec latency 70.743 ms 2023-10-06T21:43:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:43 smithi146 ceph-mon[102809]: pgmap v721: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 195 op/s 2023-10-06T21:43:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:43 smithi142 ceph-mon[102951]: pgmap v721: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 195 op/s 2023-10-06T21:43:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:43 smithi031 ceph-mon[98539]: pgmap v721: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 195 op/s 2023-10-06T21:43:43.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 67907 6.33 MB/sec execute 452 sec latency 74.204 ms 2023-10-06T21:43:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 -- 172.21.15.31:0/4254362023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c105e20 msgr2=0x7f441c0682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 --2- 172.21.15.31:0/4254362023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c105e20 0x7f441c0682d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f440c009990 tx=0x7f440c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 -- 172.21.15.31:0/4254362023 shutdown_connections 2023-10-06T21:43:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 --2- 172.21.15.31:0/4254362023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1038b0 0x7f441c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 --2- 172.21.15.31:0/4254362023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c105e20 0x7f441c0682d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 --2- 172.21.15.31:0/4254362023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c068810 0x7f441c068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 -- 172.21.15.31:0/4254362023 >> 172.21.15.31:0/4254362023 conn(0x7f441c076b70 msgr2=0x7f441c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 -- 172.21.15.31:0/4254362023 shutdown_connections 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.986+0000 7f4421fff700 1 -- 172.21.15.31:0/4254362023 wait complete. 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.987+0000 7f4421fff700 1 Processor -- start 2023-10-06T21:43:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 -- start start 2023-10-06T21:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c068810 0x7f441c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c105e20 0x7f441c1a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a2600 con 0x7f441c068810 2023-10-06T21:43:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a2770 con 0x7f441c1038b0 2023-10-06T21:43:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.988+0000 7f4421fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a2930 con 0x7f441c105e20 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43148/0 (socket says 172.21.15.31:43148) 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 -- 172.21.15.31:0/3986615274 learned_addr learned my addr 172.21.15.31:0/3986615274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441b7fe700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c068810 0x7f441c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441bfff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c105e20 0x7f441c1a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 -- 172.21.15.31:0/3986615274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c105e20 msgr2=0x7f441c1a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c105e20 0x7f441c1a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 -- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c068810 msgr2=0x7f441c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c068810 0x7f441c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.989+0000 7f441affd700 1 -- 172.21.15.31:0/3986615274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0095f0 con 0x7f441c1038b0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f441affd700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f441000e9a0 tx=0x7f441000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f4418ff9700 1 -- 172.21.15.31:0/3986615274 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410009790 con 0x7f441c1038b0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f4418ff9700 1 -- 172.21.15.31:0/3986615274 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4410010430 con 0x7f441c1038b0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c1a2c10 con 0x7f441c1038b0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f4418ff9700 1 -- 172.21.15.31:0/3986615274 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441000cea0 con 0x7f441c1038b0 2023-10-06T21:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.990+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c1aa450 con 0x7f441c1038b0 2023-10-06T21:43:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.996+0000 7f4418ff9700 1 -- 172.21.15.31:0/3986615274 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f441000ca10 con 0x7f441c1038b0 2023-10-06T21:43:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.997+0000 7f4435829b80 1 Processor -- start 2023-10-06T21:43:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 -- start start 2023-10-06T21:43:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cf6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 0x556ae0cf97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0c55400 con 0x556ae0cf73d0 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0c46bc0 con 0x556ae0cebc00 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.998+0000 7f4435829b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0c444e0 con 0x556ae0cec520 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43162/0 (socket says 172.21.15.31:43162) 2023-10-06T21:43:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 -- 172.21.15.31:0/2521036368 learned_addr learned my addr 172.21.15.31:0/2521036368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441b7fe700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cf6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f4418ff9700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 0x7f44040a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441affd700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 0x556ae0cf97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 -- 172.21.15.31:0/2521036368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 msgr2=0x556ae0cf6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cf6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 -- 172.21.15.31:0/2521036368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 msgr2=0x556ae0cf97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:43.999+0000 7f441bfff700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 0x556ae0cf97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f441bfff700 1 -- 172.21.15.31:0/2521036368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae0a6ad80 con 0x556ae0cebc00 2023-10-06T21:43:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f441bfff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 0x7f44040a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f441affd700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 0x556ae0cf97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f4418ff9700 1 -- 172.21.15.31:0/3986615274 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f4410014070 con 0x7f441c1038b0 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f441b7fe700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cf6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f441bfff700 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4414011780 tx=0x7f4414011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f20879c84cea718 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f4401ffb700 1 -- 172.21.15.31:0/2521036368 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e040 con 0x556ae0cebc00 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f4401ffb700 1 -- 172.21.15.31:0/2521036368 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f441400f040 con 0x556ae0cebc00 2023-10-06T21:43:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.000+0000 7f4401ffb700 1 -- 172.21.15.31:0/2521036368 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414017e50 con 0x556ae0cebc00 2023-10-06T21:43:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.001+0000 7f4435829b80 1 -- 172.21.15.31:0/2521036368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 msgr2=0x556ae0cebfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.001+0000 7f441bfff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 0x7f44040a3b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44140094f0 tx=0x7f4414021880 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.001+0000 7f4435829b80 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4414011780 tx=0x7f4414011a90 comp rx=0 tx=0).stop 2023-10-06T21:43:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 -- 172.21.15.31:0/2521036368 shutdown_connections 2023-10-06T21:43:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cf73d0 0x556ae0cf97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cebfe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 --2- 172.21.15.31:0/2521036368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cf6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 -- 172.21.15.31:0/2521036368 >> 172.21.15.31:0/2521036368 conn(0x556ae0c61200 msgr2=0x556ae0ce8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 -- 172.21.15.31:0/2521036368 shutdown_connections 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 -- 172.21.15.31:0/2521036368 wait complete. 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.002+0000 7f4435829b80 1 -- start start 2023-10-06T21:43:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.003+0000 7f4435829b80 1 Processor -- start 2023-10-06T21:43:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cf4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0c3fdd0 con 0x556ae0cec520 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0bdc6b0 con 0x556ae0cebc00 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0bdc350 con 0x556ae0cf73d0 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f4435829b80 10 client.0 _ll_register_callbacks cb 0x556ae0a397a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f441b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f441b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43176/0 (socket says 172.21.15.31:43176) 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f441b7fe700 1 -- 172.21.15.31:0/1137225403 learned_addr learned my addr 172.21.15.31:0/1137225403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f441bfff700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cf4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.004+0000 7f441affd700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 -- 172.21.15.31:0/1137225403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 msgr2=0x556ae0cf4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cf4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 -- 172.21.15.31:0/1137225403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 msgr2=0x556ae0cf4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 -- 172.21.15.31:0/1137225403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c114dd0 con 0x556ae0cebc00 2023-10-06T21:43:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441bfff700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cf4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f441b7fe700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f440c015000 tx=0x7f440c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4fe69e121c2403 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.005+0000 7f43faffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c01d070 con 0x556ae0cebc00 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f440c004500 con 0x556ae0cebc00 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ae0cdfd50 con 0x556ae0cebc00 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f440c0046c0 con 0x556ae0cebc00 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f4435829b80 10 client.27659 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.006+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f44140113a0 con 0x556ae0cebc00 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ae0cdfd50 con 0x556ae0cebc00 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f440c00f830 con 0x556ae0cebc00 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 10 client.27659 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 10 client.27659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 20 client.27659 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d06ee0 0x556ae0d09300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 4 client.27659 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ae0c38db0 con 0x556ae0d06ee0 2023-10-06T21:43:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.007+0000 7f441affd700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d06ee0 0x556ae0d09300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.009+0000 7f441affd700 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d06ee0 0x556ae0d09300 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731e84aa64b7571d server_cookie=ffce6fbf94df6608 in_seq=0 out_seq=0 2023-10-06T21:43:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.009+0000 7f43faffd700 10 client.27659 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f43faffd700 1 -- 172.21.15.31:0/1137225403 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ae0c38db0 con 0x556ae0d06ee0 2023-10-06T21:43:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f43faffd700 10 client.27659 handle_command_reply: tid=0 2023-10-06T21:43:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 client.27659 shutdown 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 msgr2=0x556ae0cf6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f440c015000 tx=0x7f440c0057e0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 shutdown_connections 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cf4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cf6e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d06ee0 0x556ae0d09300 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 --2- 172.21.15.31:0/1137225403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.010+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 >> 172.21.15.31:0/1137225403 conn(0x556ae0c61200 msgr2=0x556ae0ce7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.011+0000 7f43faffd700 0 client.27659 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.011+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 shutdown_connections 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.011+0000 7f4435829b80 1 -- 172.21.15.31:0/1137225403 wait complete. 2023-10-06T21:43:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.011+0000 7f4435829b80 20 client.27659 trim_cache size 0 max 16384 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.040+0000 7f4435829b80 1 Processor -- start 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 -- start start 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cfdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0ac08c0 con 0x556ae0cec520 2023-10-06T21:43:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0ced6c0 con 0x556ae0cebc00 2023-10-06T21:43:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f4435829b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0cfe4b0 con 0x556ae0cf73d0 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f441affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f441affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52170/0 (socket says 172.21.15.31:52170) 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.041+0000 7f441affd700 1 -- 172.21.15.31:0/3812589941 learned_addr learned my addr 172.21.15.31:0/3812589941 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441b7fe700 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cfdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441bfff700 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 -- 172.21.15.31:0/3812589941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 msgr2=0x556ae0cef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 -- 172.21.15.31:0/3812589941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 msgr2=0x556ae0cfdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cfdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 -- 172.21.15.31:0/3812589941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae0cf9990 con 0x556ae0cec520 2023-10-06T21:43:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.042+0000 7f441affd700 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f44100ca940 tx=0x7f4410007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e15d05bbbef6a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.043+0000 7f44027fc700 1 -- 172.21.15.31:0/3812589941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441009a870 con 0x556ae0cec520 2023-10-06T21:43:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.043+0000 7f44027fc700 1 -- 172.21.15.31:0/3812589941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f441009a170 con 0x556ae0cec520 2023-10-06T21:43:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.043+0000 7f44027fc700 1 -- 172.21.15.31:0/3812589941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100cf0b0 con 0x556ae0cec520 2023-10-06T21:43:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 -- 172.21.15.31:0/3812589941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 msgr2=0x556ae0cf6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f44100ca940 tx=0x7f4410007ed0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 -- 172.21.15.31:0/3812589941 shutdown_connections 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cf73d0 0x556ae0cef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cebc00 0x556ae0cfdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 --2- 172.21.15.31:0/3812589941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cec520 0x556ae0cf6950 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.044+0000 7f4435829b80 1 -- 172.21.15.31:0/3812589941 >> 172.21.15.31:0/3812589941 conn(0x556ae0c61200 msgr2=0x556ae0ce67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.045+0000 7f4435829b80 1 -- 172.21.15.31:0/3812589941 shutdown_connections 2023-10-06T21:43:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.045+0000 7f4435829b80 1 -- 172.21.15.31:0/3812589941 wait complete. 2023-10-06T21:43:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.045+0000 7f4435829b80 1 -- start start 2023-10-06T21:43:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 Processor -- start 2023-10-06T21:43:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cebc00 0x556ae0cf9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cfa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0d59d00 con 0x556ae0cebc00 2023-10-06T21:43:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0d59e40 con 0x556ae0cf73d0 2023-10-06T21:43:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0d59fb0 con 0x556ae0cec520 2023-10-06T21:43:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.046+0000 7f4435829b80 10 client.0 _ll_register_callbacks cb 0x7f440c005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:43196/0 (socket says 172.21.15.31:43196) 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 -- 172.21.15.31:0/548976449 learned_addr learned my addr 172.21.15.31:0/548976449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 -- 172.21.15.31:0/548976449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 msgr2=0x556ae0cfa490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cfa490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 -- 172.21.15.31:0/548976449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cebc00 msgr2=0x556ae0cf9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cebc00 0x556ae0cf9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.047+0000 7f441bfff700 1 -- 172.21.15.31:0/548976449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c1aa450 con 0x556ae0cf73d0 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f441bfff700 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4414012010 tx=0x7f4414017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216e3907910bf842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f43faffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e070 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f441400f040 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f441401c030 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4414030070 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f4435829b80 10 client.27665 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.048+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556ae0a53c00 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f441400f040 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4414007ab0 con 0x556ae0cf73d0 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 10 client.27665 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 10 client.27665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 20 client.27665 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d5b760 0x556ae0d5db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 4 client.27665 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:43:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.049+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ae0c38ad0 con 0x556ae0d5b760 2023-10-06T21:43:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.050+0000 7f441b7fe700 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d5b760 0x556ae0d5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.050+0000 7f43faffd700 10 client.27665 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.050+0000 7f441b7fe700 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d5b760 0x556ae0d5db80 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd5c0d6357b0801 server_cookie=9bd177ebbaecd998 in_seq=0 out_seq=0 2023-10-06T21:43:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.116+0000 7f43faffd700 1 -- 172.21.15.31:0/548976449 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ae0c38ad0 con 0x556ae0d5b760 2023-10-06T21:43:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.116+0000 7f43faffd700 10 client.27665 handle_command_reply: tid=0 2023-10-06T21:43:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.116+0000 7f4435829b80 1 client.27665 shutdown 2023-10-06T21:43:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 msgr2=0x556ae0cfa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4414012010 tx=0x7f4414017000 comp rx=0 tx=0).stop 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 shutdown_connections 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556ae0cf73d0 0x556ae0cfa9d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x556ae0d5b760 0x556ae0d5db80 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ae0cebc00 0x556ae0cf9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 --2- 172.21.15.31:0/548976449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ae0cec520 0x556ae0cfa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 >> 172.21.15.31:0/548976449 conn(0x556ae0c61200 msgr2=0x556ae0cef390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f43faffd700 0 client.27665 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.117+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 shutdown_connections 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.118+0000 7f4435829b80 1 -- 172.21.15.31:0/548976449 wait complete. 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.118+0000 7f4435829b80 20 client.27665 trim_cache size 0 max 16384 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:43:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:43:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 msgr2=0x7f44040a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 0x7f44040a3b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44140094f0 tx=0x7f4414021880 comp rx=0 tx=0).stop 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 msgr2=0x7f441c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f441000e9a0 tx=0x7f441000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 shutdown_connections 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f441c105e20 0x7f441c1a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f44040a1670 0x7f44040a3b30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c068810 0x7f441c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 --2- 172.21.15.31:0/3986615274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f441c1038b0 0x7f441c19d950 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 >> 172.21.15.31:0/3986615274 conn(0x7f441c076b70 msgr2=0x7f441c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.119+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 shutdown_connections 2023-10-06T21:43:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:44.120+0000 7f4421fff700 1 -- 172.21.15.31:0/3986615274 wait complete. 2023-10-06T21:43:44.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68018 6.33 MB/sec execute 453 sec latency 97.240 ms 2023-10-06T21:43:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:45 smithi146 ceph-mon[102809]: pgmap v722: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 284 op/s 2023-10-06T21:43:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:45 smithi142 ceph-mon[102951]: pgmap v722: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 284 op/s 2023-10-06T21:43:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:45 smithi031 ceph-mon[98539]: pgmap v722: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 284 op/s 2023-10-06T21:43:45.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68152 6.33 MB/sec execute 454 sec latency 70.112 ms 2023-10-06T21:43:46.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68272 6.34 MB/sec execute 455 sec latency 364.250 ms 2023-10-06T21:43:47.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:47 smithi146 ceph-mon[102809]: pgmap v723: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:47 smithi142 ceph-mon[102951]: pgmap v723: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:47 smithi031 ceph-mon[98539]: pgmap v723: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:47.790 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68401 6.35 MB/sec execute 456 sec latency 91.540 ms 2023-10-06T21:43:48.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68532 6.35 MB/sec execute 457 sec latency 84.088 ms 2023-10-06T21:43:49.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:49 smithi146 ceph-mon[102809]: pgmap v724: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:49 smithi142 ceph-mon[102951]: pgmap v724: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:49 smithi031 ceph-mon[98539]: pgmap v724: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-06T21:43:49.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68652 6.35 MB/sec execute 458 sec latency 80.998 ms 2023-10-06T21:43:50.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68765 6.36 MB/sec execute 459 sec latency 71.216 ms 2023-10-06T21:43:51.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:51 smithi146 ceph-mon[102809]: pgmap v725: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2023-10-06T21:43:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:51 smithi142 ceph-mon[102951]: pgmap v725: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2023-10-06T21:43:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:51 smithi031 ceph-mon[98539]: pgmap v725: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2023-10-06T21:43:51.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68876 6.36 MB/sec execute 460 sec latency 236.416 ms 2023-10-06T21:43:52.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 68985 6.36 MB/sec execute 461 sec latency 69.276 ms 2023-10-06T21:43:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:53 smithi146 ceph-mon[102809]: pgmap v726: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-10-06T21:43:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:53 smithi142 ceph-mon[102951]: pgmap v726: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-10-06T21:43:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:53 smithi031 ceph-mon[98539]: pgmap v726: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-10-06T21:43:53.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69110 6.36 MB/sec execute 462 sec latency 77.776 ms 2023-10-06T21:43:54.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69204 6.36 MB/sec execute 463 sec latency 72.854 ms 2023-10-06T21:43:55.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:55 smithi146 ceph-mon[102809]: pgmap v727: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2023-10-06T21:43:55.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:55 smithi142 ceph-mon[102951]: pgmap v727: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2023-10-06T21:43:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:55 smithi031 ceph-mon[98539]: pgmap v727: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2023-10-06T21:43:55.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69300 6.35 MB/sec execute 464 sec latency 71.993 ms 2023-10-06T21:43:56.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69411 6.35 MB/sec execute 465 sec latency 82.647 ms 2023-10-06T21:43:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 -- 172.21.15.31:0/977642842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0103300 msgr2=0x7f38e01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 --2- 172.21.15.31:0/977642842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0103300 0x7f38e01036e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098f0 tx=0x7f38d0009c00 comp rx=0 tx=0).stop 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 -- 172.21.15.31:0/977642842 shutdown_connections 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 --2- 172.21.15.31:0/977642842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e0108420 0x7f38e010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 --2- 172.21.15.31:0/977642842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0103300 0x7f38e01036e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 --2- 172.21.15.31:0/977642842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103cb0 0x7f38e0107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.353+0000 7f38e59ff700 1 -- 172.21.15.31:0/977642842 >> 172.21.15.31:0/977642842 conn(0x7f38e00feb20 msgr2=0x7f38e0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.354+0000 7f38e59ff700 1 -- 172.21.15.31:0/977642842 shutdown_connections 2023-10-06T21:43:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.354+0000 7f38e59ff700 1 -- 172.21.15.31:0/977642842 wait complete. 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.355+0000 7f38e59ff700 1 Processor -- start 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 -- start start 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103300 0x7f38e010fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e010d040 0x7f38e010d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01a9d60 con 0x7f38e010d040 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006a900 con 0x7f38e0108420 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.356+0000 7f38e59ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006aac0 con 0x7f38e0103300 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.357+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.357+0000 7f38deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103300 0x7f38e010fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.357+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55854/0 (socket says 172.21.15.31:55854) 2023-10-06T21:43:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.357+0000 7f38d7fff700 1 -- 172.21.15.31:0/3909773294 learned_addr learned my addr 172.21.15.31:0/3909773294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.357+0000 7f38df7fe700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e010d040 0x7f38e010d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 -- 172.21.15.31:0/3909773294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103300 msgr2=0x7f38e010fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103300 0x7f38e010fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 -- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e010d040 msgr2=0x7f38e010d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e010d040 0x7f38e010d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 -- 172.21.15.31:0/3909773294 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009c40 con 0x7f38e0108420 2023-10-06T21:43:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.358+0000 7f38d7fff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.359+0000 7f38e49fd700 1 -- 172.21.15.31:0/3909773294 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8009690 con 0x7f38e0108420 2023-10-06T21:43:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.359+0000 7f38e49fd700 1 -- 172.21.15.31:0/3909773294 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38c800ce50 con 0x7f38e0108420 2023-10-06T21:43:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.359+0000 7f38e49fd700 1 -- 172.21.15.31:0/3909773294 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8018420 con 0x7f38e0108420 2023-10-06T21:43:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.359+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d0009520 con 0x7f38e0108420 2023-10-06T21:43:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.359+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e006b0d0 con 0x7f38e0108420 2023-10-06T21:43:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.364+0000 7f38e49fd700 1 -- 172.21.15.31:0/3909773294 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f38c8010430 con 0x7f38e0108420 2023-10-06T21:43:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.365+0000 7f38f9369b80 1 Processor -- start 2023-10-06T21:43:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.365+0000 7f38f9369b80 1 -- start start 2023-10-06T21:43:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a38698fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a386a43d0 0x557a386a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a38602400 con 0x557a386a43d0 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a385f3bc0 con 0x557a38699520 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38f9369b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a385f14e0 con 0x557a38698c00 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38df7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a38698fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38deffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38deffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55860/0 (socket says 172.21.15.31:55860) 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38deffd700 1 -- 172.21.15.31:0/2542904067 learned_addr learned my addr 172.21.15.31:0/2542904067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.366+0000 7f38e49fd700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 0x7f38cc0a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38d7fff700 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a386a43d0 0x557a386a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38e49fd700 1 -- 172.21.15.31:0/3909773294 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f38c8014070 con 0x7f38e0108420 2023-10-06T21:43:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 -- 172.21.15.31:0/2542904067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 msgr2=0x557a38698fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a38698fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 -- 172.21.15.31:0/2542904067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a386a43d0 msgr2=0x557a386a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a386a43d0 0x557a386a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 -- 172.21.15.31:0/2542904067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a38417d80 con 0x557a38699520 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38df7fe700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 0x7f38cc0a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.367+0000 7f38deffd700 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098f0 tx=0x7f38d0005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=887f8c2b399adcfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.368+0000 7f38d5ffb700 1 -- 172.21.15.31:0/2542904067 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000f460 con 0x557a38699520 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.368+0000 7f38d5ffb700 1 -- 172.21.15.31:0/2542904067 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38d000b560 con 0x557a38699520 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.368+0000 7f38d5ffb700 1 -- 172.21.15.31:0/2542904067 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00052a0 con 0x557a38699520 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 -- 172.21.15.31:0/2542904067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 msgr2=0x557a386a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38df7fe700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 0x7f38cc0a3bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38d80076a0 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098f0 tx=0x7f38d0005b80 comp rx=0 tx=0).stop 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 -- 172.21.15.31:0/2542904067 shutdown_connections 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a386a43d0 0x557a386a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a38698fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2542904067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386a3e90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.369+0000 7f38f9369b80 1 -- 172.21.15.31:0/2542904067 >> 172.21.15.31:0/2542904067 conn(0x557a3860e200 msgr2=0x557a38695080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.370+0000 7f38f9369b80 1 -- 172.21.15.31:0/2542904067 shutdown_connections 2023-10-06T21:43:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.370+0000 7f38f9369b80 1 -- 172.21.15.31:0/2542904067 wait complete. 2023-10-06T21:43:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.370+0000 7f38f9369b80 1 -- start start 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 Processor -- start 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a386a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a385ecdd0 con 0x557a38698c00 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a385896b0 con 0x557a386a43d0 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.371+0000 7f38f9369b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a38589350 con 0x557a38699520 2023-10-06T21:43:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38f9369b80 10 client.0 _ll_register_callbacks cb 0x557a383e67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:36764/0 (socket says 172.21.15.31:36764) 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38d7fff700 1 -- 172.21.15.31:0/2792892990 learned_addr learned my addr 172.21.15.31:0/2792892990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38df7fe700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a386a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.372+0000 7f38df7fe700 1 -- 172.21.15.31:0/2792892990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 msgr2=0x557a386a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38df7fe700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38df7fe700 1 -- 172.21.15.31:0/2792892990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 msgr2=0x557a386a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38df7fe700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38df7fe700 1 -- 172.21.15.31:0/2792892990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e0110810 con 0x557a386a43d0 2023-10-06T21:43:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38d7fff700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38df7fe700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a386a1710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f38d8017c40 tx=0x7f38d8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e583bf34c2f0e03c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.373+0000 7f38beffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d800e040 con 0x557a386a43d0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a3868cd50 con 0x557a386a43d0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f38d80169f0 con 0x557a386a43d0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38d8016bb0 con 0x557a386a43d0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38f9369b80 10 client.27677 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557a38400c00 con 0x557a386a43d0 2023-10-06T21:43:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.374+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a3868cd50 con 0x557a386a43d0 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f38d802b3b0 con 0x557a386a43d0 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 10 client.27677 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 10 client.27677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 20 client.27677 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a386b3f00 0x557a386b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 4 client.27677 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:43:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a385e5db0 con 0x557a386b3f00 2023-10-06T21:43:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.375+0000 7f38deffd700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a386b3f00 0x557a386b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.376+0000 7f38deffd700 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a386b3f00 0x557a386b6320 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd29ab3100da1f server_cookie=b406a1f0eecbe884 in_seq=0 out_seq=0 2023-10-06T21:43:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.376+0000 7f38beffd700 10 client.27677 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.377+0000 7f38beffd700 1 -- 172.21.15.31:0/2792892990 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a385e5db0 con 0x557a386b3f00 2023-10-06T21:43:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.377+0000 7f38beffd700 10 client.27677 handle_command_reply: tid=0 2023-10-06T21:43:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.377+0000 7f38f9369b80 1 client.27677 shutdown 2023-10-06T21:43:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 msgr2=0x557a386a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a386a1710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f38d8017c40 tx=0x7f38d8004ad0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 shutdown_connections 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a386a1710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a386b3f00 0x557a386b6320 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38beffd700 0 client.27677 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 --2- 172.21.15.31:0/2792892990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38699520 0x557a386a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 >> 172.21.15.31:0/2792892990 conn(0x557a3860e200 msgr2=0x557a38694c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 shutdown_connections 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.378+0000 7f38f9369b80 1 -- 172.21.15.31:0/2792892990 wait complete. 2023-10-06T21:43:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.379+0000 7f38f9369b80 20 client.27677 trim_cache size 0 max 16384 2023-10-06T21:43:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.407+0000 7f38f9369b80 1 Processor -- start 2023-10-06T21:43:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.407+0000 7f38f9369b80 1 -- start start 2023-10-06T21:43:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a386af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 0x557a3869c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3846d8c0 con 0x557a38699520 2023-10-06T21:43:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a38695e50 con 0x557a386a43d0 2023-10-06T21:43:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38f9369b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3869f650 con 0x557a38698c00 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 0x557a3869c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55878/0 (socket says 172.21.15.31:55878) 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 -- 172.21.15.31:0/1457292282 learned_addr learned my addr 172.21.15.31:0/1457292282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.408+0000 7f38deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a386af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 -- 172.21.15.31:0/1457292282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 msgr2=0x557a386af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a386af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 -- 172.21.15.31:0/1457292282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 msgr2=0x557a3869c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 0x557a3869c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 -- 172.21.15.31:0/1457292282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a3843e1f0 con 0x557a386a43d0 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38d7fff700 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 0x557a3869c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38deffd700 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a386af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.409+0000 7f38df7fe700 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009f00 tx=0x7f38d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8121f9a2b692a079 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.410+0000 7f38d67fc700 1 -- 172.21.15.31:0/1457292282 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d803a070 con 0x557a386a43d0 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.410+0000 7f38d67fc700 1 -- 172.21.15.31:0/1457292282 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38d800f040 con 0x557a386a43d0 2023-10-06T21:43:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.410+0000 7f38d67fc700 1 -- 172.21.15.31:0/1457292282 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d800e040 con 0x557a386a43d0 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 -- 172.21.15.31:0/1457292282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 msgr2=0x557a3869c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009f00 tx=0x7f38d8009920 comp rx=0 tx=0).stop 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 -- 172.21.15.31:0/1457292282 shutdown_connections 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a386a43d0 0x557a3869c910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a38698c00 0x557a386af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1457292282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38699520 0x557a3869c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 -- 172.21.15.31:0/1457292282 >> 172.21.15.31:0/1457292282 conn(0x557a3860e200 msgr2=0x557a386bac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 -- 172.21.15.31:0/1457292282 shutdown_connections 2023-10-06T21:43:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.411+0000 7f38f9369b80 1 -- 172.21.15.31:0/1457292282 wait complete. 2023-10-06T21:43:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.412+0000 7f38f9369b80 1 -- start start 2023-10-06T21:43:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.412+0000 7f38f9369b80 1 Processor -- start 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a386a43d0 0x557a38706d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a386aaf30 con 0x557a38698c00 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a38707270 con 0x557a38699520 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a387073b0 con 0x557a386a43d0 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38f9369b80 10 client.0 _ll_register_callbacks cb 0x7f38d8017820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.413+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:55882/0 (socket says 172.21.15.31:55882) 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 -- 172.21.15.31:0/1539513846 learned_addr learned my addr 172.21.15.31:0/1539513846 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38deffd700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 -- 172.21.15.31:0/1539513846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a386a43d0 msgr2=0x557a38706d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a386a43d0 0x557a38706d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 -- 172.21.15.31:0/1539513846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 msgr2=0x557a386aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38d7fff700 1 -- 172.21.15.31:0/1539513846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a38417d80 con 0x557a38699520 2023-10-06T21:43:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.414+0000 7f38deffd700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:43:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.415+0000 7f38d7fff700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38c8096900 tx=0x7f38c801dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59955dbad4252b8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:43:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.415+0000 7f38beffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:43:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.415+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8098670 con 0x557a38699520 2023-10-06T21:43:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.415+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38d000b560 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.415+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f38c809ab90 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c809a490 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 10 client.27683 fetch_fsmap learned FSMap version 50 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f38e006b0d0 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38c80b6030 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f38c8099980 con 0x557a38699520 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 10 client.27683 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 10 client.27683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:43:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 20 client.27683 populate_metadata read hostname 'smithi031' 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a38708b80 0x557a3870afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 4 client.27683 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.416+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a385e5ad0 con 0x557a38708b80 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.417+0000 7f38deffd700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a38708b80 0x557a3870afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.418+0000 7f38deffd700 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a38708b80 0x557a3870afa0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cc0e41f84bb357 server_cookie=3c9a32b9ec88993a in_seq=0 out_seq=0 2023-10-06T21:43:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.418+0000 7f38beffd700 10 client.27683 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38beffd700 1 -- 172.21.15.31:0/1539513846 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a385e5ad0 con 0x557a38708b80 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38beffd700 10 client.27683 handle_command_reply: tid=0 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 client.27683 shutdown 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 msgr2=0x557a386aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38c8096900 tx=0x7f38c801dc80 comp rx=0 tx=0).stop 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 shutdown_connections 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a386a43d0 0x557a38706d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x557a38708b80 0x557a3870afa0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a38698c00 0x557a386aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 --2- 172.21.15.31:0/1539513846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a38699520 0x557a386aa620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 >> 172.21.15.31:0/1539513846 conn(0x557a3860e200 msgr2=0x557a386af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38beffd700 0 client.27683 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:43:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.496+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 shutdown_connections 2023-10-06T21:43:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.497+0000 7f38f9369b80 1 -- 172.21.15.31:0/1539513846 wait complete. 2023-10-06T21:43:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.497+0000 7f38f9369b80 20 client.27683 trim_cache size 0 max 16384 2023-10-06T21:43:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 msgr2=0x7f38cc0a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 0x7f38cc0a3bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38d80076a0 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2023-10-06T21:43:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 msgr2=0x7f38e010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800ed00 comp rx=0 tx=0).stop 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 shutdown_connections 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f38cc0a16f0 0x7f38cc0a3bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38e010d040 0x7f38e010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38e0103300 0x7f38e010fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 --2- 172.21.15.31:0/3909773294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38e0108420 0x7f38e010cb00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 >> 172.21.15.31:0/3909773294 conn(0x7f38e00feb20 msgr2=0x7f38e0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 shutdown_connections 2023-10-06T21:43:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:43:57.498+0000 7f38e59ff700 1 -- 172.21.15.31:0/3909773294 wait complete. 2023-10-06T21:43:57.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:57 smithi146 ceph-mon[102809]: pgmap v728: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:57 smithi142 ceph-mon[102951]: pgmap v728: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:57 smithi031 ceph-mon[98539]: pgmap v728: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:57.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69517 6.35 MB/sec execute 466 sec latency 79.624 ms 2023-10-06T21:43:58.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69633 6.35 MB/sec execute 467 sec latency 74.424 ms 2023-10-06T21:43:59.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:43:59 smithi146 ceph-mon[102809]: pgmap v729: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:59.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:43:59 smithi142 ceph-mon[102951]: pgmap v729: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:59.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:43:59 smithi031 ceph-mon[98539]: pgmap v729: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 178 op/s 2023-10-06T21:43:59.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69755 6.35 MB/sec execute 468 sec latency 68.093 ms 2023-10-06T21:44:00.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69869 6.35 MB/sec execute 469 sec latency 101.814 ms 2023-10-06T21:44:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:01 smithi146 ceph-mon[102809]: pgmap v730: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 268 op/s 2023-10-06T21:44:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:01 smithi142 ceph-mon[102951]: pgmap v730: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 268 op/s 2023-10-06T21:44:01.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:01 smithi031 ceph-mon[98539]: pgmap v730: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 268 op/s 2023-10-06T21:44:01.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 69971 6.34 MB/sec execute 470 sec latency 147.769 ms 2023-10-06T21:44:02.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70103 6.34 MB/sec execute 471 sec latency 73.668 ms 2023-10-06T21:44:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:03 smithi146 ceph-mon[102809]: pgmap v731: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:44:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:03 smithi142 ceph-mon[102951]: pgmap v731: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:44:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:03 smithi031 ceph-mon[98539]: pgmap v731: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 167 op/s 2023-10-06T21:44:03.791 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70195 6.34 MB/sec execute 472 sec latency 191.865 ms 2023-10-06T21:44:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:04 smithi142 ceph-mon[102951]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:44:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:04 smithi031 ceph-mon[98539]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:44:04.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70307 6.34 MB/sec execute 473 sec latency 83.671 ms 2023-10-06T21:44:04.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:04 smithi146 ceph-mon[102809]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:44:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:05 smithi142 ceph-mon[102951]: pgmap v733: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:05 smithi031 ceph-mon[98539]: pgmap v733: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:05.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70388 6.33 MB/sec execute 474 sec latency 332.361 ms 2023-10-06T21:44:05.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:05 smithi146 ceph-mon[102809]: pgmap v733: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:06 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:44:06.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:06 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:44:06.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70485 6.33 MB/sec execute 475 sec latency 225.340 ms 2023-10-06T21:44:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:06 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:44:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:07 smithi142 ceph-mon[102951]: pgmap v734: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:07.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:07 smithi031 ceph-mon[98539]: pgmap v734: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:07.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70619 6.34 MB/sec execute 476 sec latency 79.366 ms 2023-10-06T21:44:07.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:07 smithi146 ceph-mon[102809]: pgmap v734: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:08.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:08 smithi142 ceph-mon[102951]: pgmap v735: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:08.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:08 smithi031 ceph-mon[98539]: pgmap v735: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:08.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70712 6.33 MB/sec execute 477 sec latency 76.606 ms 2023-10-06T21:44:08.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:08 smithi146 ceph-mon[102809]: pgmap v735: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-06T21:44:09.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:44:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:44:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:09 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:44:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:44:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:44:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:09 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:44:09.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70836 6.33 MB/sec execute 478 sec latency 77.456 ms 2023-10-06T21:44:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:44:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:44:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:09 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:44:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:10 smithi142 ceph-mon[102951]: pgmap v736: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.724+0000 7f77949ff700 1 -- 172.21.15.31:0/1070293904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779010d700 msgr2=0x7f779010fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.724+0000 7f77949ff700 1 --2- 172.21.15.31:0/1070293904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779010d700 0x7f779010fb30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f778800b0a0 tx=0x7f778800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 -- 172.21.15.31:0/1070293904 shutdown_connections 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 --2- 172.21.15.31:0/1070293904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779010d700 0x7f779010fb30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 --2- 172.21.15.31:0/1070293904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f7790073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 --2- 172.21.15.31:0/1070293904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7790073ba0 0x7f779010d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 -- 172.21.15.31:0/1070293904 >> 172.21.15.31:0/1070293904 conn(0x7f77900fc950 msgr2=0x7f77900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 -- 172.21.15.31:0/1070293904 shutdown_connections 2023-10-06T21:44:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.725+0000 7f77949ff700 1 -- 172.21.15.31:0/1070293904 wait complete. 2023-10-06T21:44:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.726+0000 7f77949ff700 1 Processor -- start 2023-10-06T21:44:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.726+0000 7f77949ff700 1 -- start start 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f77901992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 0x7f7790199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f779010d700 0x7f779019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e4c0 con 0x7f7790073280 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e630 con 0x7f7790073ba0 2023-10-06T21:44:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f77949ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e7f0 con 0x7f779010d700 2023-10-06T21:44:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f778e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f77901992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f778dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 0x7f7790199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f778e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f77901992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41410/0 (socket says 172.21.15.31:41410) 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.727+0000 7f778e59c700 1 -- 172.21.15.31:0/1329822255 learned_addr learned my addr 172.21.15.31:0/1329822255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778ed9d700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f779010d700 0x7f779019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 -- 172.21.15.31:0/1329822255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f779010d700 msgr2=0x7f779019de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f779010d700 0x7f779019de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 -- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 msgr2=0x7f77901992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f77901992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 -- 172.21.15.31:0/1329822255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009d00 con 0x7f7790073ba0 2023-10-06T21:44:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.728+0000 7f778dd9b700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 0x7f7790199810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.729+0000 7f777f7fe700 1 -- 172.21.15.31:0/1329822255 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400c9c0 con 0x7f7790073ba0 2023-10-06T21:44:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.729+0000 7f777f7fe700 1 -- 172.21.15.31:0/1329822255 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7784010430 con 0x7f7790073ba0 2023-10-06T21:44:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.729+0000 7f777f7fe700 1 -- 172.21.15.31:0/1329822255 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784018680 con 0x7f7790073ba0 2023-10-06T21:44:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.729+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779019ead0 con 0x7f7790073ba0 2023-10-06T21:44:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.729+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77901a6310 con 0x7f7790073ba0 2023-10-06T21:44:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.734+0000 7f777f7fe700 1 -- 172.21.15.31:0/1329822255 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77840105a0 con 0x7f7790073ba0 2023-10-06T21:44:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.735+0000 7f77a841bb80 1 Processor -- start 2023-10-06T21:44:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.735+0000 7f77a841bb80 1 -- start start 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e7f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 0x55a74e809fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e75d400 con 0x55a74e7f3c00 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e74ebc0 con 0x55a74e807be0 2023-10-06T21:44:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f77a841bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e74c4e0 con 0x55a74e7f4520 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f778dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 0x55a74e809fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f778e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f778e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47424/0 (socket says 172.21.15.31:47424) 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f778e59c700 1 -- 172.21.15.31:0/3312850249 learned_addr learned my addr 172.21.15.31:0/3312850249 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f778ed9d700 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e7f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.736+0000 7f777f7fe700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 0x7f77780a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f777f7fe700 1 -- 172.21.15.31:0/1329822255 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f7784014070 con 0x7f7790073ba0 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778e59c700 1 -- 172.21.15.31:0/3312850249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 msgr2=0x55a74e809fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778e59c700 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 0x55a74e809fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778e59c700 1 -- 172.21.15.31:0/3312850249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 msgr2=0x55a74e7f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778e59c700 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e7f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778e59c700 1 -- 172.21.15.31:0/3312850249 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a74e572d80 con 0x55a74e7f4520 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778dd9b700 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 0x55a74e809fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:44:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.737+0000 7f778ed9d700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 0x7f77780a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.738+0000 7f778e59c700 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f778000bea0 tx=0x7f77800106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cfcbdc4d63d0d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.738+0000 7f777cff9700 1 -- 172.21.15.31:0/3312850249 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780010ed0 con 0x55a74e7f4520 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.738+0000 7f777cff9700 1 -- 172.21.15.31:0/3312850249 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7780015070 con 0x55a74e7f4520 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.738+0000 7f777cff9700 1 -- 172.21.15.31:0/3312850249 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780019730 con 0x55a74e7f4520 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.738+0000 7f778ed9d700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 0x7f77780a3b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788009cd0 tx=0x7f778800b3b0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 -- 172.21.15.31:0/3312850249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 msgr2=0x55a74e8076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f778000bea0 tx=0x7f77800106b0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 -- 172.21.15.31:0/3312850249 shutdown_connections 2023-10-06T21:44:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e807be0 0x55a74e809fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e7f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3312850249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f4520 0x55a74e8076a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.739+0000 7f77a841bb80 1 -- 172.21.15.31:0/3312850249 >> 172.21.15.31:0/3312850249 conn(0x55a74e769200 msgr2=0x55a74e7f0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.740+0000 7f77a841bb80 1 -- 172.21.15.31:0/3312850249 shutdown_connections 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.740+0000 7f77a841bb80 1 -- 172.21.15.31:0/3312850249 wait complete. 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.740+0000 7f77a841bb80 1 -- start start 2023-10-06T21:44:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 Processor -- start 2023-10-06T21:44:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 0x55a74e807580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e807be0 0x55a74e804e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e747dd0 con 0x55a74e807be0 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e6e46b0 con 0x55a74e7f4520 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.741+0000 7f77a841bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e6e4350 con 0x55a74e7f3c00 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f77a841bb80 10 client.0 _ll_register_callbacks cb 0x55a74e5417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 0x55a74e807580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57602/0 (socket says 172.21.15.31:57602) 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 -- 172.21.15.31:0/576235179 learned_addr learned my addr 172.21.15.31:0/576235179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778ed9d700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e807be0 0x55a74e804e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 -- 172.21.15.31:0/576235179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 msgr2=0x55a74e807580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 0x55a74e807580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 -- 172.21.15.31:0/576235179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e807be0 msgr2=0x55a74e804e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e807be0 0x55a74e804e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778dd9b700 1 -- 172.21.15.31:0/576235179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77901a6310 con 0x55a74e7f4520 2023-10-06T21:44:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.742+0000 7f778e59c700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 0x55a74e807580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:44:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f778dd9b700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7784096640 tx=0x7f7784099aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e8a51657be095b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f7775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:44:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778409a5a0 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a74e7e7d50 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f77840cf2f0 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.743+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778409a140 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 10 client.27695 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f778000bad0 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a74e7e7d50 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f77840b2020 con 0x55a74e7f4520 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 10 client.27695 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 10 client.27695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 20 client.27695 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e817640 0x55a74e819a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 4 client.27695 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:44:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.744+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a74e740db0 con 0x55a74e817640 2023-10-06T21:44:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.745+0000 7f778e59c700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e817640 0x55a74e819a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.746+0000 7f778e59c700 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e817640 0x55a74e819a60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a790c9d789acd4 server_cookie=8a4240d55e565794 in_seq=0 out_seq=0 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.746+0000 7f7775ffb700 10 client.27695 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.746+0000 7f7775ffb700 1 -- 172.21.15.31:0/576235179 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a74e740db0 con 0x55a74e817640 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f7775ffb700 10 client.27695 handle_command_reply: tid=0 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 client.27695 shutdown 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 msgr2=0x55a74e804800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7784096640 tx=0x7f7784099aa0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 shutdown_connections 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e817640 0x55a74e819a60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e807be0 0x55a74e804e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f7775ffb700 0 client.27695 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e7f3c00 0x55a74e807580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 --2- 172.21.15.31:0/576235179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e804800 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 >> 172.21.15.31:0/576235179 conn(0x55a74e769200 msgr2=0x55a74e7efc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 shutdown_connections 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.747+0000 7f77a841bb80 1 -- 172.21.15.31:0/576235179 wait complete. 2023-10-06T21:44:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.748+0000 7f77a841bb80 20 client.27695 trim_cache size 0 max 16384 2023-10-06T21:44:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:10 smithi031 ceph-mon[98539]: pgmap v736: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 Processor -- start 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 -- start start 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f4520 0x55a74e812eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e7ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e5c88c0 con 0x55a74e7f4520 2023-10-06T21:44:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e7ec8c0 con 0x55a74e7f3c00 2023-10-06T21:44:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f77a841bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e7f0e40 con 0x55a74e807be0 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.774+0000 7f778e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:57604/0 (socket says 172.21.15.31:57604) 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e7ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 -- 172.21.15.31:0/3242556715 learned_addr learned my addr 172.21.15.31:0/3242556715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778dd9b700 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f4520 0x55a74e812eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 -- 172.21.15.31:0/3242556715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 msgr2=0x55a74e7ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e7ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 -- 172.21.15.31:0/3242556715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f4520 msgr2=0x55a74e812eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f4520 0x55a74e812eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 -- 172.21.15.31:0/3242556715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a74e55bc00 con 0x55a74e7f3c00 2023-10-06T21:44:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.775+0000 7f778e59c700 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f778000bdc0 tx=0x7f778000b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17169881bc92f4ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.776+0000 7f777d7fa700 1 -- 172.21.15.31:0/3242556715 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780010cd0 con 0x55a74e7f3c00 2023-10-06T21:44:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.776+0000 7f777d7fa700 1 -- 172.21.15.31:0/3242556715 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7780005b00 con 0x55a74e7f3c00 2023-10-06T21:44:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.776+0000 7f777d7fa700 1 -- 172.21.15.31:0/3242556715 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780005400 con 0x55a74e7f3c00 2023-10-06T21:44:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 -- 172.21.15.31:0/3242556715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 msgr2=0x55a74e80e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f778000bdc0 tx=0x7f778000b7a0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 -- 172.21.15.31:0/3242556715 shutdown_connections 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e7ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f3c00 0x55a74e80e760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 --2- 172.21.15.31:0/3242556715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f4520 0x55a74e812eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 -- 172.21.15.31:0/3242556715 >> 172.21.15.31:0/3242556715 conn(0x55a74e769200 msgr2=0x55a74e8071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 -- 172.21.15.31:0/3242556715 shutdown_connections 2023-10-06T21:44:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.777+0000 7f77a841bb80 1 -- 172.21.15.31:0/3242556715 wait complete. 2023-10-06T21:44:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.778+0000 7f77a841bb80 1 -- start start 2023-10-06T21:44:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.778+0000 7f77a841bb80 1 Processor -- start 2023-10-06T21:44:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e80c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e80c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e8649f0 con 0x55a74e7f3c00 2023-10-06T21:44:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e864b60 con 0x55a74e7f4520 2023-10-06T21:44:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74e864d20 con 0x55a74e807be0 2023-10-06T21:44:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.779+0000 7f77a841bb80 10 client.0 _ll_register_callbacks cb 0x7f7784096550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:47460/0 (socket says 172.21.15.31:47460) 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 -- 172.21.15.31:0/2442320385 learned_addr learned my addr 172.21.15.31:0/2442320385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 -- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 msgr2=0x55a74e80c980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e80c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 -- 172.21.15.31:0/2442320385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 msgr2=0x55a74e80c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e80c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.780+0000 7f778ed9d700 1 -- 172.21.15.31:0/2442320385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778001e040 con 0x55a74e807be0 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.781+0000 7f778ed9d700 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7788015000 tx=0x7f77880099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab064cb4a915464b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.781+0000 7f7775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:44:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.781+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778800e040 con 0x55a74e807be0 2023-10-06T21:44:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.781+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f7788020690 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.781+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a74e740ad0 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.782+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77880078c0 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.782+0000 7f77a841bb80 10 client.27634 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.782+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a74e5991f0 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.782+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a74e740ad0 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.782+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7788028070 con 0x55a74e807be0 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 10 client.27634 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 10 client.27634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 20 client.27634 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e86c540 0x55a74e86e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 4 client.27634 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:44:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a74e593f10 con 0x55a74e86c540 2023-10-06T21:44:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.783+0000 7f778e59c700 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e86c540 0x55a74e86e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.784+0000 7f778e59c700 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e86c540 0x55a74e86e960 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a887b7402ef65 server_cookie=1951d9ffb4375fa0 in_seq=0 out_seq=0 2023-10-06T21:44:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.784+0000 7f7775ffb700 10 client.27634 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:10.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 70950 6.33 MB/sec execute 479 sec latency 91.545 ms 2023-10-06T21:44:10.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:10 smithi146 ceph-mon[102809]: pgmap v736: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.863+0000 7f7775ffb700 1 -- 172.21.15.31:0/2442320385 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a74e593f10 con 0x55a74e86c540 2023-10-06T21:44:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.863+0000 7f7775ffb700 10 client.27634 handle_command_reply: tid=0 2023-10-06T21:44:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.863+0000 7f77a841bb80 1 client.27634 shutdown 2023-10-06T21:44:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.864+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 msgr2=0x55a74e8644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.864+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7788015000 tx=0x7f77880099e0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.864+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 shutdown_connections 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a74e807be0 0x55a74e8644b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a74e86c540 0x55a74e86e960 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a74e7f3c00 0x55a74e80c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 --2- 172.21.15.31:0/2442320385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a74e7f4520 0x55a74e80c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 >> 172.21.15.31:0/2442320385 conn(0x55a74e769200 msgr2=0x55a74e7ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f7775ffb700 0 client.27634 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 shutdown_connections 2023-10-06T21:44:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 1 -- 172.21.15.31:0/2442320385 wait complete. 2023-10-06T21:44:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.865+0000 7f77a841bb80 20 client.27634 trim_cache size 0 max 16384 2023-10-06T21:44:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:44:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:44:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:44:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.866+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 msgr2=0x7f77780a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.866+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 0x7f77780a3b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788009cd0 tx=0x7f778800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.866+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 msgr2=0x7f7790199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.866+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 0x7f7790199810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400ed00 comp rx=0 tx=0).stop 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 shutdown_connections 2023-10-06T21:44:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f779010d700 0x7f779019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f77780a16b0 0x7f77780a3b70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7790073280 0x7f77901992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 --2- 172.21.15.31:0/1329822255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7790073ba0 0x7f7790199810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 >> 172.21.15.31:0/1329822255 conn(0x7f77900fc950 msgr2=0x7f779010f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.867+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 shutdown_connections 2023-10-06T21:44:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:10.868+0000 7f77949ff700 1 -- 172.21.15.31:0/1329822255 wait complete. 2023-10-06T21:44:11.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71065 6.32 MB/sec execute 480 sec latency 118.905 ms 2023-10-06T21:44:12.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:12 smithi031 ceph-mon[98539]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:12.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71189 6.32 MB/sec execute 481 sec latency 70.838 ms 2023-10-06T21:44:12.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:12 smithi146 ceph-mon[102809]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:12.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:12 smithi142 ceph-mon[102951]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 231 op/s 2023-10-06T21:44:13.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71308 6.32 MB/sec execute 482 sec latency 70.585 ms 2023-10-06T21:44:14.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71415 6.31 MB/sec execute 483 sec latency 172.389 ms 2023-10-06T21:44:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:15 smithi031 ceph-mon[98539]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2023-10-06T21:44:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:15 smithi031 ceph-mon[98539]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:44:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:15 smithi146 ceph-mon[102809]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2023-10-06T21:44:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:15 smithi146 ceph-mon[102809]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:44:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:15 smithi142 ceph-mon[102951]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2023-10-06T21:44:15.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:15 smithi142 ceph-mon[102951]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:44:15.792 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71542 6.31 MB/sec execute 484 sec latency 314.401 ms 2023-10-06T21:44:16.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71707 6.33 MB/sec execute 485 sec latency 221.831 ms 2023-10-06T21:44:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:17 smithi031 ceph-mon[98539]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:17 smithi146 ceph-mon[102809]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:17 smithi142 ceph-mon[102951]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:17.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71855 6.33 MB/sec execute 486 sec latency 62.451 ms 2023-10-06T21:44:18.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 71994 6.34 MB/sec execute 487 sec latency 71.695 ms 2023-10-06T21:44:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:19 smithi031 ceph-mon[98539]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:19 smithi146 ceph-mon[102809]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:19.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:19 smithi142 ceph-mon[102951]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2023-10-06T21:44:19.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72097 6.34 MB/sec execute 488 sec latency 70.624 ms 2023-10-06T21:44:20.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72223 6.35 MB/sec execute 489 sec latency 77.965 ms 2023-10-06T21:44:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:21 smithi031 ceph-mon[98539]: pgmap v742: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:21 smithi146 ceph-mon[102809]: pgmap v742: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:21.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:21 smithi142 ceph-mon[102951]: pgmap v742: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:21.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72370 6.36 MB/sec execute 490 sec latency 99.613 ms 2023-10-06T21:44:22.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72482 6.35 MB/sec execute 491 sec latency 72.267 ms 2023-10-06T21:44:23.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:23 smithi031 ceph-mon[98539]: pgmap v743: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:23 smithi146 ceph-mon[102809]: pgmap v743: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:23 smithi142 ceph-mon[102951]: pgmap v743: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 226 op/s 2023-10-06T21:44:23.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72605 6.36 MB/sec execute 492 sec latency 83.459 ms 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 -- 172.21.15.31:0/233435537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6720073280 msgr2=0x7f6720073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 --2- 172.21.15.31:0/233435537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6720073280 0x7f6720073660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6710009990 tx=0x7f6710009ca0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 -- 172.21.15.31:0/233435537 shutdown_connections 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 --2- 172.21.15.31:0/233435537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672010d6f0 0x7f672010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 --2- 172.21.15.31:0/233435537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6720073280 0x7f6720073660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 --2- 172.21.15.31:0/233435537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 0x7f672010d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.104+0000 7f67269ff700 1 -- 172.21.15.31:0/233435537 >> 172.21.15.31:0/233435537 conn(0x7f67200fc940 msgr2=0x7f67200fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.105+0000 7f67269ff700 1 -- 172.21.15.31:0/233435537 shutdown_connections 2023-10-06T21:44:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.105+0000 7f67269ff700 1 -- 172.21.15.31:0/233435537 wait complete. 2023-10-06T21:44:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.106+0000 7f67269ff700 1 Processor -- start 2023-10-06T21:44:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 -- start start 2023-10-06T21:44:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6720073280 0x7f6720106650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 0x7f6720106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720101cd0 con 0x7f6720073280 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720101e40 con 0x7f672010d6f0 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.107+0000 7f67269ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720102000 con 0x7f6720073ba0 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:50456/0 (socket says 172.21.15.31:50456) 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 -- 172.21.15.31:0/2890959025 learned_addr learned my addr 172.21.15.31:0/2890959025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f671f7fe700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 0x7f6720106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f671ffff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6720073280 0x7f6720106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 -- 172.21.15.31:0/2890959025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 msgr2=0x7f6720106bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 0x7f6720106bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 -- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6720073280 msgr2=0x7f6720106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6720073280 0x7f6720106650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.108+0000 7f6724f9c700 1 -- 172.21.15.31:0/2890959025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100095f0 con 0x7f672010d6f0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f6724f9c700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f671800ef90 tx=0x7f671800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f671d7fa700 1 -- 172.21.15.31:0/2890959025 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718018070 con 0x7f672010d6f0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f671d7fa700 1 -- 172.21.15.31:0/2890959025 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671800f040 con 0x7f672010d6f0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f671d7fa700 1 -- 172.21.15.31:0/2890959025 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671800cc40 con 0x7f672010d6f0 2023-10-06T21:44:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6720102280 con 0x7f672010d6f0 2023-10-06T21:44:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.109+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672010c670 con 0x7f672010d6f0 2023-10-06T21:44:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.115+0000 7f671d7fa700 1 -- 172.21.15.31:0/2890959025 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6718003a00 con 0x7f672010d6f0 2023-10-06T21:44:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.116+0000 7f673a3a1b80 1 Processor -- start 2023-10-06T21:44:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.116+0000 7f673a3a1b80 1 -- start start 2023-10-06T21:44:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f671d7fa700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 0x7f67080a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc410400 con 0x55aebc4a6c00 2023-10-06T21:44:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc401bc0 con 0x55aebc4a7520 2023-10-06T21:44:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc3ff4e0 con 0x55aebc4b23d0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f671d7fa700 1 -- 172.21.15.31:0/2890959025 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f67180ccc60 con 0x7f672010d6f0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f6724f9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f6724f9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43732/0 (socket says 172.21.15.31:43732) 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.117+0000 7f6724f9c700 1 -- 172.21.15.31:0/1611230849 learned_addr learned my addr 172.21.15.31:0/1611230849 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f671f7fe700 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f671f7fe700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 0x7f67080a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f671ffff700 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 -- 172.21.15.31:0/1611230849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 msgr2=0x55aebc4b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 -- 172.21.15.31:0/1611230849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 msgr2=0x55aebc4a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4a6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 -- 172.21.15.31:0/1611230849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aebc225d80 con 0x55aebc4b23d0 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.118+0000 7f6724f9c700 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67180090a0 tx=0x7f6718099aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbc9d99492bdb72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.119+0000 7f67067fc700 1 -- 172.21.15.31:0/1611230849 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671807d980 con 0x55aebc4b23d0 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.119+0000 7f67067fc700 1 -- 172.21.15.31:0/1611230849 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671800ce60 con 0x55aebc4b23d0 2023-10-06T21:44:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.119+0000 7f67067fc700 1 -- 172.21.15.31:0/1611230849 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718099c00 con 0x55aebc4b23d0 2023-10-06T21:44:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f671f7fe700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 0x7f67080a3bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f671400b6c0 tx=0x7f671400b410 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1611230849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 msgr2=0x55aebc4b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67180090a0 tx=0x7f6718099aa0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1611230849 shutdown_connections 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4b23d0 0x55aebc4b47c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1611230849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1611230849 >> 172.21.15.31:0/1611230849 conn(0x55aebc41c200 msgr2=0x55aebc4a3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1611230849 shutdown_connections 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.120+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1611230849 wait complete. 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.121+0000 7f673a3a1b80 1 -- start start 2023-10-06T21:44:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.121+0000 7f673a3a1b80 1 Processor -- start 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc3fadd0 con 0x55aebc4a6c00 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc3976b0 con 0x55aebc4b23d0 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc397350 con 0x55aebc4a7520 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f673a3a1b80 10 client.0 _ll_register_callbacks cb 0x55aebc1f47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43748/0 (socket says 172.21.15.31:43748) 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f671f7fe700 1 -- 172.21.15.31:0/977410213 learned_addr learned my addr 172.21.15.31:0/977410213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.122+0000 7f6724f9c700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671ffff700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 -- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 msgr2=0x55aebc4afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 -- 172.21.15.31:0/977410213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 msgr2=0x55aebc4af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 -- 172.21.15.31:0/977410213 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672010c670 con 0x55aebc4a7520 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f671f7fe700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67140042c0 tx=0x7f67140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f23cca972149926 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.123+0000 7f66ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:44:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.124+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714029470 con 0x55aebc4a7520 2023-10-06T21:44:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.124+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f67140295d0 con 0x55aebc4a7520 2023-10-06T21:44:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.124+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aebc49ad50 con 0x55aebc4a7520 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.125+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6714029790 con 0x55aebc4a7520 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.125+0000 7f673a3a1b80 10 client.27646 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.125+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55aebc20ec00 con 0x55aebc4a7520 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.125+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aebc49ad50 con 0x55aebc4a7520 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.125+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6714033960 con 0x55aebc4a7520 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 10 client.27646 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 10 client.27646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 20 client.27646 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc4c1f50 0x55aebc4c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 4 client.27646 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:44:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.126+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aebc3f3db0 con 0x55aebc4c1f50 2023-10-06T21:44:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.127+0000 7f671ffff700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc4c1f50 0x55aebc4c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.127+0000 7f671ffff700 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc4c1f50 0x55aebc4c4370 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8568542af33ca9d1 server_cookie=ce57591cd7dcfd87 in_seq=0 out_seq=0 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.127+0000 7f66ff7fe700 10 client.27646 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.128+0000 7f66ff7fe700 1 -- 172.21.15.31:0/977410213 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aebc3f3db0 con 0x55aebc4c1f50 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.128+0000 7f66ff7fe700 10 client.27646 handle_command_reply: tid=0 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.128+0000 7f673a3a1b80 1 client.27646 shutdown 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 msgr2=0x55aebc4af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67140042c0 tx=0x7f67140047c0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 shutdown_connections 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc4c1f50 0x55aebc4c4370 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/977410213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4af690 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f66ff7fe700 0 client.27646 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 >> 172.21.15.31:0/977410213 conn(0x55aebc41c200 msgr2=0x55aebc4a2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 shutdown_connections 2023-10-06T21:44:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.129+0000 7f673a3a1b80 1 -- 172.21.15.31:0/977410213 wait complete. 2023-10-06T21:44:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.130+0000 7f673a3a1b80 20 client.27646 trim_cache size 0 max 16384 2023-10-06T21:44:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 Processor -- start 2023-10-06T21:44:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 -- start start 2023-10-06T21:44:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc27b8c0 con 0x55aebc4a6c00 2023-10-06T21:44:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc4b21f0 con 0x55aebc4b23d0 2023-10-06T21:44:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.153+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc4a7b60 con 0x55aebc4a7520 2023-10-06T21:44:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f671ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f671ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33670/0 (socket says 172.21.15.31:33670) 2023-10-06T21:44:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f671ffff700 1 -- 172.21.15.31:0/857641430 learned_addr learned my addr 172.21.15.31:0/857641430 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f671f7fe700 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 -- 172.21.15.31:0/857641430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 msgr2=0x55aebc4be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 -- 172.21.15.31:0/857641430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 msgr2=0x55aebc4c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.154+0000 7f6724f9c700 1 -- 172.21.15.31:0/857641430 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aebc24c1f0 con 0x55aebc4b23d0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.155+0000 7f6724f9c700 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4aa530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f671800c6f0 tx=0x7f67180958b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ff2fb1992d425e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.155+0000 7f6706ffd700 1 -- 172.21.15.31:0/857641430 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671801b320 con 0x55aebc4b23d0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.155+0000 7f6706ffd700 1 -- 172.21.15.31:0/857641430 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671809a300 con 0x55aebc4b23d0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.155+0000 7f6706ffd700 1 -- 172.21.15.31:0/857641430 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180b2220 con 0x55aebc4b23d0 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 -- 172.21.15.31:0/857641430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 msgr2=0x55aebc4aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4aa530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f671800c6f0 tx=0x7f67180958b0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 -- 172.21.15.31:0/857641430 shutdown_connections 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4b23d0 0x55aebc4aa530 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4a6c00 0x55aebc4c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/857641430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a7520 0x55aebc4be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 -- 172.21.15.31:0/857641430 >> 172.21.15.31:0/857641430 conn(0x55aebc41c200 msgr2=0x55aebc4c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 -- 172.21.15.31:0/857641430 shutdown_connections 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.156+0000 7f673a3a1b80 1 -- 172.21.15.31:0/857641430 wait complete. 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.157+0000 7f673a3a1b80 1 -- start start 2023-10-06T21:44:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.157+0000 7f673a3a1b80 1 Processor -- start 2023-10-06T21:44:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.157+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 0x55aebc4b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc514f50 con 0x55aebc4b23d0 2023-10-06T21:44:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc5150c0 con 0x55aebc4a7520 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebc515280 con 0x55aebc4a6c00 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f673a3a1b80 10 client.0 _ll_register_callbacks cb 0x55aebc1f47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:43764/0 (socket says 172.21.15.31:43764) 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 -- 172.21.15.31:0/1093771627 learned_addr learned my addr 172.21.15.31:0/1093771627 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671f7fe700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f6724f9c700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 0x55aebc4b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 -- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 msgr2=0x55aebc4b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 -- 172.21.15.31:0/1093771627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 msgr2=0x55aebc4b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 0x55aebc4b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f671ffff700 1 -- 172.21.15.31:0/1093771627 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aebc225d80 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.158+0000 7f6724f9c700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 0x55aebc4b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f671ffff700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6710028bd0 tx=0x7f671000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73859f89c5457aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f66ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671001d030 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f671800ce60 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6710005690 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67100282b0 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f673a3a1b80 10 client.27652 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.159+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55aebc224250 con 0x55aebc4a6c00 2023-10-06T21:44:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671800ce60 con 0x55aebc4a6c00 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6710004590 con 0x55aebc4a6c00 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 10 client.27652 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 10 client.27652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 20 client.27652 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc516a90 0x55aebc518eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 4 client.27652 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.160+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aebc3f3ad0 con 0x55aebc516a90 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.161+0000 7f671f7fe700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc516a90 0x55aebc518eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.161+0000 7f671f7fe700 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc516a90 0x55aebc518eb0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15cefaffb845e606 server_cookie=2b25002fe7cb4d6e in_seq=0 out_seq=0 2023-10-06T21:44:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.161+0000 7f66ff7fe700 10 client.27652 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.223+0000 7f66ff7fe700 1 -- 172.21.15.31:0/1093771627 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aebc3f3ad0 con 0x55aebc516a90 2023-10-06T21:44:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.223+0000 7f66ff7fe700 10 client.27652 handle_command_reply: tid=0 2023-10-06T21:44:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.223+0000 7f673a3a1b80 1 client.27652 shutdown 2023-10-06T21:44:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 msgr2=0x55aebc4b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6710028bd0 tx=0x7f671000b260 comp rx=0 tx=0).stop 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 shutdown_connections 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55aebc516a90 0x55aebc518eb0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aebc4b23d0 0x55aebc4b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aebc4a6c00 0x55aebc4b6f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f66ff7fe700 0 client.27652 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 --2- 172.21.15.31:0/1093771627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aebc4a7520 0x55aebc4b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.224+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 >> 172.21.15.31:0/1093771627 conn(0x55aebc41c200 msgr2=0x55aebc4be150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.225+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 shutdown_connections 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.225+0000 7f673a3a1b80 1 -- 172.21.15.31:0/1093771627 wait complete. 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.225+0000 7f673a3a1b80 20 client.27652 trim_cache size 0 max 16384 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:44:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:44:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 msgr2=0x7f67080a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 0x7f67080a3bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f671400b6c0 tx=0x7f671400b410 comp rx=0 tx=0).stop 2023-10-06T21:44:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 msgr2=0x7f6720101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f671800ef90 tx=0x7f671800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 shutdown_connections 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f672010d6f0 0x7f6720101500 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f67080a16f0 0x7f67080a3bb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6720073280 0x7f6720106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 --2- 172.21.15.31:0/2890959025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720073ba0 0x7f6720106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 >> 172.21.15.31:0/2890959025 conn(0x7f67200fc940 msgr2=0x7f672010f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.226+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 shutdown_connections 2023-10-06T21:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:24.227+0000 7f67269ff700 1 -- 172.21.15.31:0/2890959025 wait complete. 2023-10-06T21:44:24.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72691 6.35 MB/sec execute 493 sec latency 91.778 ms 2023-10-06T21:44:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:25 smithi031 ceph-mon[98539]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 211 op/s 2023-10-06T21:44:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:25 smithi146 ceph-mon[102809]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 211 op/s 2023-10-06T21:44:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:25 smithi142 ceph-mon[102951]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 211 op/s 2023-10-06T21:44:25.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72802 6.35 MB/sec execute 494 sec latency 75.746 ms 2023-10-06T21:44:26.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72885 6.35 MB/sec execute 495 sec latency 350.336 ms 2023-10-06T21:44:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:27 smithi031 ceph-mon[98539]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 189 op/s 2023-10-06T21:44:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:27 smithi146 ceph-mon[102809]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 189 op/s 2023-10-06T21:44:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:27 smithi142 ceph-mon[102951]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 189 op/s 2023-10-06T21:44:27.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 72985 6.34 MB/sec execute 496 sec latency 68.120 ms 2023-10-06T21:44:28.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73091 6.34 MB/sec execute 497 sec latency 78.486 ms 2023-10-06T21:44:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:29 smithi031 ceph-mon[98539]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2023-10-06T21:44:29.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:29 smithi146 ceph-mon[102809]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2023-10-06T21:44:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:29 smithi142 ceph-mon[102951]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2023-10-06T21:44:29.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73198 6.34 MB/sec execute 498 sec latency 74.596 ms 2023-10-06T21:44:30.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73321 6.33 MB/sec execute 499 sec latency 81.774 ms 2023-10-06T21:44:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:31 smithi146 ceph-mon[102809]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 268 op/s 2023-10-06T21:44:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:31 smithi142 ceph-mon[102951]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 268 op/s 2023-10-06T21:44:31.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:31 smithi031 ceph-mon[98539]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 268 op/s 2023-10-06T21:44:31.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73435 6.33 MB/sec execute 500 sec latency 143.456 ms 2023-10-06T21:44:32.793 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73546 6.33 MB/sec execute 501 sec latency 294.839 ms 2023-10-06T21:44:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:33 smithi146 ceph-mon[102809]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 172 op/s 2023-10-06T21:44:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:33 smithi142 ceph-mon[102951]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 172 op/s 2023-10-06T21:44:33.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:33 smithi031 ceph-mon[98539]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 172 op/s 2023-10-06T21:44:33.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73677 6.33 MB/sec execute 502 sec latency 75.148 ms 2023-10-06T21:44:34.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73789 6.33 MB/sec execute 503 sec latency 96.969 ms 2023-10-06T21:44:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:35 smithi146 ceph-mon[102809]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 266 op/s 2023-10-06T21:44:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:35 smithi142 ceph-mon[102951]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 266 op/s 2023-10-06T21:44:35.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:35 smithi031 ceph-mon[98539]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 266 op/s 2023-10-06T21:44:35.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 73900 6.33 MB/sec execute 504 sec latency 72.737 ms 2023-10-06T21:44:36.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74013 6.32 MB/sec execute 505 sec latency 96.104 ms 2023-10-06T21:44:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 -- 172.21.15.31:0/444841192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9388101a80 msgr2=0x7f9388105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 --2- 172.21.15.31:0/444841192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9388101a80 0x7f9388105ac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f937c009990 tx=0x7f937c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 -- 172.21.15.31:0/444841192 shutdown_connections 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 --2- 172.21.15.31:0/444841192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 0x7f938810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 --2- 172.21.15.31:0/444841192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93881010d0 0x7f93881014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 --2- 172.21.15.31:0/444841192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9388101a80 0x7f9388105ac0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 -- 172.21.15.31:0/444841192 >> 172.21.15.31:0/444841192 conn(0x7f93880fc950 msgr2=0x7f93880fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 -- 172.21.15.31:0/444841192 shutdown_connections 2023-10-06T21:44:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.449+0000 7f938dbff700 1 -- 172.21.15.31:0/444841192 wait complete. 2023-10-06T21:44:37.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.451+0000 7f938dbff700 1 Processor -- start 2023-10-06T21:44:37.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.451+0000 7f938dbff700 1 -- start start 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 0x7f93881a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f938810cb30 0x7f938810cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938810d5b0 con 0x7f93881061f0 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938810d6f0 con 0x7f938810cb30 2023-10-06T21:44:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f938dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938810d8b0 con 0x7f93881010d0 2023-10-06T21:44:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f93877fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f93877fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42526/0 (socket says 172.21.15.31:42526) 2023-10-06T21:44:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f93877fe700 1 -- 172.21.15.31:0/1445580132 learned_addr learned my addr 172.21.15.31:0/1445580132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f9386ffd700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 0x7f93881a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.452+0000 7f9387fff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f938810cb30 0x7f938810cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 -- 172.21.15.31:0/1445580132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f938810cb30 msgr2=0x7f938810cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f938810cb30 0x7f938810cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 -- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 msgr2=0x7f93881a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 0x7f93881a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 -- 172.21.15.31:0/1445580132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0095f0 con 0x7f93881010d0 2023-10-06T21:44:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.453+0000 7f93877fe700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f937800e9a0 tx=0x7f937800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.454+0000 7f9384ff9700 1 -- 172.21.15.31:0/1445580132 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800ca10 con 0x7f93881010d0 2023-10-06T21:44:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.454+0000 7f9384ff9700 1 -- 172.21.15.31:0/1445580132 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9378010430 con 0x7f93881010d0 2023-10-06T21:44:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.454+0000 7f9384ff9700 1 -- 172.21.15.31:0/1445580132 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378018680 con 0x7f93881010d0 2023-10-06T21:44:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.454+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93881aa3b0 con 0x7f93881010d0 2023-10-06T21:44:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.454+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881aa840 con 0x7f93881010d0 2023-10-06T21:44:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.459+0000 7f9384ff9700 1 -- 172.21.15.31:0/1445580132 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f93780105a0 con 0x7f93881010d0 2023-10-06T21:44:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.461+0000 7f93a1496b80 1 Processor -- start 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.461+0000 7f93a1496b80 1 -- start start 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6205fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da62113d0 0x562da62137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da616f400 con 0x562da6205c00 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da6160bc0 con 0x562da6206520 2023-10-06T21:44:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93a1496b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da615e4e0 con 0x562da62113d0 2023-10-06T21:44:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f9384ff9700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 0x7f93700a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52528/0 (socket says 172.21.15.31:52528) 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f93877fe700 1 -- 172.21.15.31:0/1412080119 learned_addr learned my addr 172.21.15.31:0/1412080119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.462+0000 7f9386ffd700 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da62113d0 0x562da62137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f9384ff9700 1 -- 172.21.15.31:0/1445580132 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9378014070 con 0x7f93881010d0 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f9387fff700 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6205fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 -- 172.21.15.31:0/1412080119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da62113d0 msgr2=0x562da62137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da62113d0 0x562da62137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 -- 172.21.15.31:0/1412080119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 msgr2=0x562da6205fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6205fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 -- 172.21.15.31:0/1412080119 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da5f84d80 con 0x562da6206520 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f9387fff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 0x7f93700a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f93877fe700 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93780ca900 tx=0x7f9378095f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab47b60a34e479c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.463+0000 7f936dffb700 1 -- 172.21.15.31:0/1412080119 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780cd030 con 0x562da6206520 2023-10-06T21:44:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.464+0000 7f936dffb700 1 -- 172.21.15.31:0/1412080119 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9378099b80 con 0x562da6206520 2023-10-06T21:44:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.464+0000 7f936dffb700 1 -- 172.21.15.31:0/1412080119 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780a25b0 con 0x562da6206520 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 -- 172.21.15.31:0/1412080119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 msgr2=0x562da6210e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f9387fff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 0x7f93700a3bb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9380009ec0 tx=0x7f9380004ad0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93780ca900 tx=0x7f9378095f40 comp rx=0 tx=0).stop 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 -- 172.21.15.31:0/1412080119 shutdown_connections 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da62113d0 0x562da62137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6205fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1412080119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6206520 0x562da6210e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 -- 172.21.15.31:0/1412080119 >> 172.21.15.31:0/1412080119 conn(0x562da617b200 msgr2=0x562da6202080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 -- 172.21.15.31:0/1412080119 shutdown_connections 2023-10-06T21:44:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.465+0000 7f93a1496b80 1 -- 172.21.15.31:0/1412080119 wait complete. 2023-10-06T21:44:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.466+0000 7f93a1496b80 1 -- start start 2023-10-06T21:44:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.466+0000 7f93a1496b80 1 Processor -- start 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da620e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da6159dd0 con 0x562da6205c00 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da60f66b0 con 0x562da62113d0 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da60f6350 con 0x562da6206520 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93a1496b80 10 client.0 _ll_register_callbacks cb 0x562da5f537a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34388/0 (socket says 172.21.15.31:34388) 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f93877fe700 1 -- 172.21.15.31:0/3706863092 learned_addr learned my addr 172.21.15.31:0/3706863092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f9387fff700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.467+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da620e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.468+0000 7f93877fe700 1 -- 172.21.15.31:0/3706863092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 msgr2=0x562da620e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.468+0000 7f93877fe700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da620e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.468+0000 7f93877fe700 1 -- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 msgr2=0x562da620e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.468+0000 7f93877fe700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.468+0000 7f93877fe700 1 -- 172.21.15.31:0/3706863092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881aa840 con 0x562da6205c00 2023-10-06T21:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.469+0000 7f93877fe700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9378096720 tx=0x7f9378010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e302db20184d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.469+0000 7f9366ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.469+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378099910 con 0x562da6205c00 2023-10-06T21:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.469+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f937809a920 con 0x562da6205c00 2023-10-06T21:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.470+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da61f9d50 con 0x562da6205c00 2023-10-06T21:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.470+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937807b050 con 0x562da6205c00 2023-10-06T21:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.470+0000 7f93a1496b80 10 client.17988 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.470+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562da5f6dc00 con 0x562da6205c00 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.470+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562da61f9d50 con 0x562da6205c00 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9378083280 con 0x562da6205c00 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 10 client.17988 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 20 client.17988 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6220b30 0x562da6222f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 4 client.17988 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.471+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da6152db0 con 0x562da6220b30 2023-10-06T21:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.472+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6220b30 0x562da6222f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.473+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6220b30 0x562da6222f20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc86950bdd2f080a server_cookie=804a89fcba2a67a0 in_seq=0 out_seq=0 2023-10-06T21:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.473+0000 7f9366ffd700 10 client.17988 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.475+0000 7f9366ffd700 1 -- 172.21.15.31:0/3706863092 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562da6152db0 con 0x562da6220b30 2023-10-06T21:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.475+0000 7f9366ffd700 10 client.17988 handle_command_reply: tid=0 2023-10-06T21:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.475+0000 7f93a1496b80 1 client.17988 shutdown 2023-10-06T21:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.475+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 msgr2=0x562da6210e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.475+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9378096720 tx=0x7f9378010900 comp rx=0 tx=0).stop 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 shutdown_connections 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6220b30 0x562da6222f20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6205c00 0x562da6210e80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3706863092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da620e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 >> 172.21.15.31:0/3706863092 conn(0x562da617b200 msgr2=0x562da6201c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f9366ffd700 0 client.17988 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 shutdown_connections 2023-10-06T21:44:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.476+0000 7f93a1496b80 1 -- 172.21.15.31:0/3706863092 wait complete. 2023-10-06T21:44:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.477+0000 7f93a1496b80 20 client.17988 trim_cache size 0 max 16384 2023-10-06T21:44:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 Processor -- start 2023-10-06T21:44:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 -- start start 2023-10-06T21:44:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6206520 0x562da6217f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da5fda8c0 con 0x562da6206520 2023-10-06T21:44:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da620f710 con 0x562da62113d0 2023-10-06T21:44:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.501+0000 7f93a1496b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da620f080 con 0x562da6205c00 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f9386ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6206520 0x562da6217f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f9387fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42558/0 (socket says 172.21.15.31:42558) 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 -- 172.21.15.31:0/1496197242 learned_addr learned my addr 172.21.15.31:0/1496197242 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 -- 172.21.15.31:0/1496197242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 msgr2=0x562da620d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 -- 172.21.15.31:0/1496197242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6206520 msgr2=0x562da6217f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6206520 0x562da6217f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.502+0000 7f93877fe700 1 -- 172.21.15.31:0/1496197242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da5fab1f0 con 0x562da6205c00 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.503+0000 7f93877fe700 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f937809a1b0 tx=0x7f93780987d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f0a3f8057b1d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.503+0000 7f936e7fc700 1 -- 172.21.15.31:0/1496197242 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937807b070 con 0x562da6205c00 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.503+0000 7f936e7fc700 1 -- 172.21.15.31:0/1496197242 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f937809a920 con 0x562da6205c00 2023-10-06T21:44:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.503+0000 7f936e7fc700 1 -- 172.21.15.31:0/1496197242 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780a2120 con 0x562da6205c00 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 -- 172.21.15.31:0/1496197242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 msgr2=0x562da6208aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f937809a1b0 tx=0x7f93780987d0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 -- 172.21.15.31:0/1496197242 shutdown_connections 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da62113d0 0x562da620d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6205c00 0x562da6208aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 --2- 172.21.15.31:0/1496197242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da6206520 0x562da6217f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 -- 172.21.15.31:0/1496197242 >> 172.21.15.31:0/1496197242 conn(0x562da617b200 msgr2=0x562da6201250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 -- 172.21.15.31:0/1496197242 shutdown_connections 2023-10-06T21:44:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.504+0000 7f93a1496b80 1 -- 172.21.15.31:0/1496197242 wait complete. 2023-10-06T21:44:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.505+0000 7f93a1496b80 1 -- start start 2023-10-06T21:44:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.505+0000 7f93a1496b80 1 Processor -- start 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6205c00 0x562da62160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da62113d0 0x562da6273910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da6273e50 con 0x562da62113d0 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da6273f90 con 0x562da6205c00 2023-10-06T21:44:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da6274150 con 0x562da6206520 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f93a1496b80 10 client.0 _ll_register_callbacks cb 0x562da5f537a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f9386ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f9386ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42560/0 (socket says 172.21.15.31:42560) 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.506+0000 7f9386ffd700 1 -- 172.21.15.31:0/3859181783 learned_addr learned my addr 172.21.15.31:0/3859181783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9387fff700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da62113d0 0x562da6273910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f93877fe700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6205c00 0x562da62160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 -- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6205c00 msgr2=0x562da62160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6205c00 0x562da62160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 -- 172.21.15.31:0/3859181783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da62113d0 msgr2=0x562da6273910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da62113d0 0x562da6273910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 -- 172.21.15.31:0/3859181783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562da5f6dc00 con 0x562da6206520 2023-10-06T21:44:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9386ffd700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f937c009990 tx=0x7f937c005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad78c71af2d5c8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.507+0000 7f9366ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c01d040 con 0x562da6206520 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f937c00bac0 con 0x562da6206520 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9378099b80 con 0x562da6206520 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937c00bc80 con 0x562da6206520 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f93a1496b80 10 client.27670 fetch_fsmap learned FSMap version 50 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562da5f83250 con 0x562da6206520 2023-10-06T21:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.508+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9378099b80 con 0x562da6206520 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f937c004d10 con 0x562da6206520 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 10 client.27670 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 10 client.27670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 20 client.27670 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6275910 0x562da6277d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 4 client.27670 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562da6152ad0 con 0x562da6275910 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.509+0000 7f93877fe700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6275910 0x562da6277d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.510+0000 7f9366ffd700 10 client.27670 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.510+0000 7f93877fe700 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6275910 0x562da6277d30 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6a861277695c8 server_cookie=5ad6f94cc3b7db09 in_seq=0 out_seq=0 2023-10-06T21:44:37.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:37 smithi146 ceph-mon[102809]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:37 smithi142 ceph-mon[102951]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.663+0000 7f9366ffd700 1 -- 172.21.15.31:0/3859181783 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562da6152ad0 con 0x562da6275910 2023-10-06T21:44:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.663+0000 7f9366ffd700 10 client.27670 handle_command_reply: tid=0 2023-10-06T21:44:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 client.27670 shutdown 2023-10-06T21:44:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 msgr2=0x562da6216630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f937c009990 tx=0x7f937c005560 comp rx=0 tx=0).stop 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 shutdown_connections 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x562da6275910 0x562da6277d30 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562da62113d0 0x562da6273910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562da6205c00 0x562da62160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 --2- 172.21.15.31:0/3859181783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da6206520 0x562da6216630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 >> 172.21.15.31:0/3859181783 conn(0x562da617b200 msgr2=0x562da6201ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f9366ffd700 0 client.27670 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.664+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 shutdown_connections 2023-10-06T21:44:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.665+0000 7f93a1496b80 1 -- 172.21.15.31:0/3859181783 wait complete. 2023-10-06T21:44:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.665+0000 7f93a1496b80 20 client.27670 trim_cache size 0 max 16384 2023-10-06T21:44:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:44:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:44:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:44:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.666+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 msgr2=0x7f93700a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.666+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 0x7f93700a3bb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9380009ec0 tx=0x7f9380004ad0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.666+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 msgr2=0x7f93881a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.666+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f937800e9a0 tx=0x7f937800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 shutdown_connections 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f938810cb30 0x7f938810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93881010d0 0x7f93881a3540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f93700a16f0 0x7f93700a3bb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 --2- 172.21.15.31:0/1445580132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93881061f0 0x7f93881a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.667+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 >> 172.21.15.31:0/1445580132 conn(0x7f93880fc950 msgr2=0x7f93880fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.668+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 shutdown_connections 2023-10-06T21:44:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:37.668+0000 7f938dbff700 1 -- 172.21.15.31:0/1445580132 wait complete. 2023-10-06T21:44:37.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:37 smithi031 ceph-mon[98539]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:37.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74138 6.33 MB/sec execute 506 sec latency 93.788 ms 2023-10-06T21:44:38.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74251 6.32 MB/sec execute 507 sec latency 76.193 ms 2023-10-06T21:44:39.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:39 smithi146 ceph-mon[102809]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:39 smithi142 ceph-mon[102951]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:39 smithi031 ceph-mon[98539]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-06T21:44:39.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74388 6.33 MB/sec execute 508 sec latency 69.776 ms 2023-10-06T21:44:40.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:40 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:44:40.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:40 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:44:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:40 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:44:40.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74498 6.33 MB/sec execute 509 sec latency 74.327 ms 2023-10-06T21:44:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:41 smithi146 ceph-mon[102809]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 297 op/s 2023-10-06T21:44:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:41 smithi142 ceph-mon[102951]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 297 op/s 2023-10-06T21:44:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:41 smithi031 ceph-mon[98539]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 297 op/s 2023-10-06T21:44:41.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74629 6.33 MB/sec execute 510 sec latency 77.079 ms 2023-10-06T21:44:42.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74743 6.32 MB/sec execute 511 sec latency 76.183 ms 2023-10-06T21:44:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:43 smithi142 ceph-mon[102951]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2023-10-06T21:44:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:43 smithi031 ceph-mon[98539]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2023-10-06T21:44:43.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74813 6.31 MB/sec execute 512 sec latency 295.772 ms 2023-10-06T21:44:43.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:43 smithi146 ceph-mon[102809]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2023-10-06T21:44:44.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 74943 6.31 MB/sec execute 513 sec latency 92.960 ms 2023-10-06T21:44:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:45 smithi142 ceph-mon[102951]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2023-10-06T21:44:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:45 smithi031 ceph-mon[98539]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2023-10-06T21:44:45.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75066 6.31 MB/sec execute 514 sec latency 75.196 ms 2023-10-06T21:44:45.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:45 smithi146 ceph-mon[102809]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2023-10-06T21:44:46.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75205 6.32 MB/sec execute 515 sec latency 81.898 ms 2023-10-06T21:44:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:47 smithi142 ceph-mon[102951]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:47 smithi031 ceph-mon[98539]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:47.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75340 6.33 MB/sec execute 516 sec latency 73.136 ms 2023-10-06T21:44:47.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:47 smithi146 ceph-mon[102809]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:48.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75429 6.33 MB/sec execute 517 sec latency 381.764 ms 2023-10-06T21:44:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:49 smithi142 ceph-mon[102951]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:49 smithi031 ceph-mon[98539]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:49.794 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75555 6.33 MB/sec execute 518 sec latency 102.198 ms 2023-10-06T21:44:49.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:49 smithi146 ceph-mon[102809]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2023-10-06T21:44:50.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75677 6.34 MB/sec execute 519 sec latency 143.061 ms 2023-10-06T21:44:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.908+0000 7f8ae4dff700 1 -- 172.21.15.31:0/1728109626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0068c90 msgr2=0x7f8ae010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.908+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/1728109626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0068c90 0x7f8ae010dd00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800b0a0 tx=0x7f8ad800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 -- 172.21.15.31:0/1728109626 shutdown_connections 2023-10-06T21:44:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/1728109626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0068c90 0x7f8ae010dd00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/1728109626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae0106f60 0x7f8ae0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/1728109626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae00682d0 0x7f8ae0068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 -- 172.21.15.31:0/1728109626 >> 172.21.15.31:0/1728109626 conn(0x7f8ae0076b70 msgr2=0x7f8ae0076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 -- 172.21.15.31:0/1728109626 shutdown_connections 2023-10-06T21:44:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.909+0000 7f8ae4dff700 1 -- 172.21.15.31:0/1728109626 wait complete. 2023-10-06T21:44:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.911+0000 7f8ae4dff700 1 Processor -- start 2023-10-06T21:44:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.911+0000 7f8ae4dff700 1 -- start start 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae00682d0 0x7f8ae010a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 0x7f8ae010a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 0x7f8ae01a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01a9ae0 con 0x7f8ae0106f60 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01a9c20 con 0x7f8ae00682d0 2023-10-06T21:44:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ae4dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01a9d90 con 0x7f8ae0068c90 2023-10-06T21:44:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8addd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 0x7f8ae010a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8addd9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 0x7f8ae010a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52580/0 (socket says 172.21.15.31:52580) 2023-10-06T21:44:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8addd9b700 1 -- 172.21.15.31:0/2546948148 learned_addr learned my addr 172.21.15.31:0/2546948148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8aded9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 0x7f8ae01a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.912+0000 7f8ade59c700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae00682d0 0x7f8ae010a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 -- 172.21.15.31:0/2546948148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 msgr2=0x7f8ae010a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 0x7f8ae010a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 -- 172.21.15.31:0/2546948148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae00682d0 msgr2=0x7f8ae010a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae00682d0 0x7f8ae010a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 -- 172.21.15.31:0/2546948148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8009d00 con 0x7f8ae0106f60 2023-10-06T21:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.913+0000 7f8aded9d700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 0x7f8ae01a95a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad8007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.914+0000 7f8ad77fe700 1 -- 172.21.15.31:0/2546948148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800e040 con 0x7f8ae0106f60 2023-10-06T21:44:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.914+0000 7f8ad77fe700 1 -- 172.21.15.31:0/2546948148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad8003e90 con 0x7f8ae0106f60 2023-10-06T21:44:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.914+0000 7f8ad77fe700 1 -- 172.21.15.31:0/2546948148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8005040 con 0x7f8ae0106f60 2023-10-06T21:44:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.914+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae01aa010 con 0x7f8ae0106f60 2023-10-06T21:44:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.914+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae01aa3e0 con 0x7f8ae0106f60 2023-10-06T21:44:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.920+0000 7f8ad77fe700 1 -- 172.21.15.31:0/2546948148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8ad801d610 con 0x7f8ae0106f60 2023-10-06T21:44:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.921+0000 7f8af86aab80 1 Processor -- start 2023-10-06T21:44:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.921+0000 7f8af86aab80 1 -- start start 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbd23e0 0x563bcfbd47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfb1f400 con 0x563bcfbb6520 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfb10bc0 con 0x563bcfbd23e0 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8af86aab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfb0e4e0 con 0x563bcfbb5c00 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8ad77fe700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 0x7f8acc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8ad77fe700 1 -- 172.21.15.31:0/2546948148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8ad80cdaf0 con 0x7f8ae0106f60 2023-10-06T21:44:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41448/0 (socket says 172.21.15.31:41448) 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8ade59c700 1 -- 172.21.15.31:0/2849194393 learned_addr learned my addr 172.21.15.31:0/2849194393 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8addd9b700 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbd23e0 0x563bcfbd47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.922+0000 7f8addd9b700 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 -- 172.21.15.31:0/2849194393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 msgr2=0x563bcfbb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbb5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 -- 172.21.15.31:0/2849194393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbd23e0 msgr2=0x563bcfbd47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbd23e0 0x563bcfbd47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 -- 172.21.15.31:0/2849194393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcf934d80 con 0x563bcfbb6520 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8aded9d700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 0x7f8acc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ade59c700 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d00 tx=0x7f8ac8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a0255ebd6c2c63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ad4ff9700 1 -- 172.21.15.31:0/2849194393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004230 con 0x563bcfbb6520 2023-10-06T21:44:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ad4ff9700 1 -- 172.21.15.31:0/2849194393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ac8004390 con 0x563bcfbb6520 2023-10-06T21:44:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.923+0000 7f8ad4ff9700 1 -- 172.21.15.31:0/2849194393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8019750 con 0x563bcfbb6520 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 -- 172.21.15.31:0/2849194393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 msgr2=0x563bcfbd1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d00 tx=0x7f8ac8011720 comp rx=0 tx=0).stop 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8aded9d700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 0x7f8acc0a3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80a27c0 tx=0x7f8ad8009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 -- 172.21.15.31:0/2849194393 shutdown_connections 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbd23e0 0x563bcfbd47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 --2- 172.21.15.31:0/2849194393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbb6520 0x563bcfbd1ea0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.924+0000 7f8af86aab80 1 -- 172.21.15.31:0/2849194393 >> 172.21.15.31:0/2849194393 conn(0x563bcfb2b200 msgr2=0x563bcfbb2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.925+0000 7f8af86aab80 1 -- 172.21.15.31:0/2849194393 shutdown_connections 2023-10-06T21:44:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.925+0000 7f8af86aab80 1 -- 172.21.15.31:0/2849194393 wait complete. 2023-10-06T21:44:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.925+0000 7f8af86aab80 1 -- start start 2023-10-06T21:44:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.926+0000 7f8af86aab80 1 Processor -- start 2023-10-06T21:44:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb5c00 0x563bcfbd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbcf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfb09dd0 con 0x563bcfbd23e0 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfaa66b0 con 0x563bcfbb5c00 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfaa6350 con 0x563bcfbb6520 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8af86aab80 10 client.0 _ll_register_callbacks cb 0x563bcf9037a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:52602/0 (socket says 172.21.15.31:52602) 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 -- 172.21.15.31:0/4262684800 learned_addr learned my addr 172.21.15.31:0/4262684800 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8addd9b700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbcf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 -- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb5c00 msgr2=0x563bcfbd1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb5c00 0x563bcfbd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 -- 172.21.15.31:0/4262684800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 msgr2=0x563bcfbcf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbcf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.927+0000 7f8ade59c700 1 -- 172.21.15.31:0/4262684800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8009960 con 0x563bcfbb6520 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.928+0000 7f8ade59c700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009f50 tx=0x7f8ac8003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31349d9c25dd18dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.928+0000 7f8ac5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.928+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8005ce0 con 0x563bcfbb6520 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bcfba9d50 con 0x563bcfbb6520 2023-10-06T21:44:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8ac8005250 con 0x563bcfbb6520 2023-10-06T21:44:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac8005410 con 0x563bcfbb6520 2023-10-06T21:44:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8af86aab80 10 client.27679 fetch_fsmap learned FSMap version 51 2023-10-06T21:44:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563bcf91dc00 con 0x563bcfbb6520 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ac8036080 con 0x563bcfbb6520 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.929+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ac8028020 con 0x563bcfbb6520 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 10 client.27679 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 10 client.27679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 20 client.27679 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfbe1e70 0x563bcfbe4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 4 client.27679 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bcfb02db0 con 0x563bcfbe1e70 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.930+0000 7f8addd9b700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfbe1e70 0x563bcfbe4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.932+0000 7f8addd9b700 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfbe1e70 0x563bcfbe4290 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de34b1a7b227c6fb server_cookie=8bd6fcf16f970d4b in_seq=0 out_seq=0 2023-10-06T21:44:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.932+0000 7f8ac5ffb700 10 client.27679 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.932+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/4262684800 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563bcfb02db0 con 0x563bcfbe1e70 2023-10-06T21:44:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.932+0000 7f8ac5ffb700 10 client.27679 handle_command_reply: tid=0 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 client.27679 shutdown 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 msgr2=0x563bcfbcf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009f50 tx=0x7f8ac8003960 comp rx=0 tx=0).stop 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 shutdown_connections 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfbe1e70 0x563bcfbe4290 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb5c00 0x563bcfbd1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 --2- 172.21.15.31:0/4262684800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb6520 0x563bcfbcf0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 >> 172.21.15.31:0/4262684800 conn(0x563bcfb2b200 msgr2=0x563bcfbb1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8ac5ffb700 0 client.27679 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.933+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 shutdown_connections 2023-10-06T21:44:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.934+0000 7f8af86aab80 1 -- 172.21.15.31:0/4262684800 wait complete. 2023-10-06T21:44:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.934+0000 7f8af86aab80 20 client.27679 trim_cache size 0 max 16384 2023-10-06T21:44:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.959+0000 7f8af86aab80 1 Processor -- start 2023-10-06T21:44:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.959+0000 7f8af86aab80 1 -- start start 2023-10-06T21:44:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbd47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfbdd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcf98a8c0 con 0x563bcfbd23e0 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfbb3610 con 0x563bcfbb6520 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8af86aab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfbb6e50 con 0x563bcfbb5c00 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8addd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8addd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41466/0 (socket says 172.21.15.31:41466) 2023-10-06T21:44:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.960+0000 7f8addd9b700 1 -- 172.21.15.31:0/3645245025 learned_addr learned my addr 172.21.15.31:0/3645245025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.961+0000 7f8addd9b700 1 -- 172.21.15.31:0/3645245025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 msgr2=0x563bcfbd47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.961+0000 7f8addd9b700 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbd47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.961+0000 7f8addd9b700 1 -- 172.21.15.31:0/3645245025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 msgr2=0x563bcfbdd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.961+0000 7f8addd9b700 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfbdd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.961+0000 7f8addd9b700 1 -- 172.21.15.31:0/3645245025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcf95b1f0 con 0x563bcfbd23e0 2023-10-06T21:44:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.962+0000 7f8addd9b700 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x563bcfbe4b30 tx=0x7f8ad0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2d8e4392fbc38b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.962+0000 7f8ad57fa700 1 -- 172.21.15.31:0/3645245025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0020bf0 con 0x563bcfbd23e0 2023-10-06T21:44:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.962+0000 7f8ad57fa700 1 -- 172.21.15.31:0/3645245025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad0020d50 con 0x563bcfbd23e0 2023-10-06T21:44:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.962+0000 7f8ad57fa700 1 -- 172.21.15.31:0/3645245025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00056f0 con 0x563bcfbd23e0 2023-10-06T21:44:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 -- 172.21.15.31:0/3645245025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 msgr2=0x563bcfbd19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x563bcfbe4b30 tx=0x7f8ad0004750 comp rx=0 tx=0).stop 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 -- 172.21.15.31:0/3645245025 shutdown_connections 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd19c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfbd47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 --2- 172.21.15.31:0/3645245025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfbdd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 -- 172.21.15.31:0/3645245025 >> 172.21.15.31:0/3645245025 conn(0x563bcfb2b200 msgr2=0x563bcfbe8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 -- 172.21.15.31:0/3645245025 shutdown_connections 2023-10-06T21:44:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.963+0000 7f8af86aab80 1 -- 172.21.15.31:0/3645245025 wait complete. 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.964+0000 7f8af86aab80 1 -- start start 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.964+0000 7f8af86aab80 1 Processor -- start 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfc2ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfbd74c0 con 0x563bcfbd23e0 2023-10-06T21:44:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfbd7630 con 0x563bcfbb6520 2023-10-06T21:44:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfbd77f0 con 0x563bcfbb5c00 2023-10-06T21:44:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8af86aab80 10 client.0 _ll_register_callbacks cb 0x7f8ac8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:44:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8addd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8ade59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:40602/0 (socket says 172.21.15.31:40602) 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.965+0000 7f8ade59c700 1 -- 172.21.15.31:0/979630559 learned_addr learned my addr 172.21.15.31:0/979630559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 -- 172.21.15.31:0/979630559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 msgr2=0x563bcfc2ee20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfc2ee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 -- 172.21.15.31:0/979630559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 msgr2=0x563bcfbd6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 -- 172.21.15.31:0/979630559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad0019040 con 0x563bcfbb6520 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ade59c700 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c6f9b1514b72db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:44:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.966+0000 7f8ac5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:44:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8009c30 con 0x563bcfbb6520 2023-10-06T21:44:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8ac8000e40 con 0x563bcfbb6520 2023-10-06T21:44:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ac8004390 con 0x563bcfbb6520 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac80092a0 con 0x563bcfbb6520 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8af86aab80 10 client.27746 fetch_fsmap learned FSMap version 51 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.967+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563bcf933250 con 0x563bcfbb6520 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ac8004390 con 0x563bcfbb6520 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ac8028070 con 0x563bcfbb6520 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 10 client.27746 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 10 client.27746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 20 client.27746 populate_metadata read hostname 'smithi031' 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfc368d0 0x563bcfc38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 4 client.27746 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:44:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.968+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563bcfb02ad0 con 0x563bcfc368d0 2023-10-06T21:44:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.969+0000 7f8addd9b700 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfc368d0 0x563bcfc38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:44:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.970+0000 7f8addd9b700 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfc368d0 0x563bcfc38cf0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3bab6809f5fdbe7 server_cookie=d7fdef926102a5e4 in_seq=0 out_seq=0 2023-10-06T21:44:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:50.970+0000 7f8ac5ffb700 10 client.27746 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.025+0000 7f8ac5ffb700 1 -- 172.21.15.31:0/979630559 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563bcfb02ad0 con 0x563bcfc368d0 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.025+0000 7f8ac5ffb700 10 client.27746 handle_command_reply: tid=0 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.025+0000 7f8af86aab80 1 client.27746 shutdown 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 msgr2=0x563bcfc2f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac8005e50 comp rx=0 tx=0).stop 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 shutdown_connections 2023-10-06T21:44:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x563bcfc368d0 0x563bcfc38cf0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563bcfbd23e0 0x563bcfbd6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcfbb5c00 0x563bcfc2ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 --2- 172.21.15.31:0/979630559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bcfbb6520 0x563bcfc2f360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 >> 172.21.15.31:0/979630559 conn(0x563bcfb2b200 msgr2=0x563bcfbdd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.026+0000 7f8ac5ffb700 0 client.27746 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.027+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 shutdown_connections 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.027+0000 7f8af86aab80 1 -- 172.21.15.31:0/979630559 wait complete. 2023-10-06T21:44:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.027+0000 7f8af86aab80 20 client.27746 trim_cache size 0 max 16384 2023-10-06T21:44:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:44:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:44:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:44:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:44:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 msgr2=0x7f8acc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 0x7f8acc0a3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80a27c0 tx=0x7f8ad8009040 comp rx=0 tx=0).stop 2023-10-06T21:44:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 msgr2=0x7f8ae01a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:44:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 0x7f8ae01a95a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad8007840 comp rx=0 tx=0).stop 2023-10-06T21:44:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 shutdown_connections 2023-10-06T21:44:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8acc0a1670 0x7f8acc0a3b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae0106f60 0x7f8ae01a95a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae00682d0 0x7f8ae010a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 --2- 172.21.15.31:0/2546948148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ae0068c90 0x7f8ae010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:44:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 >> 172.21.15.31:0/2546948148 conn(0x7f8ae0076b70 msgr2=0x7f8ae0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:44:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 shutdown_connections 2023-10-06T21:44:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:44:51.028+0000 7f8ae4dff700 1 -- 172.21.15.31:0/2546948148 wait complete. 2023-10-06T21:44:51.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:51 smithi142 ceph-mon[102951]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 274 op/s 2023-10-06T21:44:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:51 smithi031 ceph-mon[98539]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 274 op/s 2023-10-06T21:44:51.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75812 6.35 MB/sec execute 520 sec latency 169.999 ms 2023-10-06T21:44:51.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:51 smithi146 ceph-mon[102809]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 274 op/s 2023-10-06T21:44:52.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 75935 6.35 MB/sec execute 521 sec latency 72.719 ms 2023-10-06T21:44:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:53 smithi142 ceph-mon[102951]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 163 op/s 2023-10-06T21:44:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:53 smithi031 ceph-mon[98539]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 163 op/s 2023-10-06T21:44:53.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76038 6.35 MB/sec execute 522 sec latency 93.780 ms 2023-10-06T21:44:53.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:53 smithi146 ceph-mon[102809]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 163 op/s 2023-10-06T21:44:54.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76136 6.35 MB/sec execute 523 sec latency 83.460 ms 2023-10-06T21:44:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:55 smithi142 ceph-mon[102951]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 253 op/s 2023-10-06T21:44:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:55 smithi031 ceph-mon[98539]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 253 op/s 2023-10-06T21:44:55.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76240 6.35 MB/sec execute 524 sec latency 71.018 ms 2023-10-06T21:44:55.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:55 smithi146 ceph-mon[102809]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 253 op/s 2023-10-06T21:44:56.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76349 6.34 MB/sec execute 525 sec latency 75.438 ms 2023-10-06T21:44:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:57 smithi142 ceph-mon[102951]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:57 smithi031 ceph-mon[98539]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:57.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76457 6.34 MB/sec execute 526 sec latency 73.154 ms 2023-10-06T21:44:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:57 smithi146 ceph-mon[102809]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:58.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:44:58 smithi142 ceph-mon[102951]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:58.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:44:58 smithi031 ceph-mon[98539]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:58.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76563 6.34 MB/sec execute 527 sec latency 88.209 ms 2023-10-06T21:44:58.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:44:58 smithi146 ceph-mon[102809]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2023-10-06T21:44:59.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76660 6.33 MB/sec execute 528 sec latency 71.407 ms 2023-10-06T21:45:00.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76764 6.32 MB/sec execute 529 sec latency 71.974 ms 2023-10-06T21:45:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:01 smithi031 ceph-mon[98539]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 256 op/s 2023-10-06T21:45:01.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:01 smithi146 ceph-mon[102809]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 256 op/s 2023-10-06T21:45:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:01 smithi142 ceph-mon[102951]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 256 op/s 2023-10-06T21:45:01.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76855 6.32 MB/sec execute 530 sec latency 309.711 ms 2023-10-06T21:45:02.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 76945 6.31 MB/sec execute 531 sec latency 377.417 ms 2023-10-06T21:45:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:03 smithi031 ceph-mon[98539]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 172 op/s 2023-10-06T21:45:03.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:03 smithi146 ceph-mon[102809]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 172 op/s 2023-10-06T21:45:03.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:03 smithi142 ceph-mon[102951]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 172 op/s 2023-10-06T21:45:03.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77069 6.31 MB/sec execute 532 sec latency 163.531 ms 2023-10-06T21:45:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 -- 172.21.15.31:0/3562712174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf4108440 msgr2=0x7f1cf410ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/3562712174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf4108440 0x7f1cf410ffc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0098f0 tx=0x7f1cec009c00 comp rx=0 tx=0).stop 2023-10-06T21:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 -- 172.21.15.31:0/3562712174 shutdown_connections 2023-10-06T21:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/3562712174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf4108440 0x7f1cf410ffc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/3562712174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4103320 0x7f1cf4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/3562712174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103cd0 0x7f1cf4107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 -- 172.21.15.31:0/3562712174 >> 172.21.15.31:0/3562712174 conn(0x7f1cf40feb60 msgr2=0x7f1cf4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 -- 172.21.15.31:0/3562712174 shutdown_connections 2023-10-06T21:45:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.263+0000 7f1cf8fff700 1 -- 172.21.15.31:0/3562712174 wait complete. 2023-10-06T21:45:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.265+0000 7f1cf8fff700 1 Processor -- start 2023-10-06T21:45:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.265+0000 7f1cf8fff700 1 -- start start 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 0x7f1cf410caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf410cfe0 0x7f1cf410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41a9e00 con 0x7f1cf4108440 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41a9f40 con 0x7f1cf4103320 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41aa0d0 con 0x7f1cf410cfe0 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:37918/0 (socket says 172.21.15.31:37918) 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf259c700 1 -- 172.21.15.31:0/2219965796 learned_addr learned my addr 172.21.15.31:0/2219965796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf410cfe0 0x7f1cf410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.266+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 0x7f1cf410caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cf259c700 1 -- 172.21.15.31:0/2219965796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf410cfe0 msgr2=0x7f1cf410d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf410cfe0 0x7f1cf410d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cf259c700 1 -- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 msgr2=0x7f1cf410caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 0x7f1cf410caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cf259c700 1 -- 172.21.15.31:0/2219965796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009520 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.267+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 0x7f1cf410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4007fa0 tx=0x7f1ce400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1ceb7fe700 1 -- 172.21.15.31:0/2219965796 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400dcf0 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1ceb7fe700 1 -- 172.21.15.31:0/2219965796 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ce400f040 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1ceb7fe700 1 -- 172.21.15.31:0/2219965796 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4014600 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf41aa3b0 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.268+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf41aa870 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.272+0000 7f1ceb7fe700 1 -- 172.21.15.31:0/2219965796 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1ce4015920 con 0x7f1cf4103320 2023-10-06T21:45:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.274+0000 7f1ceb7fe700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 0x7f1ce00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.275+0000 7f1ceb7fe700 1 -- 172.21.15.31:0/2219965796 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f1ce40cc740 con 0x7f1cf4103320 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.275+0000 7f1d0c834b80 1 Processor -- start 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.275+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 0x7f1ce00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.275+0000 7f1d0c834b80 1 -- start start 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d221400 con 0x55a93d2b8520 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d212bc0 con 0x55a93d2c33d0 2023-10-06T21:45:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d2104e0 con 0x55a93d2b7c00 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 0x7f1ce00a3c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cf410e4a0 tx=0x7f1cdc009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cf2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cf2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:37922/0 (socket says 172.21.15.31:37922) 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/1702289173 learned_addr learned my addr 172.21.15.31:0/1702289173 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cebfff700 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.276+0000 7f1cf259c700 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/1702289173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 msgr2=0x55a93d2b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/1702289173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 msgr2=0x55a93d2c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/1702289173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93d036d80 con 0x55a93d2c33d0 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0098f0 tx=0x7f1cec005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7307f7b45600aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1ce8ff9700 1 -- 172.21.15.31:0/1702289173 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00fce0 con 0x55a93d2c33d0 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1ce8ff9700 1 -- 172.21.15.31:0/1702289173 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cec01d040 con 0x55a93d2c33d0 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.277+0000 7f1ce8ff9700 1 -- 172.21.15.31:0/1702289173 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec005040 con 0x55a93d2c33d0 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 -- 172.21.15.31:0/1702289173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 msgr2=0x55a93d2c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0098f0 tx=0x7f1cec005de0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 -- 172.21.15.31:0/1702289173 shutdown_connections 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c57c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/1702289173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.278+0000 7f1d0c834b80 1 -- 172.21.15.31:0/1702289173 >> 172.21.15.31:0/1702289173 conn(0x55a93d22d200 msgr2=0x55a93d2b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.279+0000 7f1d0c834b80 1 -- 172.21.15.31:0/1702289173 shutdown_connections 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.279+0000 7f1d0c834b80 1 -- 172.21.15.31:0/1702289173 wait complete. 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.279+0000 7f1d0c834b80 1 -- start start 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.280+0000 7f1d0c834b80 1 Processor -- start 2023-10-06T21:45:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d20bdd0 con 0x55a93d2b8520 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d1a86b0 con 0x55a93d2c33d0 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d1a8350 con 0x55a93d2b7c00 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1d0c834b80 10 client.0 _ll_register_callbacks cb 0x55a93d0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1cebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1cebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43870/0 (socket says 172.21.15.31:43870) 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.281+0000 7f1cebfff700 1 -- 172.21.15.31:0/3221547007 learned_addr learned my addr 172.21.15.31:0/3221547007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf259c700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/3221547007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 msgr2=0x55a93d2c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/3221547007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 msgr2=0x55a93d2c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/3221547007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf4110830 con 0x55a93d2c33d0 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009810 tx=0x7f1cec00bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387089ace6fc1033 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.282+0000 7f1cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:45:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec030070 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f1cec00f460 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a93d2abd50 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec00f620 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1d0c834b80 10 client.27758 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.283+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a93d01fc00 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a93d2abd50 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cec033030 con 0x55a93d2c33d0 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 10 client.27758 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 10 client.27758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 20 client.27758 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d2e3980 0x55a93d2e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 4 client.27758 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.284+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a93d204db0 con 0x55a93d2e3980 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.285+0000 7f1cf259c700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d2e3980 0x55a93d2e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.286+0000 7f1cf259c700 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d2e3980 0x55a93d2e5d70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fbdca6c8b7a3d6 server_cookie=c24b43902d578f69 in_seq=0 out_seq=0 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.286+0000 7f1cd9ffb700 10 client.27758 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/3221547007 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a93d204db0 con 0x55a93d2e3980 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1cd9ffb700 10 client.27758 handle_command_reply: tid=0 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 client.27758 shutdown 2023-10-06T21:45:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 msgr2=0x55a93d2c0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009810 tx=0x7f1cec00bbf0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 shutdown_connections 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0c20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d2e3980 0x55a93d2e5d70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/3221547007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 >> 172.21.15.31:0/3221547007 conn(0x55a93d22d200 msgr2=0x55a93d2b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1cd9ffb700 0 client.27758 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.287+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 shutdown_connections 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.288+0000 7f1d0c834b80 1 -- 172.21.15.31:0/3221547007 wait complete. 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.288+0000 7f1d0c834b80 20 client.27758 trim_cache size 0 max 16384 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.308+0000 7f1d0c834b80 1 Processor -- start 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.308+0000 7f1d0c834b80 1 -- start start 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d08c8c0 con 0x55a93d2b8520 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d2b4be0 con 0x55a93d2c33d0 2023-10-06T21:45:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d2ca3f0 con 0x55a93d2b7c00 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1cebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1cf2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42660/0 (socket says 172.21.15.31:42660) 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.309+0000 7f1cf259c700 1 -- 172.21.15.31:0/823477337 learned_addr learned my addr 172.21.15.31:0/823477337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/823477337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 msgr2=0x55a93d2bab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2bab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/823477337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 msgr2=0x55a93d2c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 -- 172.21.15.31:0/823477337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93d05d1f0 con 0x55a93d2c33d0 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0a50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0092a0 tx=0x7f1cec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b995fcab65b521d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1ce97fa700 1 -- 172.21.15.31:0/823477337 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec030070 con 0x55a93d2c33d0 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.310+0000 7f1ce97fa700 1 -- 172.21.15.31:0/823477337 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cec00fb90 con 0x55a93d2c33d0 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.311+0000 7f1ce97fa700 1 -- 172.21.15.31:0/823477337 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec005480 con 0x55a93d2c33d0 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.311+0000 7f1d0c834b80 1 -- 172.21.15.31:0/823477337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 msgr2=0x55a93d2c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.311+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0a50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0092a0 tx=0x7f1cec009c00 comp rx=0 tx=0).stop 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 -- 172.21.15.31:0/823477337 shutdown_connections 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2c33d0 0x55a93d2c0a50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2bab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/823477337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2b8520 0x55a93d2c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 -- 172.21.15.31:0/823477337 >> 172.21.15.31:0/823477337 conn(0x55a93d22d200 msgr2=0x55a93d2bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 -- 172.21.15.31:0/823477337 shutdown_connections 2023-10-06T21:45:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 -- 172.21.15.31:0/823477337 wait complete. 2023-10-06T21:45:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.312+0000 7f1d0c834b80 1 -- start start 2023-10-06T21:45:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 Processor -- start 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2b8520 0x55a93d2c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2c33d0 0x55a93d2c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d336900 con 0x55a93d2c33d0 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d336a70 con 0x55a93d2b8520 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93d336c30 con 0x55a93d2b7c00 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.313+0000 7f1d0c834b80 10 client.0 _ll_register_callbacks cb 0x7f1ce40993f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:42672/0 (socket says 172.21.15.31:42672) 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 -- 172.21.15.31:0/2218312638 learned_addr learned my addr 172.21.15.31:0/2218312638 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2b8520 0x55a93d2c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf2d9d700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2c33d0 0x55a93d2c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 -- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2b8520 msgr2=0x55a93d2c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2b8520 0x55a93d2c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 -- 172.21.15.31:0/2218312638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2c33d0 msgr2=0x55a93d2c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2c33d0 0x55a93d2c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 -- 172.21.15.31:0/2218312638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a93d01fc00 con 0x55a93d2b7c00 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cf259c700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4099320 tx=0x7f1ce4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78dd9601d3d226cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.314+0000 7f1cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40a19b0 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f1ce400de50 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a93d204ad0 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ce4095d30 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 10 client.27697 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a93d035250 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ce40ac030 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ce4099880 con 0x55a93d2b7c00 2023-10-06T21:45:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 10 client.27697 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 10 client.27697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.315+0000 7f1d0c834b80 20 client.27697 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.316+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d338380 0x55a93d33a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.316+0000 7f1d0c834b80 4 client.27697 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.316+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a93d057f10 con 0x55a93d338380 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.316+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d338380 0x55a93d33a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.317+0000 7f1cebfff700 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d338380 0x55a93d33a770 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a32d6a6568b0538 server_cookie=8586a5f27fd9641b in_seq=0 out_seq=0 2023-10-06T21:45:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.317+0000 7f1cd9ffb700 10 client.27697 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.383+0000 7f1cd9ffb700 1 -- 172.21.15.31:0/2218312638 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a93d057f10 con 0x55a93d338380 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.383+0000 7f1cd9ffb700 10 client.27697 handle_command_reply: tid=0 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.383+0000 7f1d0c834b80 1 client.27697 shutdown 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 msgr2=0x55a93d2c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4099320 tx=0x7f1ce4009780 comp rx=0 tx=0).stop 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 shutdown_connections 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55a93d338380 0x55a93d33a770 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a93d2c33d0 0x55a93d2c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a93d2b7c00 0x55a93d2c7ea0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 --2- 172.21.15.31:0/2218312638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a93d2b8520 0x55a93d2c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 >> 172.21.15.31:0/2218312638 conn(0x55a93d22d200 msgr2=0x55a93d2be950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1cd9ffb700 0 client.27697 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 shutdown_connections 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.384+0000 7f1d0c834b80 1 -- 172.21.15.31:0/2218312638 wait complete. 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.385+0000 7f1d0c834b80 20 client.27697 trim_cache size 0 max 16384 2023-10-06T21:45:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:45:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:45:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:45:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:45:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.386+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 msgr2=0x7f1ce00a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.386+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 0x7f1ce00a3c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cf410e4a0 tx=0x7f1cdc009260 comp rx=0 tx=0).stop 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.386+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 msgr2=0x7f1cf410fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.386+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4007fa0 tx=0x7f1ce400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 shutdown_connections 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf410cfe0 0x7f1cf410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf4103320 0x7f1cf410fca0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1ce00a1740 0x7f1ce00a3c00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 --2- 172.21.15.31:0/2219965796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf4108440 0x7f1cf410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.387+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 >> 172.21.15.31:0/2219965796 conn(0x7f1cf40feb60 msgr2=0x7f1cf40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.389+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 shutdown_connections 2023-10-06T21:45:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:04.389+0000 7f1cf8fff700 1 -- 172.21.15.31:0/2219965796 wait complete. 2023-10-06T21:45:04.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:04 smithi146 ceph-mon[102809]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:45:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:04 smithi142 ceph-mon[102951]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:45:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:04 smithi031 ceph-mon[98539]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:45:04.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77202 6.32 MB/sec execute 533 sec latency 98.684 ms 2023-10-06T21:45:05.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:05 smithi146 ceph-mon[102809]: pgmap v765: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:05.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:05 smithi142 ceph-mon[102951]: pgmap v765: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:05 smithi031 ceph-mon[98539]: pgmap v765: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:05.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77331 6.32 MB/sec execute 534 sec latency 73.069 ms 2023-10-06T21:45:06.795 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77440 6.32 MB/sec execute 535 sec latency 69.282 ms 2023-10-06T21:45:07.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:07 smithi146 ceph-mon[102809]: pgmap v766: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:07.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:07 smithi142 ceph-mon[102951]: pgmap v766: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:07.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:07 smithi031 ceph-mon[98539]: pgmap v766: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:07.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77517 6.31 MB/sec execute 536 sec latency 317.544 ms 2023-10-06T21:45:08.797 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77646 6.31 MB/sec execute 537 sec latency 66.898 ms 2023-10-06T21:45:09.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:09 smithi146 ceph-mon[102809]: pgmap v767: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:09.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:09 smithi142 ceph-mon[102951]: pgmap v767: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:09 smithi031 ceph-mon[98539]: pgmap v767: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 229 op/s 2023-10-06T21:45:09.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77767 6.32 MB/sec execute 538 sec latency 79.031 ms 2023-10-06T21:45:10.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:10 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:45:10.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:10 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:45:10.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:10 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:45:10.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77888 6.32 MB/sec execute 539 sec latency 150.861 ms 2023-10-06T21:45:11.551 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:11 smithi146 ceph-mon[102809]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:11 smithi142 ceph-mon[102951]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:11.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:11 smithi031 ceph-mon[98539]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:11.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 77986 6.31 MB/sec execute 540 sec latency 160.953 ms 2023-10-06T21:45:12.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78096 6.31 MB/sec execute 541 sec latency 82.281 ms 2023-10-06T21:45:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:13 smithi146 ceph-mon[102809]: pgmap v769: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:45:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:45:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:45:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:13 smithi142 ceph-mon[102951]: pgmap v769: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:45:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:45:13.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:45:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:13 smithi031 ceph-mon[98539]: pgmap v769: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2023-10-06T21:45:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:45:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:45:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:45:13.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78172 6.30 MB/sec execute 542 sec latency 294.800 ms 2023-10-06T21:45:14.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78266 6.29 MB/sec execute 543 sec latency 82.938 ms 2023-10-06T21:45:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:15 smithi031 ceph-mon[98539]: pgmap v770: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 254 op/s 2023-10-06T21:45:15.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:15 smithi031 ceph-mon[98539]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:45:15.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:15 smithi146 ceph-mon[102809]: pgmap v770: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 254 op/s 2023-10-06T21:45:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:15 smithi146 ceph-mon[102809]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:45:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:15 smithi142 ceph-mon[102951]: pgmap v770: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 254 op/s 2023-10-06T21:45:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:15 smithi142 ceph-mon[102951]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:45:15.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78389 6.29 MB/sec execute 544 sec latency 71.084 ms 2023-10-06T21:45:16.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78526 6.30 MB/sec execute 545 sec latency 78.589 ms 2023-10-06T21:45:17.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:17 smithi031 ceph-mon[98539]: pgmap v772: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:17.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:17 smithi146 ceph-mon[102809]: pgmap v772: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/2624356089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc068c90 msgr2=0x7fc2cc10db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/2624356089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc068c90 0x7fc2cc10db40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40099c0 tx=0x7fc2c4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/2624356089 shutdown_connections 2023-10-06T21:45:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/2624356089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc068c90 0x7fc2cc10db40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/2624356089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc106d30 0x7fc2cc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/2624356089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/2624356089 >> 172.21.15.31:0/2624356089 conn(0x7fc2cc0756e0 msgr2=0x7fc2cc075af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/2624356089 shutdown_connections 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.619+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/2624356089 wait complete. 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.620+0000 7fc2d0d9d700 1 Processor -- start 2023-10-06T21:45:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.620+0000 7fc2d0d9d700 1 -- start start 2023-10-06T21:45:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 0x7fc2cc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 0x7fc2cc19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc19bd00 con 0x7fc2cc106d30 2023-10-06T21:45:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc19be70 con 0x7fc2cc068c90 2023-10-06T21:45:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2d0d9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc19c030 con 0x7fc2cc0682d0 2023-10-06T21:45:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.621+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2caffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 0x7fc2cc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60702/0 (socket says 172.21.15.31:60702) 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/3287424525 learned_addr learned my addr 172.21.15.31:0/3287424525 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 0x7fc2cc19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/3287424525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 msgr2=0x7fc2cc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 0x7fc2cc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 msgr2=0x7fc2cc19b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 0x7fc2cc19b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/3287424525 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40095f0 con 0x7fc2cc0682d0 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2caffd700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 0x7fc2cc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 0x7fc2cc19b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.622+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00ba50 tx=0x7fc2bc00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.623+0000 7fc2c8ff9700 1 -- 172.21.15.31:0/3287424525 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00c930 con 0x7fc2cc0682d0 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.623+0000 7fc2c8ff9700 1 -- 172.21.15.31:0/3287424525 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2bc014440 con 0x7fc2cc0682d0 2023-10-06T21:45:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.623+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc19c310 con 0x7fc2cc0682d0 2023-10-06T21:45:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.623+0000 7fc2c8ff9700 1 -- 172.21.15.31:0/3287424525 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0075d0 con 0x7fc2cc0682d0 2023-10-06T21:45:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.623+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc1a3b50 con 0x7fc2cc0682d0 2023-10-06T21:45:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.627+0000 7fc2c8ff9700 1 -- 172.21.15.31:0/3287424525 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc2bc0145b0 con 0x7fc2cc0682d0 2023-10-06T21:45:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.628+0000 7fc2c8ff9700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 0x7fc2b40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.629+0000 7fc2e59f2b80 1 Processor -- start 2023-10-06T21:45:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.629+0000 7fc2c8ff9700 1 -- 172.21.15.31:0/3287424525 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc2bc006e40 con 0x7fc2cc0682d0 2023-10-06T21:45:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.629+0000 7fc2caffd700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 0x7fc2b40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.629+0000 7fc2e59f2b80 1 -- start start 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e39920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d95400 con 0x55c741e37530 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d86bc0 con 0x55c741e2c680 2023-10-06T21:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d844e0 con 0x55c741e2bd60 2023-10-06T21:45:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e39920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60718/0 (socket says 172.21.15.31:60718) 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/743565228 learned_addr learned my addr 172.21.15.31:0/743565228 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.630+0000 7fc2caffd700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2caffd700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 0x7fc2b40a3b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc198330 tx=0x7fc2c0009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/743565228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 msgr2=0x55c741e36ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e36ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/743565228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 msgr2=0x55c741e39920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e39920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/743565228 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c741baad80 con 0x55c741e2bd60 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2caffd700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0963a0 tx=0x7fc2bc09a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecddbd366fef5a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.631+0000 7fc2b1ffb700 1 -- 172.21.15.31:0/743565228 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc09da80 con 0x55c741e2bd60 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.632+0000 7fc2b1ffb700 1 -- 172.21.15.31:0/743565228 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2bc0a1cc0 con 0x55c741e2bd60 2023-10-06T21:45:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.632+0000 7fc2b1ffb700 1 -- 172.21.15.31:0/743565228 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc099df0 con 0x55c741e2bd60 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.632+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/743565228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 msgr2=0x55c741e2c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.632+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0963a0 tx=0x7fc2bc09a0b0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/743565228 shutdown_connections 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e2c140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/743565228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/743565228 >> 172.21.15.31:0/743565228 conn(0x55c741da1200 msgr2=0x55c741e281b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/743565228 shutdown_connections 2023-10-06T21:45:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/743565228 wait complete. 2023-10-06T21:45:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.633+0000 7fc2e59f2b80 1 -- start start 2023-10-06T21:45:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.634+0000 7fc2e59f2b80 1 Processor -- start 2023-10-06T21:45:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.634+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e36fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e34910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d7fdd0 con 0x55c741e37530 2023-10-06T21:45:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d1c6b0 con 0x55c741e2bd60 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741d1c350 con 0x55c741e2c680 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2e59f2b80 10 client.0 _ll_register_callbacks cb 0x55c741b797a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e36fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e34910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e34910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35152/0 (socket says 172.21.15.31:35152) 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2cbfff700 1 -- 172.21.15.31:0/4123937455 learned_addr learned my addr 172.21.15.31:0/4123937455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e36fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33780/0 (socket says 172.21.15.31:33780) 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.635+0000 7fc2caffd700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 -- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 msgr2=0x55c741e36fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e36fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 -- 172.21.15.31:0/4123937455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 msgr2=0x55c741e34910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e34910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 -- 172.21.15.31:0/4123937455 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c0017070 con 0x55c741e2c680 2023-10-06T21:45:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2caffd700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e34220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003680 tx=0x7fc2c0013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60666bf5c6db8097 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:45:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.636+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0021040 con 0x55c741e2c680 2023-10-06T21:45:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc2c0003e20 con 0x55c741e2c680 2023-10-06T21:45:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c741e1feb0 con 0x55c741e2c680 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c001c830 con 0x55c741e2c680 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2e59f2b80 10 client.27709 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fc2cc10e440 con 0x55c741e2c680 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.637+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c741e1feb0 con 0x55c741e2c680 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2c0026bc0 con 0x55c741e2c680 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 10 client.27709 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 10 client.27709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 20 client.27709 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741e4b560 0x55c741e4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 4 client.27709 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:45:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c741d78db0 con 0x55c741e4b560 2023-10-06T21:45:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.638+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741e4b560 0x55c741e4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.639+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741e4b560 0x55c741e4d980 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a61f7fb026c2f1 server_cookie=ae628cca568b121a in_seq=0 out_seq=0 2023-10-06T21:45:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.639+0000 7fc2aaffd700 10 client.27709 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2aaffd700 1 -- 172.21.15.31:0/4123937455 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c741d78db0 con 0x55c741e4b560 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2aaffd700 10 client.27709 handle_command_reply: tid=0 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 client.27709 shutdown 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 msgr2=0x55c741e34220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e34220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003680 tx=0x7fc2c0013040 comp rx=0 tx=0).stop 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 shutdown_connections 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741e4b560 0x55c741e4d980 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/4123937455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e34220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 >> 172.21.15.31:0/4123937455 conn(0x55c741da1200 msgr2=0x55c741e27d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.640+0000 7fc2aaffd700 0 client.27709 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.641+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 shutdown_connections 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.641+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/4123937455 wait complete. 2023-10-06T21:45:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.641+0000 7fc2e59f2b80 20 client.27709 trim_cache size 0 max 16384 2023-10-06T21:45:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:17 smithi142 ceph-mon[102951]: pgmap v772: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.666+0000 7fc2e59f2b80 1 Processor -- start 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.666+0000 7fc2e59f2b80 1 -- start start 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e33a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741c008c0 con 0x55c741e37530 2023-10-06T21:45:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741e226b0 con 0x55c741e2bd60 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741e28ef0 con 0x55c741e2c680 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:33784/0 (socket says 172.21.15.31:33784) 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2331698655 learned_addr learned my addr 172.21.15.31:0/2331698655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.667+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2331698655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 msgr2=0x55c741e33a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2caffd700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e33a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e33a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2331698655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 msgr2=0x55c741e36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e36240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2331698655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc1a3b50 con 0x55c741e2bd60 2023-10-06T21:45:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:45:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2caffd700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e33a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:45:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.668+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc099630 tx=0x7fc2bc095da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1220ef101a9e23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.669+0000 7fc2b27fc700 1 -- 172.21.15.31:0/2331698655 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0b2230 con 0x55c741e2bd60 2023-10-06T21:45:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.669+0000 7fc2b27fc700 1 -- 172.21.15.31:0/2331698655 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2bc009930 con 0x55c741e2bd60 2023-10-06T21:45:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.669+0000 7fc2b27fc700 1 -- 172.21.15.31:0/2331698655 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc099b90 con 0x55c741e2bd60 2023-10-06T21:45:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.670+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2331698655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 msgr2=0x55c741e2ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.670+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc099630 tx=0x7fc2bc095da0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.672+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2331698655 shutdown_connections 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.672+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.672+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2bd60 0x55c741e2ec00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.673+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2331698655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2c680 0x55c741e33a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.673+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2331698655 >> 172.21.15.31:0/2331698655 conn(0x55c741da1200 msgr2=0x55c741e35b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.673+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2331698655 shutdown_connections 2023-10-06T21:45:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.673+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2331698655 wait complete. 2023-10-06T21:45:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 -- start start 2023-10-06T21:45:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 Processor -- start 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e50440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e50980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741e50f50 con 0x55c741e37530 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741e51090 con 0x55c741e2c680 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.674+0000 7fc2e59f2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741e51250 con 0x55c741e2bd60 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2e59f2b80 10 client.0 _ll_register_callbacks cb 0x7fc2c00043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:60752/0 (socket says 172.21.15.31:60752) 2023-10-06T21:45:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2590261833 learned_addr learned my addr 172.21.15.31:0/2590261833 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2caffd700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e50440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cbfff700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e50980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 msgr2=0x55c741e50440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e50440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2590261833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 msgr2=0x55c741e50980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e50980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.675+0000 7fc2cb7fe700 1 -- 172.21.15.31:0/2590261833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c741baad80 con 0x55c741e2bd60 2023-10-06T21:45:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2cb7fe700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc084010 tx=0x7fc2bc0b21b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a899ea1782a229 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc014ae0 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc2bc0b90d0 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c741d78ad0 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2bc0b9290 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2e59f2b80 10 client.27712 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.676+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c741b93c00 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c741d78ad0 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2bc0a1830 con 0x55c741e2bd60 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 10 client.27712 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 10 client.27712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 20 client.27712 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741ea0220 0x55c741ea2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 4 client.27712 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:45:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.677+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c741bcbf10 con 0x55c741ea0220 2023-10-06T21:45:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.678+0000 7fc2caffd700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741ea0220 0x55c741ea2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.679+0000 7fc2caffd700 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741ea0220 0x55c741ea2610 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e11aea19dae014 server_cookie=e385bf00b56a3329 in_seq=0 out_seq=0 2023-10-06T21:45:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.679+0000 7fc2aaffd700 10 client.27712 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2aaffd700 1 -- 172.21.15.31:0/2590261833 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c741bcbf10 con 0x55c741ea0220 2023-10-06T21:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2aaffd700 10 client.27712 handle_command_reply: tid=0 2023-10-06T21:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 client.27712 shutdown 2023-10-06T21:45:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 msgr2=0x55c741e52050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc084010 tx=0x7fc2bc0b21b0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 shutdown_connections 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55c741ea0220 0x55c741ea2610 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c741e37530 0x55c741e50980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c741e2bd60 0x55c741e52050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2aaffd700 0 client.27712 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 --2- 172.21.15.31:0/2590261833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c741e2c680 0x55c741e50440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.746+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 >> 172.21.15.31:0/2590261833 conn(0x55c741da1200 msgr2=0x55c741e46d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.747+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 shutdown_connections 2023-10-06T21:45:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.747+0000 7fc2e59f2b80 1 -- 172.21.15.31:0/2590261833 wait complete. 2023-10-06T21:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.747+0000 7fc2e59f2b80 20 client.27712 trim_cache size 0 max 16384 2023-10-06T21:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:45:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 msgr2=0x7fc2b40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 0x7fc2b40a3b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc198330 tx=0x7fc2c0009260 comp rx=0 tx=0).stop 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 msgr2=0x7fc2cc196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00ba50 tx=0x7fc2bc00bd60 comp rx=0 tx=0).stop 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 shutdown_connections 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2b40a1670 0x7fc2b40a3b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2cc106d30 0x7fc2cc19b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2cc0682d0 0x7fc2cc196d20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 --2- 172.21.15.31:0/3287424525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc068c90 0x7fc2cc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.748+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 >> 172.21.15.31:0/3287424525 conn(0x7fc2cc0756e0 msgr2=0x7fc2cc0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.749+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 shutdown_connections 2023-10-06T21:45:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:17.750+0000 7fc2d0d9d700 1 -- 172.21.15.31:0/3287424525 wait complete. 2023-10-06T21:45:17.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78656 6.30 MB/sec execute 546 sec latency 98.807 ms 2023-10-06T21:45:18.796 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78806 6.30 MB/sec execute 547 sec latency 65.467 ms 2023-10-06T21:45:19.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:19 smithi031 ceph-mon[98539]: pgmap v773: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:19.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:19 smithi146 ceph-mon[102809]: pgmap v773: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:19.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:19 smithi142 ceph-mon[102951]: pgmap v773: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-06T21:45:19.798 INFO:tasks.workunit.client.0.smithi031.stdout: 10 78942 6.31 MB/sec execute 548 sec latency 69.525 ms 2023-10-06T21:45:20.798 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79047 6.31 MB/sec execute 549 sec latency 67.539 ms 2023-10-06T21:45:21.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:21 smithi146 ceph-mon[102809]: pgmap v774: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:21 smithi142 ceph-mon[102951]: pgmap v774: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:21 smithi031 ceph-mon[98539]: pgmap v774: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:21.798 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79219 6.33 MB/sec execute 550 sec latency 78.650 ms 2023-10-06T21:45:22.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79340 6.33 MB/sec execute 551 sec latency 72.677 ms 2023-10-06T21:45:23.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:23 smithi146 ceph-mon[102809]: pgmap v775: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:23 smithi142 ceph-mon[102951]: pgmap v775: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:23.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:23 smithi031 ceph-mon[98539]: pgmap v775: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2023-10-06T21:45:23.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79468 6.34 MB/sec execute 552 sec latency 79.442 ms 2023-10-06T21:45:24.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79563 6.34 MB/sec execute 553 sec latency 147.746 ms 2023-10-06T21:45:25.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:25 smithi146 ceph-mon[102809]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 206 op/s 2023-10-06T21:45:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:25 smithi142 ceph-mon[102951]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 206 op/s 2023-10-06T21:45:25.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:25 smithi031 ceph-mon[98539]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 206 op/s 2023-10-06T21:45:25.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79660 6.34 MB/sec execute 554 sec latency 226.366 ms 2023-10-06T21:45:26.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79776 6.34 MB/sec execute 555 sec latency 259.259 ms 2023-10-06T21:45:27.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:27 smithi146 ceph-mon[102809]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 185 op/s 2023-10-06T21:45:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:27 smithi142 ceph-mon[102951]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 185 op/s 2023-10-06T21:45:27.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:27 smithi031 ceph-mon[98539]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 185 op/s 2023-10-06T21:45:27.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79886 6.34 MB/sec execute 556 sec latency 98.757 ms 2023-10-06T21:45:28.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 79986 6.33 MB/sec execute 557 sec latency 75.278 ms 2023-10-06T21:45:29.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:29 smithi146 ceph-mon[102809]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 172 op/s 2023-10-06T21:45:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:29 smithi142 ceph-mon[102951]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 172 op/s 2023-10-06T21:45:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:29 smithi031 ceph-mon[98539]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 172 op/s 2023-10-06T21:45:29.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80085 6.32 MB/sec execute 558 sec latency 71.927 ms 2023-10-06T21:45:30.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80179 6.32 MB/sec execute 559 sec latency 70.082 ms 2023-10-06T21:45:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.988+0000 7f114539d700 1 -- 172.21.15.31:0/3808546369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140105cc0 msgr2=0x7f11400682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.988+0000 7f114539d700 1 --2- 172.21.15.31:0/3808546369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140105cc0 0x7f11400682d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11300098f0 tx=0x7f1130009c00 comp rx=0 tx=0).stop 2023-10-06T21:45:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 -- 172.21.15.31:0/3808546369 shutdown_connections 2023-10-06T21:45:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 --2- 172.21.15.31:0/3808546369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140103710 0x7f1140103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 --2- 172.21.15.31:0/3808546369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140105cc0 0x7f11400682d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 --2- 172.21.15.31:0/3808546369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f1140068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 -- 172.21.15.31:0/3808546369 >> 172.21.15.31:0/3808546369 conn(0x7f11400756e0 msgr2=0x7f1140075af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 -- 172.21.15.31:0/3808546369 shutdown_connections 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.989+0000 7f114539d700 1 -- 172.21.15.31:0/3808546369 wait complete. 2023-10-06T21:45:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.990+0000 7f114539d700 1 Processor -- start 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 -- start start 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140103710 0x7f114019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401a2320 con 0x7f1140068810 2023-10-06T21:45:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401a2490 con 0x7f1140103710 2023-10-06T21:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.991+0000 7f114539d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401a2650 con 0x7f1140105cc0 2023-10-06T21:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f113ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55486/0 (socket says 172.21.15.31:55486) 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f113ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52146/0 (socket says 172.21.15.31:52146) 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 -- 172.21.15.31:0/4243756127 learned_addr learned my addr 172.21.15.31:0/4243756127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 -- 172.21.15.31:0/4243756127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140103710 msgr2=0x7f114019d880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140103710 0x7f114019d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 -- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 msgr2=0x7f114019d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f1144b9c700 1 -- 172.21.15.31:0/4243756127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009d00 con 0x7f1140105cc0 2023-10-06T21:45:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.992+0000 7f113ffff700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f1144b9c700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f113800e3c0 tx=0x7f113800e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f113d7fa700 1 -- 172.21.15.31:0/4243756127 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380092d0 con 0x7f1140105cc0 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f113d7fa700 1 -- 172.21.15.31:0/4243756127 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f113800f040 con 0x7f1140105cc0 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1130009520 con 0x7f1140105cc0 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f113d7fa700 1 -- 172.21.15.31:0/4243756127 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138007bf0 con 0x7f1140105cc0 2023-10-06T21:45:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.993+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11401a9f90 con 0x7f1140105cc0 2023-10-06T21:45:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:30.998+0000 7f113d7fa700 1 -- 172.21.15.31:0/4243756127 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1138007d50 con 0x7f1140105cc0 2023-10-06T21:45:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.000+0000 7f115a001b80 1 Processor -- start 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.000+0000 7f115a001b80 1 -- start start 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113d7fa700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 0x7f11280a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5e3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d53f400 con 0x56074d5d6680 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d530bc0 con 0x56074d5e1530 2023-10-06T21:45:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f115a001b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d52e4e0 con 0x56074d5d5d60 2023-10-06T21:45:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113d7fa700 1 -- 172.21.15.31:0/4243756127 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f11380cced0 con 0x7f1140105cc0 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55488/0 (socket says 172.21.15.31:55488) 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.001+0000 7f113ffff700 1 -- 172.21.15.31:0/1009326903 learned_addr learned my addr 172.21.15.31:0/1009326903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f1144b9c700 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5e3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113f7fe700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 0x7f11280a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 -- 172.21.15.31:0/1009326903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 msgr2=0x56074d5e3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5e3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 -- 172.21.15.31:0/1009326903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 msgr2=0x56074d5e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 -- 172.21.15.31:0/1009326903 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56074d354d80 con 0x56074d5d5d60 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f1144b9c700 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5e3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.002+0000 7f113ffff700 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f1130005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b35c1401f72355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.003+0000 7f113f7fe700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 0x7f11280a3b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f114019e950 tx=0x7f1134009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.003+0000 7f11267fc700 1 -- 172.21.15.31:0/1009326903 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113001c050 con 0x56074d5d5d60 2023-10-06T21:45:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.003+0000 7f11267fc700 1 -- 172.21.15.31:0/1009326903 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f113001d040 con 0x56074d5d5d60 2023-10-06T21:45:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.003+0000 7f11267fc700 1 -- 172.21.15.31:0/1009326903 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300052a0 con 0x56074d5d5d60 2023-10-06T21:45:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 -- 172.21.15.31:0/1009326903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 msgr2=0x56074d5d6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f1130005dc0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 -- 172.21.15.31:0/1009326903 shutdown_connections 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5e3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5d6140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 --2- 172.21.15.31:0/1009326903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 -- 172.21.15.31:0/1009326903 >> 172.21.15.31:0/1009326903 conn(0x56074d54b200 msgr2=0x56074d5d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 -- 172.21.15.31:0/1009326903 shutdown_connections 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.004+0000 7f115a001b80 1 -- 172.21.15.31:0/1009326903 wait complete. 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.005+0000 7f115a001b80 1 -- start start 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.005+0000 7f115a001b80 1 Processor -- start 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d529dd0 con 0x56074d5d6680 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d4c66b0 con 0x56074d5e1530 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d4c6350 con 0x56074d5d5d60 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f115a001b80 10 client.0 _ll_register_callbacks cb 0x56074d3237a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f113f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5de940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:51978/0 (socket says 172.21.15.31:51978) 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f1144b9c700 1 -- 172.21.15.31:0/1019225984 learned_addr learned my addr 172.21.15.31:0/1019225984 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.006+0000 7f113ffff700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113ffff700 1 -- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 msgr2=0x56074d5de940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113ffff700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5de940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113ffff700 1 -- 172.21.15.31:0/1019225984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 msgr2=0x56074d5de200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113ffff700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5de200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113ffff700 1 -- 172.21.15.31:0/1019225984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1140114be0 con 0x56074d5d5d60 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.007+0000 7f113f7fe700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.008+0000 7f113ffff700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1130009ff0 tx=0x7f113000b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c2de2fa0078766 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.008+0000 7f111f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:45:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.008+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113001c040 con 0x56074d5d5d60 2023-10-06T21:45:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.008+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1130005240 con 0x56074d5d5d60 2023-10-06T21:45:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.008+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56074d5c9eb0 con 0x56074d5d5d60 2023-10-06T21:45:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.009+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f113000f8e0 con 0x56074d5d5d60 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.009+0000 7f115a001b80 10 client.27724 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.009+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x56074d33dc00 con 0x56074d5d5d60 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.009+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56074d5c9eb0 con 0x56074d5d5d60 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f113002a040 con 0x56074d5d5d60 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 10 client.27724 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 10 client.27724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 20 client.27724 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d5f0bc0 0x56074d5f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 4 client.27724 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56074d522db0 con 0x56074d5f0bc0 2023-10-06T21:45:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.010+0000 7f113f7fe700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d5f0bc0 0x56074d5f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.011+0000 7f113f7fe700 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d5f0bc0 0x56074d5f2fb0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115a4df139fd2e1d server_cookie=f1ebbfc9114d1563 in_seq=0 out_seq=0 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.011+0000 7f111f7fe700 10 client.27724 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f111f7fe700 1 -- 172.21.15.31:0/1019225984 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56074d522db0 con 0x56074d5f0bc0 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f111f7fe700 10 client.27724 handle_command_reply: tid=0 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 client.27724 shutdown 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 msgr2=0x56074d5e0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1130009ff0 tx=0x7f113000b7f0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 shutdown_connections 2023-10-06T21:45:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0ff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d5f0bc0 0x56074d5f2fb0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 --2- 172.21.15.31:0/1019225984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 >> 172.21.15.31:0/1019225984 conn(0x56074d54b200 msgr2=0x56074d5d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.012+0000 7f111f7fe700 0 client.27724 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.013+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 shutdown_connections 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.013+0000 7f115a001b80 1 -- 172.21.15.31:0/1019225984 wait complete. 2023-10-06T21:45:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.013+0000 7f115a001b80 20 client.27724 trim_cache size 0 max 16384 2023-10-06T21:45:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.039+0000 7f115a001b80 1 Processor -- start 2023-10-06T21:45:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 -- start start 2023-10-06T21:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d3aa8c0 con 0x56074d5d6680 2023-10-06T21:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d5d6ac0 con 0x56074d5e1530 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f115a001b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d5d0f90 con 0x56074d5d5d60 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f113ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f113ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:55498/0 (socket says 172.21.15.31:55498) 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.040+0000 7f113ffff700 1 -- 172.21.15.31:0/1638645603 learned_addr learned my addr 172.21.15.31:0/1638645603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f1144b9c700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113ffff700 1 -- 172.21.15.31:0/1638645603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 msgr2=0x56074d5ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113ffff700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113f7fe700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113ffff700 1 -- 172.21.15.31:0/1638645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 msgr2=0x56074d5ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113ffff700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113ffff700 1 -- 172.21.15.31:0/1638645603 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56074d37b1f0 con 0x56074d5d5d60 2023-10-06T21:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.041+0000 7f113f7fe700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.042+0000 7f113ffff700 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1130009c40 tx=0x7f113000bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2706fb08da7a9549 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.042+0000 7f1126ffd700 1 -- 172.21.15.31:0/1638645603 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113000b860 con 0x56074d5d5d60 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.042+0000 7f1126ffd700 1 -- 172.21.15.31:0/1638645603 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1130003b10 con 0x56074d5d5d60 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.042+0000 7f1126ffd700 1 -- 172.21.15.31:0/1638645603 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130005670 con 0x56074d5d5d60 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 -- 172.21.15.31:0/1638645603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 msgr2=0x56074d5e0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1130009c40 tx=0x7f113000bf80 comp rx=0 tx=0).stop 2023-10-06T21:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 -- 172.21.15.31:0/1638645603 shutdown_connections 2023-10-06T21:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5e1530 0x56074d5ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d5d60 0x56074d5e0b20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 --2- 172.21.15.31:0/1638645603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5d6680 0x56074d5ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 -- 172.21.15.31:0/1638645603 >> 172.21.15.31:0/1638645603 conn(0x56074d54b200 msgr2=0x56074d5d9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 -- 172.21.15.31:0/1638645603 shutdown_connections 2023-10-06T21:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.043+0000 7f115a001b80 1 -- 172.21.15.31:0/1638645603 wait complete. 2023-10-06T21:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.044+0000 7f115a001b80 1 -- start start 2023-10-06T21:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.044+0000 7f115a001b80 1 Processor -- start 2023-10-06T21:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5d5d60 0x56074d5e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d6680 0x56074d5e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d643af0 con 0x56074d5e1530 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d643c60 con 0x56074d5d5d60 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074d643e20 con 0x56074d5d6680 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.045+0000 7f115a001b80 10 client.0 _ll_register_callbacks cb 0x7f11340096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52190/0 (socket says 172.21.15.31:52190) 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 -- 172.21.15.31:0/2182701593 learned_addr learned my addr 172.21.15.31:0/2182701593 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f113f7fe700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d6680 0x56074d5e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 -- 172.21.15.31:0/2182701593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d6680 msgr2=0x56074d5e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d6680 0x56074d5e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 -- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5d5d60 msgr2=0x56074d5e5860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5d5d60 0x56074d5e5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.046+0000 7f1144b9c700 1 -- 172.21.15.31:0/2182701593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56074d33dc00 con 0x56074d5e1530 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f1144b9c700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f11380090a0 tx=0x7f1138096210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4911829694e4148d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f111f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138099ed0 con 0x56074d5e1530 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f113809e020 con 0x56074d5e1530 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f113001d040 con 0x56074d5e1530 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11380a21d0 con 0x56074d5e1530 2023-10-06T21:45:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f115a001b80 10 client.18063 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.047+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x56074d353250 con 0x56074d5e1530 2023-10-06T21:45:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f113001d040 con 0x56074d5e1530 2023-10-06T21:45:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1138099430 con 0x56074d5e1530 2023-10-06T21:45:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 10 client.18063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 20 client.18063 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d6454e0 0x56074d647900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 4 client.18063 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.048+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56074d522ad0 con 0x56074d6454e0 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.049+0000 7f113ffff700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d6454e0 0x56074d647900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.049+0000 7f111f7fe700 10 client.18063 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.049+0000 7f113ffff700 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d6454e0 0x56074d647900 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dcea02479b55bd server_cookie=391dc9a040ca27c8 in_seq=0 out_seq=0 2023-10-06T21:45:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.108+0000 7f111f7fe700 1 -- 172.21.15.31:0/2182701593 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56074d522ad0 con 0x56074d6454e0 2023-10-06T21:45:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.108+0000 7f111f7fe700 10 client.18063 handle_command_reply: tid=0 2023-10-06T21:45:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.108+0000 7f115a001b80 1 client.18063 shutdown 2023-10-06T21:45:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 msgr2=0x56074d5e62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f11380090a0 tx=0x7f1138096210 comp rx=0 tx=0).stop 2023-10-06T21:45:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 shutdown_connections 2023-10-06T21:45:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x56074d6454e0 0x56074d647900 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56074d5e1530 0x56074d5e62e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56074d5d5d60 0x56074d5e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 --2- 172.21.15.31:0/2182701593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56074d5d6680 0x56074d5e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 >> 172.21.15.31:0/2182701593 conn(0x56074d54b200 msgr2=0x56074d5d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f111f7fe700 0 client.18063 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 shutdown_connections 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.109+0000 7f115a001b80 1 -- 172.21.15.31:0/2182701593 wait complete. 2023-10-06T21:45:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.110+0000 7f115a001b80 20 client.18063 trim_cache size 0 max 16384 2023-10-06T21:45:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:45:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:45:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:45:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 msgr2=0x7f11280a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 0x7f11280a3b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f114019e950 tx=0x7f1134009260 comp rx=0 tx=0).stop 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 msgr2=0x7f11401a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f113800e3c0 tx=0x7f113800e6d0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 shutdown_connections 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140105cc0 0x7f11401a1ce0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f11280a1670 0x7f11280a3b30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1140068810 0x7f114019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 --2- 172.21.15.31:0/4243756127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140103710 0x7f114019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 >> 172.21.15.31:0/4243756127 conn(0x7f11400756e0 msgr2=0x7f11400fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 shutdown_connections 2023-10-06T21:45:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:31.111+0000 7f114539d700 1 -- 172.21.15.31:0/4243756127 wait complete. 2023-10-06T21:45:31.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:31 smithi146 ceph-mon[102809]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 257 op/s 2023-10-06T21:45:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:31 smithi142 ceph-mon[102951]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 257 op/s 2023-10-06T21:45:31.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:31 smithi031 ceph-mon[98539]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 257 op/s 2023-10-06T21:45:31.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80281 6.31 MB/sec execute 560 sec latency 80.336 ms 2023-10-06T21:45:32.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80385 6.31 MB/sec execute 561 sec latency 115.624 ms 2023-10-06T21:45:33.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:33 smithi146 ceph-mon[102809]: pgmap v780: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 171 op/s 2023-10-06T21:45:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:33 smithi142 ceph-mon[102951]: pgmap v780: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 171 op/s 2023-10-06T21:45:33.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:33 smithi031 ceph-mon[98539]: pgmap v780: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 171 op/s 2023-10-06T21:45:33.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80485 6.30 MB/sec execute 562 sec latency 250.562 ms 2023-10-06T21:45:34.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80603 6.30 MB/sec execute 563 sec latency 324.739 ms 2023-10-06T21:45:35.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:35 smithi146 ceph-mon[102809]: pgmap v781: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 274 op/s 2023-10-06T21:45:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:35 smithi142 ceph-mon[102951]: pgmap v781: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 274 op/s 2023-10-06T21:45:35.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:35 smithi031 ceph-mon[98539]: pgmap v781: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 274 op/s 2023-10-06T21:45:35.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80724 6.30 MB/sec execute 564 sec latency 77.794 ms 2023-10-06T21:45:36.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80841 6.30 MB/sec execute 565 sec latency 94.992 ms 2023-10-06T21:45:37.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:37 smithi146 ceph-mon[102809]: pgmap v782: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:37.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:37 smithi142 ceph-mon[102951]: pgmap v782: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:37.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:37 smithi031 ceph-mon[98539]: pgmap v782: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:37.799 INFO:tasks.workunit.client.0.smithi031.stdout: 10 80959 6.31 MB/sec execute 566 sec latency 71.697 ms 2023-10-06T21:45:38.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81102 6.31 MB/sec execute 567 sec latency 73.231 ms 2023-10-06T21:45:39.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:39 smithi146 ceph-mon[102809]: pgmap v783: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:39.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:39 smithi142 ceph-mon[102951]: pgmap v783: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:39.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:39 smithi031 ceph-mon[98539]: pgmap v783: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2023-10-06T21:45:39.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81234 6.31 MB/sec execute 568 sec latency 70.676 ms 2023-10-06T21:45:40.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81346 6.32 MB/sec execute 569 sec latency 219.366 ms 2023-10-06T21:45:41.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:41 smithi146 ceph-mon[102809]: pgmap v784: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 291 op/s 2023-10-06T21:45:41.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:41 smithi142 ceph-mon[102951]: pgmap v784: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 291 op/s 2023-10-06T21:45:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:41 smithi031 ceph-mon[98539]: pgmap v784: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 291 op/s 2023-10-06T21:45:41.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81454 6.31 MB/sec execute 570 sec latency 246.044 ms 2023-10-06T21:45:42.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81564 6.31 MB/sec execute 571 sec latency 76.779 ms 2023-10-06T21:45:43.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:43 smithi142 ceph-mon[102951]: pgmap v785: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 205 op/s 2023-10-06T21:45:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:43 smithi031 ceph-mon[98539]: pgmap v785: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 205 op/s 2023-10-06T21:45:43.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:43 smithi146 ceph-mon[102809]: pgmap v785: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 205 op/s 2023-10-06T21:45:43.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81676 6.30 MB/sec execute 572 sec latency 67.989 ms 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 -- 172.21.15.31:0/23602856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 msgr2=0x7ff4701003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 --2- 172.21.15.31:0/23602856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff4701003b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff458009990 tx=0x7ff458009ca0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 -- 172.21.15.31:0/23602856 shutdown_connections 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 --2- 172.21.15.31:0/23602856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff47010cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 --2- 172.21.15.31:0/23602856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff4701003b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 --2- 172.21.15.31:0/23602856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff470100980 0x7ff4701049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 -- 172.21.15.31:0/23602856 >> 172.21.15.31:0/23602856 conn(0x7ff470076ea0 msgr2=0x7ff4700772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.351+0000 7ff476dff700 1 -- 172.21.15.31:0/23602856 shutdown_connections 2023-10-06T21:45:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.352+0000 7ff476dff700 1 -- 172.21.15.31:0/23602856 wait complete. 2023-10-06T21:45:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.353+0000 7ff476dff700 1 Processor -- start 2023-10-06T21:45:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.353+0000 7ff476dff700 1 -- start start 2023-10-06T21:45:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff470199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff470199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 0x7ff47019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47019e770 con 0x7ff4701050f0 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47019e8e0 con 0x7ff47019dcf0 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff476dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47019eaa0 con 0x7ff4700fffd0 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff47539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 0x7ff47019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff470199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff47539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 0x7ff47019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52950/0 (socket says 172.21.15.31:52950) 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff47539c700 1 -- 172.21.15.31:0/2220773968 learned_addr learned my addr 172.21.15.31:0/2220773968 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.354+0000 7ff46ffff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff470199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 -- 172.21.15.31:0/2220773968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 msgr2=0x7ff47019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 0x7ff47019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 -- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 msgr2=0x7ff470199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff470199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 -- 172.21.15.31:0/2220773968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580095f0 con 0x7ff4700fffd0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff46ffff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff470199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.355+0000 7ff474b9b700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff470199140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4580036c0 tx=0x7ff458005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.356+0000 7ff46dffb700 1 -- 172.21.15.31:0/2220773968 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45801d070 con 0x7ff4700fffd0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.356+0000 7ff46dffb700 1 -- 172.21.15.31:0/2220773968 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff458003ec0 con 0x7ff4700fffd0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.356+0000 7ff46dffb700 1 -- 172.21.15.31:0/2220773968 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458022b20 con 0x7ff4700fffd0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.356+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4701a5fa0 con 0x7ff4700fffd0 2023-10-06T21:45:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.356+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4701a6310 con 0x7ff4700fffd0 2023-10-06T21:45:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.360+0000 7ff46dffb700 1 -- 172.21.15.31:0/2220773968 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff458004030 con 0x7ff4700fffd0 2023-10-06T21:45:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.362+0000 7ff48a68eb80 1 Processor -- start 2023-10-06T21:45:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.362+0000 7ff48a68eb80 1 -- start start 2023-10-06T21:45:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130f9afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130f04400 con 0x55b130f9b520 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130ef5bc0 con 0x55b130fa63d0 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130ef34e0 con 0x55b130f9ac00 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff46dffb700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 0x7ff45c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.363+0000 7ff46dffb700 1 -- 172.21.15.31:0/2220773968 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff4580ce6a0 con 0x7ff4700fffd0 2023-10-06T21:45:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52960/0 (socket says 172.21.15.31:52960) 2023-10-06T21:45:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 -- 172.21.15.31:0/3920305799 learned_addr learned my addr 172.21.15.31:0/3920305799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff47539c700 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130f9afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff47539c700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 0x7ff45c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 -- 172.21.15.31:0/3920305799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 msgr2=0x55b130f9afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130f9afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 -- 172.21.15.31:0/3920305799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 msgr2=0x55b130fa5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.364+0000 7ff46ffff700 1 -- 172.21.15.31:0/3920305799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b130d19d80 con 0x55b130fa63d0 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.365+0000 7ff46ffff700 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff46000bd30 tx=0x7ff46000f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5826cbbe557ac196 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.365+0000 7ff466ffd700 1 -- 172.21.15.31:0/3920305799 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000fe50 con 0x55b130fa63d0 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.365+0000 7ff466ffd700 1 -- 172.21.15.31:0/3920305799 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff46000be80 con 0x55b130fa63d0 2023-10-06T21:45:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.365+0000 7ff466ffd700 1 -- 172.21.15.31:0/3920305799 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46001b770 con 0x55b130fa63d0 2023-10-06T21:45:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3920305799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 msgr2=0x55b130fa87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff46000bd30 tx=0x7ff46000f640 comp rx=0 tx=0).stop 2023-10-06T21:45:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff47539c700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 0x7ff45c0a3ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff4680099a0 tx=0x7ff46800c040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3920305799 shutdown_connections 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa87c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130f9afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3920305799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3920305799 >> 172.21.15.31:0/3920305799 conn(0x55b130f10200 msgr2=0x55b130f97080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3920305799 shutdown_connections 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.366+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3920305799 wait complete. 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.367+0000 7ff48a68eb80 1 -- start start 2023-10-06T21:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.367+0000 7ff48a68eb80 1 Processor -- start 2023-10-06T21:45:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130fa63d0 0x55b130fa3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130eeedd0 con 0x55b130f9b520 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130e8b6b0 con 0x55b130f9ac00 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130e8b350 con 0x55b130fa63d0 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.368+0000 7ff48a68eb80 10 client.0 _ll_register_callbacks cb 0x55b130ce87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.369+0000 7ff46ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.369+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.369+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52974/0 (socket says 172.21.15.31:52974) 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.369+0000 7ff474b9b700 1 -- 172.21.15.31:0/3186215346 learned_addr learned my addr 172.21.15.31:0/3186215346 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.369+0000 7ff47539c700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130fa63d0 0x55b130fa3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 -- 172.21.15.31:0/3186215346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130fa63d0 msgr2=0x55b130fa3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130fa63d0 0x55b130fa3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 -- 172.21.15.31:0/3186215346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 msgr2=0x55b130fa3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 -- 172.21.15.31:0/3186215346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46000b960 con 0x55b130f9ac00 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4580cc380 tx=0x7ff458000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf8af9dadc31897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff44ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:45:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45809aeb0 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.370+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff45809b7c0 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b130ee7db0 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4580671a0 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff48a68eb80 10 client.27800 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b130d02c00 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b130ee7db0 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4580a3630 con 0x55b130f9ac00 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff48a68eb80 10 client.27800 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.371+0000 7ff48a68eb80 10 client.27800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.372+0000 7ff48a68eb80 20 client.27800 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.372+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b130fb5b40 0x55b130fb7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.372+0000 7ff48a68eb80 4 client.27800 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:45:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.372+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b130ee7ad0 con 0x55b130fb5b40 2023-10-06T21:45:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.372+0000 7ff46ffff700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b130fb5b40 0x55b130fb7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.373+0000 7ff46ffff700 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b130fb5b40 0x55b130fb7f60 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa1507110cc6bf1 server_cookie=cb5338cf5c23fb88 in_seq=0 out_seq=0 2023-10-06T21:45:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.373+0000 7ff44ffff700 10 client.27800 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.373+0000 7ff44ffff700 1 -- 172.21.15.31:0/3186215346 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b130ee7ad0 con 0x55b130fb5b40 2023-10-06T21:45:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.373+0000 7ff44ffff700 10 client.27800 handle_command_reply: tid=0 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.373+0000 7ff48a68eb80 1 client.27800 shutdown 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 msgr2=0x55b130fa5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4580cc380 tx=0x7ff458000bc0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 shutdown_connections 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130fa63d0 0x55b130fa3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9ac00 0x55b130fa5e20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b130fb5b40 0x55b130fb7f60 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3186215346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9b520 0x55b130fa3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 >> 172.21.15.31:0/3186215346 conn(0x55b130f10200 msgr2=0x55b130f96c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff44ffff700 0 client.27800 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 shutdown_connections 2023-10-06T21:45:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.374+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3186215346 wait complete. 2023-10-06T21:45:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.375+0000 7ff48a68eb80 20 client.27800 trim_cache size 0 max 16384 2023-10-06T21:45:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 Processor -- start 2023-10-06T21:45:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 -- start start 2023-10-06T21:45:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9b520 0x55b130fb16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130d6f8c0 con 0x55b130f9ac00 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130f9ef00 con 0x55b130fa63d0 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.398+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130f9bb90 con 0x55b130f9b520 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43602/0 (socket says 172.21.15.31:43602) 2023-10-06T21:45:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 -- 172.21.15.31:0/3882332648 learned_addr learned my addr 172.21.15.31:0/3882332648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff47539c700 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 -- 172.21.15.31:0/3882332648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9b520 msgr2=0x55b130fb16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9b520 0x55b130fb16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 -- 172.21.15.31:0/3882332648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 msgr2=0x55b130fa5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff474b9b700 1 -- 172.21.15.31:0/3882332648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b130d401f0 con 0x55b130f9ac00 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.399+0000 7ff47539c700 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.400+0000 7ff474b9b700 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff458097c70 tx=0x7ff4580b4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17373f9dfbc3e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.400+0000 7ff4677fe700 1 -- 172.21.15.31:0/3882332648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580a22e0 con 0x55b130f9ac00 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.400+0000 7ff4677fe700 1 -- 172.21.15.31:0/3882332648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4580671a0 con 0x55b130f9ac00 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.400+0000 7ff4677fe700 1 -- 172.21.15.31:0/3882332648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45809aed0 con 0x55b130f9ac00 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3882332648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 msgr2=0x55b130fb7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff458097c70 tx=0x7ff4580b4120 comp rx=0 tx=0).stop 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3882332648 shutdown_connections 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130fa63d0 0x55b130fa5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130f9ac00 0x55b130fb7b50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/3882332648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9b520 0x55b130fb16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3882332648 >> 172.21.15.31:0/3882332648 conn(0x55b130f10200 msgr2=0x55b130f957b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3882332648 shutdown_connections 2023-10-06T21:45:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.401+0000 7ff48a68eb80 1 -- 172.21.15.31:0/3882332648 wait complete. 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.402+0000 7ff48a68eb80 1 -- start start 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.402+0000 7ff48a68eb80 1 Processor -- start 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.402+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130fa8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 0x55b130fa9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131011150 con 0x55b130fa63d0 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131011290 con 0x55b130f9b520 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131011400 con 0x55b130f9ac00 2023-10-06T21:45:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff48a68eb80 10 client.0 _ll_register_callbacks cb 0x7ff45809b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff47539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130fa8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff474b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130fa8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:46082/0 (socket says 172.21.15.31:46082) 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff474b9b700 1 -- 172.21.15.31:0/2348317511 learned_addr learned my addr 172.21.15.31:0/2348317511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff47539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43614/0 (socket says 172.21.15.31:43614) 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.403+0000 7ff46ffff700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 0x55b130fa9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 -- 172.21.15.31:0/2348317511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 msgr2=0x55b130fa8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130fa8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 -- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 msgr2=0x55b130fa9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 0x55b130fa9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 -- 172.21.15.31:0/2348317511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b130d02c00 con 0x55b130fa63d0 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff46ffff700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 0x55b130fa9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff47539c700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff468009550 tx=0x7ff468004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a56fd92011ca56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff44ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.404+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468021070 con 0x55b130fa63d0 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff468014e40 con 0x55b130fa63d0 2023-10-06T21:45:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b130f8ed50 con 0x55b130fa63d0 2023-10-06T21:45:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff46800f070 con 0x55b130fa63d0 2023-10-06T21:45:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff48a68eb80 10 client.18078 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b130d18250 con 0x55b130fa63d0 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.405+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b130f8ed50 con 0x55b130fa63d0 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff468004020 con 0x55b130fa63d0 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 10 client.18078 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 20 client.18078 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b131012b30 0x55b131014f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 4 client.18078 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b130d3af10 con 0x55b131012b30 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.406+0000 7ff474b9b700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b131012b30 0x55b131014f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.407+0000 7ff474b9b700 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b131012b30 0x55b131014f50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c183055ffbb5fec server_cookie=ed43a045b093aa19 in_seq=0 out_seq=0 2023-10-06T21:45:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.407+0000 7ff44ffff700 10 client.18078 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.485+0000 7ff44ffff700 1 -- 172.21.15.31:0/2348317511 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b130d3af10 con 0x55b131012b30 2023-10-06T21:45:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.485+0000 7ff44ffff700 10 client.18078 handle_command_reply: tid=0 2023-10-06T21:45:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.485+0000 7ff48a68eb80 1 client.18078 shutdown 2023-10-06T21:45:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 msgr2=0x55b130fa99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff468009550 tx=0x7ff468004b40 comp rx=0 tx=0).stop 2023-10-06T21:45:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 shutdown_connections 2023-10-06T21:45:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55b131012b30 0x55b131014f50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b130fa63d0 0x55b130fa99a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b130f9ac00 0x55b130fa8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff44ffff700 0 client.18078 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 --2- 172.21.15.31:0/2348317511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b130f9b520 0x55b130fa9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 >> 172.21.15.31:0/2348317511 conn(0x55b130f10200 msgr2=0x55b130fb16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 shutdown_connections 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.486+0000 7ff48a68eb80 1 -- 172.21.15.31:0/2348317511 wait complete. 2023-10-06T21:45:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.487+0000 7ff48a68eb80 20 client.18078 trim_cache size 0 max 16384 2023-10-06T21:45:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:45:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:45:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:45:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 msgr2=0x7ff45c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 0x7ff45c0a3ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff4680099a0 tx=0x7ff46800c040 comp rx=0 tx=0).stop 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 msgr2=0x7ff470199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff470199140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4580036c0 tx=0x7ff458005720 comp rx=0 tx=0).stop 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 shutdown_connections 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff47019dcf0 0x7ff47019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4700fffd0 0x7ff470199140 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff45c0a15f0 0x7ff45c0a3ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 --2- 172.21.15.31:0/2220773968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4701050f0 0x7ff470199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.488+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 >> 172.21.15.31:0/2220773968 conn(0x7ff470076ea0 msgr2=0x7ff470069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.489+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 shutdown_connections 2023-10-06T21:45:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:44.489+0000 7ff476dff700 1 -- 172.21.15.31:0/2220773968 wait complete. 2023-10-06T21:45:44.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81783 6.30 MB/sec execute 573 sec latency 94.228 ms 2023-10-06T21:45:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:45 smithi142 ceph-mon[102951]: pgmap v786: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2023-10-06T21:45:45.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:45 smithi031 ceph-mon[98539]: pgmap v786: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2023-10-06T21:45:45.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:45 smithi146 ceph-mon[102809]: pgmap v786: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 290 op/s 2023-10-06T21:45:45.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 81918 6.30 MB/sec execute 574 sec latency 77.572 ms 2023-10-06T21:45:46.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82031 6.30 MB/sec execute 575 sec latency 244.058 ms 2023-10-06T21:45:47.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:47 smithi142 ceph-mon[102951]: pgmap v787: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:47 smithi031 ceph-mon[98539]: pgmap v787: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:47.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:47 smithi146 ceph-mon[102809]: pgmap v787: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:47.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82163 6.30 MB/sec execute 576 sec latency 280.862 ms 2023-10-06T21:45:48.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82283 6.30 MB/sec execute 577 sec latency 89.317 ms 2023-10-06T21:45:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:49 smithi142 ceph-mon[102951]: pgmap v788: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:49 smithi031 ceph-mon[98539]: pgmap v788: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:49.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:49 smithi146 ceph-mon[102809]: pgmap v788: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 187 op/s 2023-10-06T21:45:49.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82388 6.30 MB/sec execute 578 sec latency 64.905 ms 2023-10-06T21:45:50.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82514 6.31 MB/sec execute 579 sec latency 73.092 ms 2023-10-06T21:45:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:51 smithi142 ceph-mon[102951]: pgmap v789: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 281 op/s 2023-10-06T21:45:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:51 smithi031 ceph-mon[98539]: pgmap v789: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 281 op/s 2023-10-06T21:45:51.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:51 smithi146 ceph-mon[102809]: pgmap v789: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 281 op/s 2023-10-06T21:45:51.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82702 6.33 MB/sec execute 580 sec latency 76.368 ms 2023-10-06T21:45:52.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82820 6.33 MB/sec execute 581 sec latency 68.678 ms 2023-10-06T21:45:53.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:53 smithi142 ceph-mon[102951]: pgmap v790: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2023-10-06T21:45:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:53 smithi031 ceph-mon[98539]: pgmap v790: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2023-10-06T21:45:53.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:53 smithi146 ceph-mon[102809]: pgmap v790: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2023-10-06T21:45:53.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 82944 6.33 MB/sec execute 582 sec latency 83.421 ms 2023-10-06T21:45:54.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83018 6.32 MB/sec execute 583 sec latency 226.229 ms 2023-10-06T21:45:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:55 smithi142 ceph-mon[102951]: pgmap v791: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:45:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:55 smithi031 ceph-mon[98539]: pgmap v791: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:45:55.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:55 smithi146 ceph-mon[102809]: pgmap v791: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2023-10-06T21:45:55.800 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83126 6.33 MB/sec execute 584 sec latency 336.181 ms 2023-10-06T21:45:56.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83211 6.32 MB/sec execute 585 sec latency 122.154 ms 2023-10-06T21:45:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:57 smithi142 ceph-mon[102951]: pgmap v792: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 -- 172.21.15.31:0/1748071017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15981038b0 msgr2=0x7f1598103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 --2- 172.21.15.31:0/1748071017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15981038b0 0x7f1598103d30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1590009f90 tx=0x7f159000b330 comp rx=0 tx=0).stop 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 -- 172.21.15.31:0/1748071017 shutdown_connections 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 --2- 172.21.15.31:0/1748071017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15981038b0 0x7f1598103d30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 --2- 172.21.15.31:0/1748071017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1598105e20 0x7f15980682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 --2- 172.21.15.31:0/1748071017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598068810 0x7f1598068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.716+0000 7f15a07ff700 1 -- 172.21.15.31:0/1748071017 >> 172.21.15.31:0/1748071017 conn(0x7f1598076b70 msgr2=0x7f1598076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.718+0000 7f15a07ff700 1 -- 172.21.15.31:0/1748071017 shutdown_connections 2023-10-06T21:45:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.718+0000 7f15a07ff700 1 -- 172.21.15.31:0/1748071017 wait complete. 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 Processor -- start 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 -- start start 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15981038b0 0x7f15981147b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 0x7f159810d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159810d950 con 0x7f1598105e20 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159810dac0 con 0x7f15981038b0 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.720+0000 7f15a07ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159810dc80 con 0x7f1598068810 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159e59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159e59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48562/0 (socket says 172.21.15.31:48562) 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159e59b700 1 -- 172.21.15.31:0/710318175 learned_addr learned my addr 172.21.15.31:0/710318175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 0x7f159810d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159dd9a700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15981038b0 0x7f15981147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 -- 172.21.15.31:0/710318175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 msgr2=0x7f1598114230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 -- 172.21.15.31:0/710318175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15981038b0 msgr2=0x7f15981147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15981038b0 0x7f15981147b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159ed9c700 1 -- 172.21.15.31:0/710318175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590009c40 con 0x7f1598105e20 2023-10-06T21:45:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.721+0000 7f159e59b700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 0x7f159810d180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1590009790 tx=0x7f1590007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f158f7fe700 1 -- 172.21.15.31:0/710318175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159000e040 con 0x7f1598105e20 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1598111dc0 con 0x7f1598105e20 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f158f7fe700 1 -- 172.21.15.31:0/710318175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f159001d480 con 0x7f1598105e20 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f158f7fe700 1 -- 172.21.15.31:0/710318175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159001b6b0 con 0x7f1598105e20 2023-10-06T21:45:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.722+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981122d0 con 0x7f1598105e20 2023-10-06T21:45:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.728+0000 7f158f7fe700 1 -- 172.21.15.31:0/710318175 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f159001d5f0 con 0x7f1598105e20 2023-10-06T21:45:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.729+0000 7f15b40edb80 1 Processor -- start 2023-10-06T21:45:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.729+0000 7f15b40edb80 1 -- start start 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2682dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d26797400 con 0x559d2682dc00 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d26788bc0 con 0x559d2682e520 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f15b40edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d267864e0 con 0x559d268393d0 2023-10-06T21:45:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f158f7fe700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 0x7f15840a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f158f7fe700 1 -- 172.21.15.31:0/710318175 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f15900ce700 con 0x7f1598105e20 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.730+0000 7f159e59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52544/0 (socket says 172.21.15.31:52544) 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 -- 172.21.15.31:0/256683210 learned_addr learned my addr 172.21.15.31:0/256683210 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159dd9a700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159dd9a700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2682dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 0x7f15840a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 -- 172.21.15.31:0/256683210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 msgr2=0x559d2683b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 -- 172.21.15.31:0/256683210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 msgr2=0x559d2682dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2682dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159e59b700 1 -- 172.21.15.31:0/256683210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d265acd80 con 0x559d2682e520 2023-10-06T21:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.731+0000 7f159dd9a700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2682dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:45:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.732+0000 7f159e59b700 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1594011650 tx=0x7f1594011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34802cf9441165c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.732+0000 7f159ed9c700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 0x7f15840a3b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15900cbde0 tx=0x7f1590009040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.732+0000 7f158cff9700 1 -- 172.21.15.31:0/256683210 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594019630 con 0x559d2682e520 2023-10-06T21:45:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.732+0000 7f158cff9700 1 -- 172.21.15.31:0/256683210 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1594019c20 con 0x559d2682e520 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.732+0000 7f158cff9700 1 -- 172.21.15.31:0/256683210 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159400eb90 con 0x559d2682e520 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 -- 172.21.15.31:0/256683210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 msgr2=0x559d26838e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1594011650 tx=0x7f1594011960 comp rx=0 tx=0).stop 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 -- 172.21.15.31:0/256683210 shutdown_connections 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2682dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 --2- 172.21.15.31:0/256683210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d26838e90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 -- 172.21.15.31:0/256683210 >> 172.21.15.31:0/256683210 conn(0x559d267a3200 msgr2=0x559d2682a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 -- 172.21.15.31:0/256683210 shutdown_connections 2023-10-06T21:45:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.733+0000 7f15b40edb80 1 -- 172.21.15.31:0/256683210 wait complete. 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.734+0000 7f15b40edb80 1 -- start start 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.734+0000 7f15b40edb80 1 Processor -- start 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682dc00 0x559d26838df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d268393d0 0x559d26836780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d26781dd0 con 0x559d268393d0 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2671e6b0 con 0x559d2682e520 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.735+0000 7f15b40edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2671e350 con 0x559d2682dc00 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f15b40edb80 10 client.0 _ll_register_callbacks cb 0x559d2657b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:52552/0 (socket says 172.21.15.31:52552) 2023-10-06T21:45:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 -- 172.21.15.31:0/654699304 learned_addr learned my addr 172.21.15.31:0/654699304 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159dd9a700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682dc00 0x559d26838df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159dd9a700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d268393d0 0x559d26836780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 -- 172.21.15.31:0/654699304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682dc00 msgr2=0x559d26838df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682dc00 0x559d26838df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 -- 172.21.15.31:0/654699304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d268393d0 msgr2=0x559d26836780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d268393d0 0x559d26836780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.736+0000 7f159e59b700 1 -- 172.21.15.31:0/654699304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1594009e10 con 0x559d2682e520 2023-10-06T21:45:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.737+0000 7f159e59b700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f159401c040 tx=0x7f1594004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df79fea60696455a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.737+0000 7f1581ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.737+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594005d60 con 0x559d2682e520 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.737+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f159400ec20 con 0x559d2682e520 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d26821d50 con 0x559d2682e520 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f159400ede0 con 0x559d2682e520 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f15b40edb80 10 client.27815 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559d26595c00 con 0x559d2682e520 2023-10-06T21:45:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d26821d50 con 0x559d2682e520 2023-10-06T21:45:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.738+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1594003960 con 0x559d2682e520 2023-10-06T21:45:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 10 client.27815 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 10 client.27815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 20 client.27815 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d26848e10 0x559d2684b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 4 client.27815 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d2677adb0 con 0x559d26848e10 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.739+0000 7f159dd9a700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d26848e10 0x559d2684b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.740+0000 7f159dd9a700 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d26848e10 0x559d2684b230 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5c3f4a9d0f10fe server_cookie=6cb018468754cf9c in_seq=0 out_seq=0 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.740+0000 7f1581ffb700 10 client.27815 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.740+0000 7f1581ffb700 1 -- 172.21.15.31:0/654699304 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d2677adb0 con 0x559d26848e10 2023-10-06T21:45:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.740+0000 7f1581ffb700 10 client.27815 handle_command_reply: tid=0 2023-10-06T21:45:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 client.27815 shutdown 2023-10-06T21:45:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 msgr2=0x559d268360e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f159401c040 tx=0x7f1594004680 comp rx=0 tx=0).stop 2023-10-06T21:45:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 shutdown_connections 2023-10-06T21:45:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d26848e10 0x559d2684b230 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d268393d0 0x559d26836780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682dc00 0x559d26838df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 --2- 172.21.15.31:0/654699304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268360e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 >> 172.21.15.31:0/654699304 conn(0x559d267a3200 msgr2=0x559d26829c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f1581ffb700 0 client.27815 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 shutdown_connections 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.741+0000 7f15b40edb80 1 -- 172.21.15.31:0/654699304 wait complete. 2023-10-06T21:45:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.742+0000 7f15b40edb80 20 client.27815 trim_cache size 0 max 16384 2023-10-06T21:45:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:57 smithi031 ceph-mon[98539]: pgmap v792: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 Processor -- start 2023-10-06T21:45:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 -- start start 2023-10-06T21:45:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2683ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 0x559d268313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d266028c0 con 0x559d2682dc00 2023-10-06T21:45:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d26826890 con 0x559d268393d0 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.766+0000 7f15b40edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2682ae40 con 0x559d2682e520 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:48588/0 (socket says 172.21.15.31:48588) 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 -- 172.21.15.31:0/586194402 learned_addr learned my addr 172.21.15.31:0/586194402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159dd9a700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2683ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159dd9a700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 0x559d268313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 -- 172.21.15.31:0/586194402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 msgr2=0x559d268313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 0x559d268313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 -- 172.21.15.31:0/586194402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 msgr2=0x559d2683ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2683ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 -- 172.21.15.31:0/586194402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d265d31f0 con 0x559d2682e520 2023-10-06T21:45:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159dd9a700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 0x559d268313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.767+0000 7f159e59b700 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1594005850 tx=0x7f159400e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e0e7884182356 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.768+0000 7f158d7fa700 1 -- 172.21.15.31:0/586194402 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594011cb0 con 0x559d2682e520 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.768+0000 7f158d7fa700 1 -- 172.21.15.31:0/586194402 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f159400eeb0 con 0x559d2682e520 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.768+0000 7f158d7fa700 1 -- 172.21.15.31:0/586194402 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594019430 con 0x559d2682e520 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.768+0000 7f15b40edb80 1 -- 172.21.15.31:0/586194402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 msgr2=0x559d2684ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.768+0000 7f15b40edb80 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1594005850 tx=0x7f159400e3b0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 -- 172.21.15.31:0/586194402 shutdown_connections 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d268393d0 0x559d268313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d2683ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 --2- 172.21.15.31:0/586194402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d2682e520 0x559d2684ae20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 -- 172.21.15.31:0/586194402 >> 172.21.15.31:0/586194402 conn(0x559d267a3200 msgr2=0x559d26844670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 -- 172.21.15.31:0/586194402 shutdown_connections 2023-10-06T21:45:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 -- 172.21.15.31:0/586194402 wait complete. 2023-10-06T21:45:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.769+0000 7f15b40edb80 1 -- start start 2023-10-06T21:45:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 Processor -- start 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d268953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2683f5b0 con 0x559d2682dc00 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2683f720 con 0x559d2682e520 2023-10-06T21:45:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.770+0000 7f15b40edb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2683f8e0 con 0x559d268393d0 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f15b40edb80 10 client.0 _ll_register_callbacks cb 0x7f1588009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f159dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d268953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f159dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d268953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47724/0 (socket says 172.21.15.31:47724) 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f159dd9a700 1 -- 172.21.15.31:0/507943256 learned_addr learned my addr 172.21.15.31:0/507943256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f159dd9a700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.771+0000 7f159e59b700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 -- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 msgr2=0x559d268958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 -- 172.21.15.31:0/507943256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 msgr2=0x559d268953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d268953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 -- 172.21.15.31:0/507943256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d26595c00 con 0x559d268393d0 2023-10-06T21:45:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159e59b700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:45:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f159dd9a700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683efd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f159810ea30 tx=0x7f1594004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aea418687411e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:45:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f1581ffb700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004a30 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.772+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1588004b90 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.773+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1594019c20 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.773+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1588004d50 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.773+0000 7f15b40edb80 10 client.27760 fetch_fsmap learned FSMap version 51 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.773+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559d265ab250 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.773+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1594019c20 con 0x559d268393d0 2023-10-06T21:45:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f15880050d0 con 0x559d268393d0 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 10 client.27760 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 10 client.27760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 20 client.27760 populate_metadata read hostname 'smithi031' 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d2689d1e0 0x559d2689f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 4 client.27760 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.774+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d2677aad0 con 0x559d2689d1e0 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.775+0000 7f159e59b700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d2689d1e0 0x559d2689f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.776+0000 7f159e59b700 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d2689d1e0 0x559d2689f600 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca368b045cea17d server_cookie=bfe4aa9a7b69322f in_seq=0 out_seq=0 2023-10-06T21:45:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.776+0000 7f1581ffb700 10 client.27760 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:57.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83330 6.33 MB/sec execute 586 sec latency 87.842 ms 2023-10-06T21:45:57.802 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:57 smithi146 ceph-mon[102809]: pgmap v792: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f1581ffb700 1 -- 172.21.15.31:0/507943256 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d2677aad0 con 0x559d2689d1e0 2023-10-06T21:45:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f1581ffb700 10 client.27760 handle_command_reply: tid=0 2023-10-06T21:45:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 client.27760 shutdown 2023-10-06T21:45:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 msgr2=0x559d2683efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683efd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f159810ea30 tx=0x7f1594004350 comp rx=0 tx=0).stop 2023-10-06T21:45:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 shutdown_connections 2023-10-06T21:45:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d268393d0 0x559d2683efd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x559d2689d1e0 0x559d2689f600 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559d2682dc00 0x559d268953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 --2- 172.21.15.31:0/507943256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2682e520 0x559d268958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f1581ffb700 0 client.27760 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.843+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 >> 172.21.15.31:0/507943256 conn(0x559d267a3200 msgr2=0x559d26845110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.844+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 shutdown_connections 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.844+0000 7f15b40edb80 1 -- 172.21.15.31:0/507943256 wait complete. 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.844+0000 7f15b40edb80 20 client.27760 trim_cache size 0 max 16384 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:45:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:45:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:45:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:45:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 msgr2=0x7f15840a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 0x7f15840a3b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15900cbde0 tx=0x7f1590009040 comp rx=0 tx=0).stop 2023-10-06T21:45:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 msgr2=0x7f159810d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 0x7f159810d180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1590009790 tx=0x7f1590007de0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 shutdown_connections 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f15840a1670 0x7f15840a3b30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1598105e20 0x7f159810d180 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598068810 0x7f1598114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 --2- 172.21.15.31:0/710318175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15981038b0 0x7f15981147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 >> 172.21.15.31:0/710318175 conn(0x7f1598076b70 msgr2=0x7f15980fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.845+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 shutdown_connections 2023-10-06T21:45:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:45:57.846+0000 7f15a07ff700 1 -- 172.21.15.31:0/710318175 wait complete. 2023-10-06T21:45:58.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83437 6.33 MB/sec execute 587 sec latency 87.210 ms 2023-10-06T21:45:59.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:45:59 smithi142 ceph-mon[102951]: pgmap v793: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:59.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:45:59 smithi031 ceph-mon[98539]: pgmap v793: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:59.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:45:59 smithi146 ceph-mon[102809]: pgmap v793: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 179 op/s 2023-10-06T21:45:59.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83542 6.32 MB/sec execute 588 sec latency 129.102 ms 2023-10-06T21:46:00.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83619 6.31 MB/sec execute 589 sec latency 316.252 ms 2023-10-06T21:46:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:01 smithi142 ceph-mon[102951]: pgmap v794: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 265 op/s 2023-10-06T21:46:01.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:01 smithi031 ceph-mon[98539]: pgmap v794: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 265 op/s 2023-10-06T21:46:01.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:01 smithi146 ceph-mon[102809]: pgmap v794: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 265 op/s 2023-10-06T21:46:01.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83708 6.31 MB/sec execute 590 sec latency 690.213 ms 2023-10-06T21:46:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:02 smithi142 ceph-mon[102951]: pgmap v795: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 171 op/s 2023-10-06T21:46:02.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:02 smithi031 ceph-mon[98539]: pgmap v795: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 171 op/s 2023-10-06T21:46:02.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:02 smithi146 ceph-mon[102809]: pgmap v795: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 171 op/s 2023-10-06T21:46:02.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83807 6.30 MB/sec execute 591 sec latency 1690.263 ms 2023-10-06T21:46:03.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 83903 6.30 MB/sec execute 592 sec latency 2690.321 ms 2023-10-06T21:46:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:04 smithi142 ceph-mon[102951]: pgmap v796: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 264 op/s 2023-10-06T21:46:04.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:04 smithi142 ceph-mon[102951]: osdmap e131: 12 total, 12 up, 12 in 2023-10-06T21:46:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:04 smithi031 ceph-mon[98539]: pgmap v796: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 264 op/s 2023-10-06T21:46:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:04 smithi031 ceph-mon[98539]: osdmap e131: 12 total, 12 up, 12 in 2023-10-06T21:46:04.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:04 smithi146 ceph-mon[102809]: pgmap v796: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 264 op/s 2023-10-06T21:46:04.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:04 smithi146 ceph-mon[102809]: osdmap e131: 12 total, 12 up, 12 in 2023-10-06T21:46:04.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84030 6.30 MB/sec execute 593 sec latency 3690.372 ms 2023-10-06T21:46:05.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84161 6.30 MB/sec execute 594 sec latency 4690.441 ms 2023-10-06T21:46:06.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:06 smithi142 ceph-mon[102951]: pgmap v798: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:06.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:06 smithi031 ceph-mon[98539]: pgmap v798: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:06.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:06 smithi146 ceph-mon[102809]: pgmap v798: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:06.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84262 6.30 MB/sec execute 595 sec latency 5690.490 ms 2023-10-06T21:46:07.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84366 6.29 MB/sec execute 596 sec latency 6690.535 ms 2023-10-06T21:46:08.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:08 smithi031 ceph-mon[98539]: pgmap v799: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:08.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:08 smithi146 ceph-mon[102809]: pgmap v799: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:08.801 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84522 6.31 MB/sec execute 597 sec latency 7690.582 ms 2023-10-06T21:46:08.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:08 smithi142 ceph-mon[102951]: pgmap v799: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 214 op/s 2023-10-06T21:46:09.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84605 6.30 MB/sec execute 598 sec latency 8690.640 ms 2023-10-06T21:46:10.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 84750 6.31 MB/sec execute 599 sec latency 9690.707 ms 2023-10-06T21:46:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.074+0000 7f70bcf9d700 1 -- 172.21.15.31:0/2661913023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b8075d00 msgr2=0x7f70b8111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.074+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/2661913023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b8075d00 0x7f70b8111610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009990 tx=0x7f70ac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 -- 172.21.15.31:0/2661913023 shutdown_connections 2023-10-06T21:46:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/2661913023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 0x7f70b8113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/2661913023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b80753e0 0x7f70b80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/2661913023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b8075d00 0x7f70b8111610 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 -- 172.21.15.31:0/2661913023 >> 172.21.15.31:0/2661913023 conn(0x7f70b80fe9c0 msgr2=0x7f70b8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 -- 172.21.15.31:0/2661913023 shutdown_connections 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.075+0000 7f70bcf9d700 1 -- 172.21.15.31:0/2661913023 wait complete. 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.076+0000 7f70bcf9d700 1 Processor -- start 2023-10-06T21:46:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 -- start start 2023-10-06T21:46:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b80753e0 0x7f70b819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 0x7f70b81a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b81a2350 con 0x7f70b8111b50 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b81a24c0 con 0x7f70b8075d00 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70bcf9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b81a2680 con 0x7f70b80753e0 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.077+0000 7f70b6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b80753e0 0x7f70b819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.31:45780/0 (socket says 172.21.15.31:45780) 2023-10-06T21:46:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 -- 172.21.15.31:0/1762215180 learned_addr learned my addr 172.21.15.31:0/1762215180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b7fff700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 0x7f70b81a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 -- 172.21.15.31:0/1762215180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b80753e0 msgr2=0x7f70b819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b80753e0 0x7f70b819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 -- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 msgr2=0x7f70b81a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 0x7f70b81a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 -- 172.21.15.31:0/1762215180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac0095f0 con 0x7f70b8075d00 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.078+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005740 tx=0x7f70ac005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.079+0000 7f70b4ff9700 1 -- 172.21.15.31:0/1762215180 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac01d070 con 0x7f70b8075d00 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.079+0000 7f70b4ff9700 1 -- 172.21.15.31:0/1762215180 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70ac022470 con 0x7f70b8075d00 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.079+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b81a2900 con 0x7f70b8075d00 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.079+0000 7f70b4ff9700 1 -- 172.21.15.31:0/1762215180 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac017650 con 0x7f70b8075d00 2023-10-06T21:46:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.079+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b81aa1a0 con 0x7f70b8075d00 2023-10-06T21:46:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.084+0000 7f70b4ff9700 1 -- 172.21.15.31:0/1762215180 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f70ac00fb00 con 0x7f70b8075d00 2023-10-06T21:46:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.086+0000 7f70d1a64b80 1 Processor -- start 2023-10-06T21:46:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.086+0000 7f70d1a64b80 1 -- start start 2023-10-06T21:46:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.086+0000 7f70b4ff9700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 0x7f70a00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b4ff9700 1 -- 172.21.15.31:0/1762215180 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f70ac0ce7a0 con 0x7f70b8075d00 2023-10-06T21:46:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d297531140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d29753bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d29753c530 0x55d29753e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29749a400 con 0x55d29753c530 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29748bbc0 con 0x55d297530d60 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2974894e0 con 0x55d297531680 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b7fff700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 0x7f70a00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d29753bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d29753bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57954/0 (socket says 172.21.15.31:57954) 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b6ffd700 1 -- 172.21.15.31:0/3546100526 learned_addr learned my addr 172.21.15.31:0/3546100526 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b7fff700 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d29753c530 0x55d29753e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.087+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d297531140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 -- 172.21.15.31:0/3546100526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 msgr2=0x55d29753bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d29753bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 -- 172.21.15.31:0/3546100526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d29753c530 msgr2=0x55d29753e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d29753c530 0x55d29753e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 -- 172.21.15.31:0/3546100526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2972afd80 con 0x55d297530d60 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d297531140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007bf0 tx=0x7f70a8007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f27cedb385c35b41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.088+0000 7f70b7fff700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 0x7f70a00a3b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70b0015da0 tx=0x7f70b0015420 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.089+0000 7f709dffb700 1 -- 172.21.15.31:0/3546100526 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8010c40 con 0x55d297530d60 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.089+0000 7f709dffb700 1 -- 172.21.15.31:0/3546100526 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70a80044c0 con 0x55d297530d60 2023-10-06T21:46:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.089+0000 7f709dffb700 1 -- 172.21.15.31:0/3546100526 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8005020 con 0x55d297530d60 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3546100526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 msgr2=0x55d297531140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d297531140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007bf0 tx=0x7f70a8007f00 comp rx=0 tx=0).stop 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3546100526 shutdown_connections 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d29753c530 0x55d29753e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d297531140 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3546100526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d29753bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3546100526 >> 172.21.15.31:0/3546100526 conn(0x55d2974a6200 msgr2=0x55d29752d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.090+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3546100526 shutdown_connections 2023-10-06T21:46:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.091+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3546100526 wait complete. 2023-10-06T21:46:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.091+0000 7f70d1a64b80 1 -- start start 2023-10-06T21:46:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 Processor -- start 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d2975391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d297539920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d297484dd0 con 0x55d297530d60 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2974216b0 con 0x55d29753c530 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.092+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d297421350 con 0x55d297531680 2023-10-06T21:46:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70d1a64b80 10 client.0 _ll_register_callbacks cb 0x55d29727e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:46:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49306/0 (socket says 172.21.15.31:49306) 2023-10-06T21:46:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b77fe700 1 -- 172.21.15.31:0/3773828164 learned_addr learned my addr 172.21.15.31:0/3773828164 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b7fff700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d297539920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b7fff700 1 -- 172.21.15.31:0/3773828164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 msgr2=0x55d2975391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.093+0000 7f70b7fff700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d2975391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70b7fff700 1 -- 172.21.15.31:0/3773828164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 msgr2=0x55d29753bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70b7fff700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70b7fff700 1 -- 172.21.15.31:0/3773828164 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a80078a0 con 0x55d29753c530 2023-10-06T21:46:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70b7fff700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d297539920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70b0019a40 tx=0x7f70b000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c1b6fe74a3dc88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f7096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000e040 con 0x55d29753c530 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f70b0020030 con 0x55d29753c530 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.094+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d297524eb0 con 0x55d29753c530 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b0015ee0 con 0x55d29753c530 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f70d1a64b80 10 client.27830 fetch_fsmap learned FSMap version 51 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d297298c00 con 0x55d29753c530 2023-10-06T21:46:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d297524eb0 con 0x55d29753c530 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70b002e070 con 0x55d29753c530 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f70d1a64b80 10 client.27830 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.095+0000 7f70d1a64b80 10 client.27830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.096+0000 7f70d1a64b80 20 client.27830 populate_metadata read hostname 'smithi031' 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.096+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d29754c070 0x55d29754e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.096+0000 7f70d1a64b80 4 client.27830 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:46:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.096+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d29747ddb0 con 0x55d29754c070 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.096+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d29754c070 0x55d29754e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.097+0000 7f70b77fe700 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d29754c070 0x55d29754e4b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f984b2530912e9 server_cookie=43c37a5c9e200058 in_seq=0 out_seq=0 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.097+0000 7f7096ffd700 10 client.27830 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.097+0000 7f7096ffd700 1 -- 172.21.15.31:0/3773828164 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d29747ddb0 con 0x55d29754c070 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.097+0000 7f7096ffd700 10 client.27830 handle_command_reply: tid=0 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.097+0000 7f70d1a64b80 1 client.27830 shutdown 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 msgr2=0x55d297539920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d297539920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70b0019a40 tx=0x7f70b000b040 comp rx=0 tx=0).stop 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 shutdown_connections 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d297539920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d29754c070 0x55d29754e4b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297530d60 0x55d29753bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/3773828164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297531680 0x55d2975391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 >> 172.21.15.31:0/3773828164 conn(0x55d2974a6200 msgr2=0x55d29752cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f7096ffd700 0 client.27830 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 shutdown_connections 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.098+0000 7f70d1a64b80 1 -- 172.21.15.31:0/3773828164 wait complete. 2023-10-06T21:46:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.099+0000 7f70d1a64b80 20 client.27830 trim_cache size 0 max 16384 2023-10-06T21:46:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.125+0000 7f70d1a64b80 1 Processor -- start 2023-10-06T21:46:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 -- start start 2023-10-06T21:46:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d2975430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2973058c0 con 0x55d297531680 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2975436d0 con 0x55d297530d60 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2975377c0 con 0x55d29753c530 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.126+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57980/0 (socket says 172.21.15.31:57980) 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 -- 172.21.15.31:0/454960462 learned_addr learned my addr 172.21.15.31:0/454960462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b77fe700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d2975430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 -- 172.21.15.31:0/454960462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 msgr2=0x55d2975430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d2975430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 -- 172.21.15.31:0/454960462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 msgr2=0x55d29753bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 -- 172.21.15.31:0/454960462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2972d61f0 con 0x55d29753c530 2023-10-06T21:46:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:46:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.127+0000 7f70b7fff700 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f70b0015a20 tx=0x7f70b0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526d94fde6ec66a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.128+0000 7f709e7fc700 1 -- 172.21.15.31:0/454960462 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0015670 con 0x55d29753c530 2023-10-06T21:46:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.128+0000 7f709e7fc700 1 -- 172.21.15.31:0/454960462 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70b0019eb0 con 0x55d29753c530 2023-10-06T21:46:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.128+0000 7f709e7fc700 1 -- 172.21.15.31:0/454960462 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0026460 con 0x55d29753c530 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 -- 172.21.15.31:0/454960462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 msgr2=0x55d2975344f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f70b0015a20 tx=0x7f70b0009920 comp rx=0 tx=0).stop 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 -- 172.21.15.31:0/454960462 shutdown_connections 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d29753c530 0x55d2975344f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d297530d60 0x55d2975430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/454960462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 -- 172.21.15.31:0/454960462 >> 172.21.15.31:0/454960462 conn(0x55d2974a6200 msgr2=0x55d29754e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 -- 172.21.15.31:0/454960462 shutdown_connections 2023-10-06T21:46:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.129+0000 7f70d1a64b80 1 -- 172.21.15.31:0/454960462 wait complete. 2023-10-06T21:46:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.130+0000 7f70d1a64b80 1 -- start start 2023-10-06T21:46:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.130+0000 7f70d1a64b80 1 Processor -- start 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d29759eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29753ff30 con 0x55d297531680 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29759f430 con 0x55d29753c530 2023-10-06T21:46:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29759f5f0 con 0x55d297530d60 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70d1a64b80 10 client.0 _ll_register_callbacks cb 0x7f70b0009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70b77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70b77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:57992/0 (socket says 172.21.15.31:57992) 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70b77fe700 1 -- 172.21.15.31:0/590344919 learned_addr learned my addr 172.21.15.31:0/590344919 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.131+0000 7f70b7fff700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d29759eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 -- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 msgr2=0x55d29759eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d29759eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 -- 172.21.15.31:0/590344919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 msgr2=0x55d29753f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 -- 172.21.15.31:0/590344919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b81aa1a0 con 0x55d297530d60 2023-10-06T21:46:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f70b77fe700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f70a8008010 tx=0x7f70a80059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81541b0f6c549c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.132+0000 7f7096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2023-10-06T21:46:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a801d040 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8681+0+0 (secure 0 0 0) 0x7f70a8027560 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70a80044c0 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a8010e70 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f70d1a64b80 10 client.27775 fetch_fsmap learned FSMap version 51 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.133+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d297298c00 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70a80044c0 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70a80103d0 con 0x55d297530d60 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 10 client.27775 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 10 client.27775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 20 client.27775 populate_metadata read hostname 'smithi031' 2023-10-06T21:46:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d2975a0d20 0x55d2975a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 4 client.27775 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:46:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 --> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d29747dad0 con 0x55d2975a0d20 2023-10-06T21:46:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.134+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d2975a0d20 0x55d2975a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.135+0000 7f70b6ffd700 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d2975a0d20 0x55d2975a3140 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0882c9416608d3 server_cookie=63ab90d91ab89835 in_seq=0 out_seq=0 2023-10-06T21:46:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:11.135+0000 7f7096ffd700 10 client.27775 ms_handle_connect on v2:172.21.15.142:6838/2643221655 2023-10-06T21:46:11.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:11 smithi146 ceph-mon[102809]: pgmap v800: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:11 smithi142 ceph-mon[102951]: pgmap v800: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:11.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:11 smithi031 ceph-mon[98539]: pgmap v800: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:11.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 cleanup 600 sec 2023-10-06T21:46:12.802 INFO:tasks.workunit.client.0.smithi031.stdout: 10 cleanup 601 sec 2023-10-06T21:46:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f7096ffd700 1 -- 172.21.15.31:0/590344919 <== mds.0 v2:172.21.15.142:6838/2643221655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d29747dad0 con 0x55d2975a0d20 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f7096ffd700 10 client.27775 handle_command_reply: tid=0 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 client.27775 shutdown 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 msgr2=0x55d29753f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f70a8008010 tx=0x7f70a80059d0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 shutdown_connections 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d29753c530 0x55d29759eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d297530d60 0x55d29753f2a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.142:6838/2643221655,v1:172.21.15.142:6839/2643221655] conn(0x55d2975a0d20 0x55d2975a3140 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 --2- 172.21.15.31:0/590344919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d297531680 0x55d29753f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 >> 172.21.15.31:0/590344919 conn(0x55d2974a6200 msgr2=0x55d2975344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.285+0000 7f7096ffd700 0 client.27775 ms_handle_reset on v2:172.21.15.142:6838/2643221655 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.286+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 shutdown_connections 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.286+0000 7f70d1a64b80 1 -- 172.21.15.31:0/590344919 wait complete. 2023-10-06T21:46:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.286+0000 7f70d1a64b80 20 client.27775 trim_cache size 0 max 16384 2023-10-06T21:46:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-10-06T21:46:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-10-06T21:46:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-10-06T21:46:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 msgr2=0x7f70a00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 0x7f70a00a3b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70b0015da0 tx=0x7f70b0015420 comp rx=0 tx=0).stop 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 msgr2=0x7f70b819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005740 tx=0x7f70ac005770 comp rx=0 tx=0).stop 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 shutdown_connections 2023-10-06T21:46:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f70a00a1670 0x7f70a00a3b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b8111b50 0x7f70b81a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70b80753e0 0x7f70b819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 --2- 172.21.15.31:0/1762215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70b8075d00 0x7f70b819d8b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 >> 172.21.15.31:0/1762215180 conn(0x7f70b80fe9c0 msgr2=0x7f70b81127c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 shutdown_connections 2023-10-06T21:46:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-10-06T21:46:13.287+0000 7f70bcf9d700 1 -- 172.21.15.31:0/1762215180 wait complete. 2023-10-06T21:46:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:13 smithi146 ceph-mon[102809]: pgmap v801: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:13.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:13 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:46:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:13 smithi142 ceph-mon[102951]: pgmap v801: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:13 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:46:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:13 smithi031 ceph-mon[98539]: pgmap v801: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-06T21:46:13.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:13 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:46:13.802 INFO:tasks.workunit.client.0.smithi031.stdout: 1 cleanup 602 sec 2023-10-06T21:46:14.802 INFO:tasks.workunit.client.0.smithi031.stdout: 1 cleanup 603 sec 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: 0 cleanup 603 sec 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: Operation Count AvgLat MaxLat 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: ---------------------------------------- 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: NTCreateX 121529 15.421 586.911 2023-10-06T21:46:15.124 INFO:tasks.workunit.client.0.smithi031.stdout: Close 89406 0.292 77.639 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Rename 5157 37.306 12170.225 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Unlink 24437 12.328 399.085 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Qpathinfo 110095 9.596 580.875 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Qfileinfo 19218 0.621 11.043 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Qfsinfo 20146 1.550 356.796 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Sfileinfo 9972 4.888 305.789 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: Find 42507 54.472 621.714 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: WriteX 60050 1.078 291.902 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: ReadX 190572 0.016 332.987 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: LockX 394 2.335 9.919 2023-10-06T21:46:15.125 INFO:tasks.workunit.client.0.smithi031.stdout: UnlockX 394 2.247 14.785 2023-10-06T21:46:15.126 INFO:tasks.workunit.client.0.smithi031.stdout: Flush 8545 8.308 381.811 2023-10-06T21:46:15.126 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-10-06T21:46:15.126 INFO:tasks.workunit.client.0.smithi031.stdout:Throughput 6.30805 MB/sec 10 clients 10 procs max_latency=9690.707 ms 2023-10-06T21:46:15.393 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T21:46:15.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T21:46:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:15 smithi146 ceph-mon[102809]: pgmap v802: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:15 smithi146 ceph-mon[102809]: osdmap e132: 12 total, 12 up, 12 in 2023-10-06T21:46:15.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:15 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:15.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:15 smithi031 ceph-mon[98539]: pgmap v802: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:15.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:15 smithi031 ceph-mon[98539]: osdmap e132: 12 total, 12 up, 12 in 2023-10-06T21:46:15.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:15 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:15 smithi142 ceph-mon[102951]: pgmap v802: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:15 smithi142 ceph-mon[102951]: osdmap e132: 12 total, 12 up, 12 in 2023-10-06T21:46:15.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:15 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:16 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:46:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:16 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:46:16.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:16 smithi146 ceph-mon[102809]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:46:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:16 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:46:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:16 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:46:16.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:16 smithi142 ceph-mon[102951]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:46:16.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:16 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:46:16.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:16 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:46:16.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:16 smithi031 ceph-mon[98539]: from='mgr.15570 172.21.15.31:0/358735296' entity='mgr.x' 2023-10-06T21:46:17.378 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-10-06T21:46:17.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-06T21:46:17.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:17 smithi146 ceph-mon[102809]: pgmap v804: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:17.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:17 smithi142 ceph-mon[102951]: pgmap v804: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:17.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:17 smithi031 ceph-mon[98539]: pgmap v804: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:18.204 DEBUG:teuthology.parallel:result is None 2023-10-06T21:46:18.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T21:46:18.290 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T21:46:18.290 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-06T21:46:18.302 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:46:18.313 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-06T21:46:18.323 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-06T21:46:18.323 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T21:46:18.323 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:46:18.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-06T21:46:18.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-06T21:46:18.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:18 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:18.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T21:46:18.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:18 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:18.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:18 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 -- 172.21.15.31:0/2417904340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281061f0 msgr2=0x7ff32810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/2417904340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281061f0 0x7ff32810dd70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b0a0 tx=0x7ff32000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 -- 172.21.15.31:0/2417904340 shutdown_connections 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/2417904340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281061f0 0x7ff32810dd70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/2417904340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3281010d0 0x7ff3281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/2417904340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff328101a80 0x7ff328105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 -- 172.21.15.31:0/2417904340 >> 172.21.15.31:0/2417904340 conn(0x7ff3280fc950 msgr2=0x7ff3280fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 -- 172.21.15.31:0/2417904340 shutdown_connections 2023-10-06T21:46:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.727+0000 7ff32c9ff700 1 -- 172.21.15.31:0/2417904340 wait complete. 2023-10-06T21:46:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.728+0000 7ff32c9ff700 1 Processor -- start 2023-10-06T21:46:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.728+0000 7ff32c9ff700 1 -- start start 2023-10-06T21:46:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3281061f0 0x7ff328199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 0x7ff32819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819e830 con 0x7ff3281010d0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819e9a0 con 0x7ff3281061f0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819eb60 con 0x7ff32819ddb0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34048/0 (socket says 172.21.15.31:34048) 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 -- 172.21.15.31:0/1279155509 learned_addr learned my addr 172.21.15.31:0/1279155509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff31ffff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3281061f0 0x7ff328199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff326d9d700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 0x7ff32819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 -- 172.21.15.31:0/1279155509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 msgr2=0x7ff32819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 0x7ff32819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 -- 172.21.15.31:0/1279155509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3281061f0 msgr2=0x7ff328199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3281061f0 0x7ff328199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff32659c700 1 -- 172.21.15.31:0/1279155509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320009d00 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.729+0000 7ff326d9d700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 0x7ff32819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff32659c700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff31800d980 tx=0x7ff31800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318004220 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff318007640 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180055e0 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3281a60c0 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.730+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3281a65a0 con 0x7ff3281010d0 2023-10-06T21:46:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.731+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3281005a0 con 0x7ff3281010d0 2023-10-06T21:46:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.733+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff318004750 con 0x7ff3281010d0 2023-10-06T21:46:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.734+0000 7ff31f7fe700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 0x7ff3140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.735+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff3180ccf40 con 0x7ff3281010d0 2023-10-06T21:46:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.735+0000 7ff31ffff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 0x7ff3140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.735+0000 7ff31ffff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 0x7ff3140a3b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff32819aa20 tx=0x7ff31000b410 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.739+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff318096190 con 0x7ff3281010d0 2023-10-06T21:46:18.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.982+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff328101a80 con 0x7ff3140a1670 2023-10-06T21:46:18.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.993+0000 7ff31f7fe700 1 -- 172.21.15.31:0/1279155509 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff328101a80 con 0x7ff3140a1670 2023-10-06T21:46:18.996 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T21:20:57", "first": "2023-10-06T21:21:00", "last": "2023-10-06T21:46:00", "last_pruned": "2023-10-06T21:46:01", "created_count": 26, "pruned_count": 20, "active": true} 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 msgr2=0x7ff3140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 0x7ff3140a3b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff32819aa20 tx=0x7ff31000b410 comp rx=0 tx=0).stop 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 msgr2=0x7ff328199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff31800d980 tx=0x7ff31800dc90 comp rx=0 tx=0).stop 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 shutdown_connections 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff32819ddb0 0x7ff32819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:18.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ff3140a1670 0x7ff3140a3b30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3281010d0 0x7ff328199200 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 --2- 172.21.15.31:0/1279155509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3281061f0 0x7ff328199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 >> 172.21.15.31:0/1279155509 conn(0x7ff3280fc950 msgr2=0x7ff3280fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:19.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 shutdown_connections 2023-10-06T21:46:19.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:18.996+0000 7ff32c9ff700 1 -- 172.21.15.31:0/1279155509 wait complete. 2023-10-06T21:46:19.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 -- 172.21.15.31:0/1954607985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 msgr2=0x7ffabc1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 --2- 172.21.15.31:0/1954607985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc1014b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009990 tx=0x7ffaac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 -- 172.21.15.31:0/1954607985 shutdown_connections 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 --2- 172.21.15.31:0/1954607985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 0x7ffabc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 --2- 172.21.15.31:0/1954607985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc1014b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 --2- 172.21.15.31:0/1954607985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc101a80 0x7ffabc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.214+0000 7ffac1dff700 1 -- 172.21.15.31:0/1954607985 >> 172.21.15.31:0/1954607985 conn(0x7ffabc0fc930 msgr2=0x7ffabc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:19.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.215+0000 7ffac1dff700 1 -- 172.21.15.31:0/1954607985 shutdown_connections 2023-10-06T21:46:19.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.215+0000 7ffac1dff700 1 -- 172.21.15.31:0/1954607985 wait complete. 2023-10-06T21:46:19.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.215+0000 7ffac1dff700 1 Processor -- start 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 -- start start 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 0x7ffabc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1a3070 0x7ffabc1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a59a0 con 0x7ffabc1010d0 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a5ae0 con 0x7ffabc1061f0 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffac1dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a5c50 con 0x7ffabc1a3070 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34064/0 (socket says 172.21.15.31:34064) 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffabb7fe700 1 -- 172.21.15.31:0/8791170 learned_addr learned my addr 172.21.15.31:0/8791170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.216+0000 7ffabaffd700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 0x7ffabc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabbfff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1a3070 0x7ffabc1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 -- 172.21.15.31:0/8791170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1a3070 msgr2=0x7ffabc1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1a3070 0x7ffabc1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 -- 172.21.15.31:0/8791170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 msgr2=0x7ffabc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 0x7ffabc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 -- 172.21.15.31:0/8791170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0095f0 con 0x7ffabc1010d0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffabb7fe700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.217+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d070 con 0x7ffabc1010d0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.218+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffaac004040 con 0x7ffabc1010d0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.218+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1a5ed0 con 0x7ffabc1010d0 2023-10-06T21:46:19.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.218+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac022dd0 con 0x7ffabc1010d0 2023-10-06T21:46:19.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.218+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1a6290 con 0x7ffabc1010d0 2023-10-06T21:46:19.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.219+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc1005a0 con 0x7ffabc1010d0 2023-10-06T21:46:19.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.221+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffaac022470 con 0x7ffabc1010d0 2023-10-06T21:46:19.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.222+0000 7ffab8ff9700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.222+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ffaac0ce680 con 0x7ffabc1010d0 2023-10-06T21:46:19.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.222+0000 7ffabaffd700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.223+0000 7ffabaffd700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffabc074bb0 tx=0x7ffab0009190 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:19.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.226+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffaac097960 con 0x7ffabc1010d0 2023-10-06T21:46:19.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.470+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 --> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ffabc1019f0 con 0x7ffaa40a15f0 2023-10-06T21:46:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.479+0000 7ffab8ff9700 1 -- 172.21.15.31:0/8791170 <== mgr.15570 v2:172.21.15.31:6800/2497223303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ffabc1019f0 con 0x7ffaa40a15f0 2023-10-06T21:46:19.483 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2023-10-06T21:46:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 msgr2=0x7ffaa40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffabc074bb0 tx=0x7ffab0009190 comp rx=0 tx=0).stop 2023-10-06T21:46:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 msgr2=0x7ffabc073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac004730 comp rx=0 tx=0).stop 2023-10-06T21:46:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 shutdown_connections 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1a3070 0x7ffabc1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffabc1010d0 0x7ffabc073390 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 --2- 172.21.15.31:0/8791170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffabc1061f0 0x7ffabc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 >> 172.21.15.31:0/8791170 conn(0x7ffabc0fc930 msgr2=0x7ffabc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 shutdown_connections 2023-10-06T21:46:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.482+0000 7ffac1dff700 1 -- 172.21.15.31:0/8791170 wait complete. 2023-10-06T21:46:19.503 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-06T21:46:19.514 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-06T21:46:19.515 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-06T21:46:19.515 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-06T21:46:19.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:46:19.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:19 smithi146 ceph-mon[102809]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:19.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:19 smithi031 ceph-mon[98539]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:19.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:19 smithi142 ceph-mon[102951]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 -- 172.21.15.31:0/4016362266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df41010b0 msgr2=0x7f1df4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/4016362266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df41010b0 0x7f1df4101490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009990 tx=0x7f1df0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 -- 172.21.15.31:0/4016362266 shutdown_connections 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/4016362266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41061d0 0x7f1df410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/4016362266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df41010b0 0x7f1df4101490 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/4016362266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df4101a60 0x7f1df4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.735+0000 7f1dfbfff700 1 -- 172.21.15.31:0/4016362266 >> 172.21.15.31:0/4016362266 conn(0x7f1df40fc8d0 msgr2=0x7f1df40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:19.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.736+0000 7f1dfbfff700 1 -- 172.21.15.31:0/4016362266 shutdown_connections 2023-10-06T21:46:19.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.736+0000 7f1dfbfff700 1 -- 172.21.15.31:0/4016362266 wait complete. 2023-10-06T21:46:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.736+0000 7f1dfbfff700 1 Processor -- start 2023-10-06T21:46:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 -- start start 2023-10-06T21:46:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41010b0 0x7f1df4196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df41061d0 0x7f1df4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df419c350 con 0x7f1df419b8d0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df419c4c0 con 0x7f1df41010b0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df419c680 con 0x7f1df41061d0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34084/0 (socket says 172.21.15.31:34084) 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.737+0000 7f1dfa59c700 1 -- 172.21.15.31:0/3787444951 learned_addr learned my addr 172.21.15.31:0/3787444951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1df9d9b700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41010b0 0x7f1df4196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1df959a700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df41061d0 0x7f1df4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 -- 172.21.15.31:0/3787444951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df41061d0 msgr2=0x7f1df4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df41061d0 0x7f1df4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 -- 172.21.15.31:0/3787444951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41010b0 msgr2=0x7f1df4196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41010b0 0x7f1df4196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 -- 172.21.15.31:0/3787444951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00095f0 con 0x7f1df419b8d0 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1dfa59c700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1dec007ee0 tx=0x7f1dec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.738+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec017070 con 0x7f1df419b8d0 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.739+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41a3be0 con 0x7f1df419b8d0 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.739+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dec00f040 con 0x7f1df419b8d0 2023-10-06T21:46:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.739+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec005710 con 0x7f1df419b8d0 2023-10-06T21:46:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.739+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df41a4070 con 0x7f1df419b8d0 2023-10-06T21:46:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.740+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df4067cd0 con 0x7f1df419b8d0 2023-10-06T21:46:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.742+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1dec005870 con 0x7f1df419b8d0 2023-10-06T21:46:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.743+0000 7f1deaffd700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 0x7f1de00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.743+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1dec0cc940 con 0x7f1df419b8d0 2023-10-06T21:46:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.743+0000 7f1df9d9b700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 0x7f1de00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.744+0000 7f1df9d9b700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 0x7f1de00a3af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b5c0 tx=0x7f1df0005fb0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.747+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1dec095b90 con 0x7f1df419b8d0 2023-10-06T21:46:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.961+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df4101c70 con 0x7f1df419b8d0 2023-10-06T21:46:19.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.963+0000 7f1deaffd700 1 -- 172.21.15.31:0/3787444951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+23280 (secure 0 0 0) 0x7f1dec0991b0 con 0x7f1df419b8d0 2023-10-06T21:46:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:46:19.966 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:46:14.268743+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1070417538":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/126831737":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1598815521":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/751435865":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 msgr2=0x7f1de00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 0x7f1de00a3af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b5c0 tx=0x7f1df0005fb0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 msgr2=0x7f1df419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1dec007ee0 tx=0x7f1dec00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 shutdown_connections 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1de00a1630 0x7f1de00a3af0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df419b8d0 0x7f1df419bd50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df41010b0 0x7f1df4196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 --2- 172.21.15.31:0/3787444951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df41061d0 0x7f1df4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.965+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 >> 172.21.15.31:0/3787444951 conn(0x7f1df40fc8d0 msgr2=0x7f1df4104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.966+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 shutdown_connections 2023-10-06T21:46:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:19.966+0000 7f1dfbfff700 1 -- 172.21.15.31:0/3787444951 wait complete. 2023-10-06T21:46:19.987 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:46:19.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:46:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 -- 172.21.15.31:0/368820504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1010b0 msgr2=0x7fc09c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/368820504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1010b0 0x7fc09c101490 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc090009990 tx=0x7fc090009ca0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 -- 172.21.15.31:0/368820504 shutdown_connections 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/368820504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 0x7fc09c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/368820504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1010b0 0x7fc09c101490 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/368820504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c101a60 0x7fc09c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 -- 172.21.15.31:0/368820504 >> 172.21.15.31:0/368820504 conn(0x7fc09c0fc950 msgr2=0x7fc09c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 -- 172.21.15.31:0/368820504 shutdown_connections 2023-10-06T21:46:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.192+0000 7fc0a31ff700 1 -- 172.21.15.31:0/368820504 wait complete. 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.193+0000 7fc0a31ff700 1 Processor -- start 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.193+0000 7fc0a31ff700 1 -- start start 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c1010b0 0x7fc09c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 0x7fc09c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c19c340 con 0x7fc09c19b8c0 2023-10-06T21:46:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c19c4b0 con 0x7fc09c1010b0 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a31ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c19c670 con 0x7fc09c1061d0 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34106/0 (socket says 172.21.15.31:34106) 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 -- 172.21.15.31:0/383497705 learned_addr learned my addr 172.21.15.31:0/383497705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc09bfff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 0x7fc09c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a0f9b700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c1010b0 0x7fc09c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 -- 172.21.15.31:0/383497705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 msgr2=0x7fc09c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 0x7fc09c197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 -- 172.21.15.31:0/383497705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c1010b0 msgr2=0x7fc09c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c1010b0 0x7fc09c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.194+0000 7fc0a179c700 1 -- 172.21.15.31:0/383497705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900095f0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc0a179c700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc094007ee0 tx=0x7fc09400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094017070 con 0x7fc09c19b8c0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc09400f040 con 0x7fc09c19b8c0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c1a3bd0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094005570 con 0x7fc09c19b8c0 2023-10-06T21:46:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.195+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c1a40b0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.196+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c067c60 con 0x7fc09c19b8c0 2023-10-06T21:46:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.198+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc09400db20 con 0x7fc09c19b8c0 2023-10-06T21:46:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.199+0000 7fc099ffb700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 0x7fc08c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.200+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc0940ccec0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.200+0000 7fc0a0f9b700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 0x7fc08c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.200+0000 7fc0a0f9b700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 0x7fc08c0a3b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc090006010 tx=0x7fc09000b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.202+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc094096110 con 0x7fc09c19b8c0 2023-10-06T21:46:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.426+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc09c101cb0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.426+0000 7fc099ffb700 1 -- 172.21.15.31:0/383497705 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc09c101cb0 con 0x7fc09c19b8c0 2023-10-06T21:46:20.430 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.428+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 msgr2=0x7fc08c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 0x7fc08c0a3b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc090006010 tx=0x7fc09000b260 comp rx=0 tx=0).stop 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 msgr2=0x7fc09c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc094007ee0 tx=0x7fc09400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 shutdown_connections 2023-10-06T21:46:20.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc08c0a1670 0x7fc08c0a3b30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c19b8c0 0x7fc09c19bd40 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc09c1010b0 0x7fc09c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 --2- 172.21.15.31:0/383497705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc09c1061d0 0x7fc09c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 >> 172.21.15.31:0/383497705 conn(0x7fc09c0fc950 msgr2=0x7fc09c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 shutdown_connections 2023-10-06T21:46:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.429+0000 7fc0a31ff700 1 -- 172.21.15.31:0/383497705 wait complete. 2023-10-06T21:46:20.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:46:20.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:20 smithi142 ceph-mon[102951]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:20.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:20 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:20.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:20 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3787444951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:46:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 -- 172.21.15.31:0/833874464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1061d0 msgr2=0x7f9e4c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 --2- 172.21.15.31:0/833874464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c10dd50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b0a0 tx=0x7f9e4400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 -- 172.21.15.31:0/833874464 shutdown_connections 2023-10-06T21:46:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 --2- 172.21.15.31:0/833874464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c10dd50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 --2- 172.21.15.31:0/833874464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 --2- 172.21.15.31:0/833874464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c101a60 0x7f9e4c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 -- 172.21.15.31:0/833874464 >> 172.21.15.31:0/833874464 conn(0x7f9e4c0fc950 msgr2=0x7f9e4c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 -- 172.21.15.31:0/833874464 shutdown_connections 2023-10-06T21:46:20.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.654+0000 7f9e523ff700 1 -- 172.21.15.31:0/833874464 wait complete. 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.655+0000 7f9e523ff700 1 Processor -- start 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.655+0000 7f9e523ff700 1 -- start start 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.655+0000 7f9e523ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.655+0000 7f9e523ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e523ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c073c30 0x7f9e4c1a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e523ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c1a58c0 con 0x7f9e4c1010b0 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e523ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c1a5a30 con 0x7f9e4c1061d0 2023-10-06T21:46:20.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e523ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c1a5bf0 con 0x7f9e4c073c30 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34130/0 (socket says 172.21.15.31:34130) 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 -- 172.21.15.31:0/2760215095 learned_addr learned my addr 172.21.15.31:0/2760215095 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e5099c700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c073c30 0x7f9e4c1a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e43fff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 -- 172.21.15.31:0/2760215095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c073c30 msgr2=0x7f9e4c1a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c073c30 0x7f9e4c1a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 -- 172.21.15.31:0/2760215095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 msgr2=0x7f9e4c0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 -- 172.21.15.31:0/2760215095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009d00 con 0x7f9e4c1010b0 2023-10-06T21:46:20.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e43fff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.656+0000 7f9e4bfff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00e9a0 tx=0x7f9e3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.657+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c009790 con 0x7f9e4c1010b0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.657+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e3c00cbd0 con 0x7f9e4c1010b0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.657+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c1a5ed0 con 0x7f9e4c1010b0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.657+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c018680 con 0x7f9e4c1010b0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.657+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c1a6350 con 0x7f9e4c1010b0 2023-10-06T21:46:20.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.658+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c067c60 con 0x7f9e4c1010b0 2023-10-06T21:46:20.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.660+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9e3c00cd40 con 0x7f9e4c1010b0 2023-10-06T21:46:20.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.661+0000 7f9e49ffb700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 0x7f9e380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:20.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.662+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f9e3c014070 con 0x7f9e4c1010b0 2023-10-06T21:46:20.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.662+0000 7f9e43fff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 0x7f9e380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:20.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.662+0000 7f9e43fff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 0x7f9e380a3ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0749a0 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:20.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.663+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e3c096a50 con 0x7f9e4c1010b0 2023-10-06T21:46:20.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:20 smithi031 ceph-mon[98539]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:20.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:20 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:20.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:20 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3787444951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:46:20.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:20 smithi146 ceph-mon[102809]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:20.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:20 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:20.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:20 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3787444951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:46:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.874+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e4c101c80 con 0x7f9e4c1010b0 2023-10-06T21:46:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.875+0000 7f9e49ffb700 1 -- 172.21.15.31:0/2760215095 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e3c09a070 con 0x7f9e4c1010b0 2023-10-06T21:46:20.878 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:46:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 msgr2=0x7f9e380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 0x7f9e380a3ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0749a0 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2023-10-06T21:46:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 msgr2=0x7f9e4c0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00e9a0 tx=0x7f9e3c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 shutdown_connections 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e4c073c30 0x7f9e4c1a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f9e380a15f0 0x7f9e380a3ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e4c1010b0 0x7f9e4c0731b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 --2- 172.21.15.31:0/2760215095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e4c1061d0 0x7f9e4c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 >> 172.21.15.31:0/2760215095 conn(0x7f9e4c0fc950 msgr2=0x7f9e4c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 shutdown_connections 2023-10-06T21:46:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:20.877+0000 7f9e523ff700 1 -- 172.21.15.31:0/2760215095 wait complete. 2023-10-06T21:46:20.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 -- 172.21.15.31:0/1664896407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca6010b470 msgr2=0x7fca6010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 --2- 172.21.15.31:0/1664896407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca6010b470 0x7fca6010d930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fca5800b0a0 tx=0x7fca5800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 -- 172.21.15.31:0/1664896407 shutdown_connections 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 --2- 172.21.15.31:0/1664896407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca6010b470 0x7fca6010d930 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 --2- 172.21.15.31:0/1664896407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102050 0x7fca60102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 --2- 172.21.15.31:0/1664896407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca60102970 0x7fca6010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 -- 172.21.15.31:0/1664896407 >> 172.21.15.31:0/1664896407 conn(0x7fca600fb890 msgr2=0x7fca600fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 -- 172.21.15.31:0/1664896407 shutdown_connections 2023-10-06T21:46:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.102+0000 7fca64dff700 1 -- 172.21.15.31:0/1664896407 wait complete. 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.103+0000 7fca64dff700 1 Processor -- start 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.103+0000 7fca64dff700 1 -- start start 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 0x7fca601996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 0x7fca6019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6019e180 con 0x7fca60102050 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6019e2f0 con 0x7fca60102970 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca64dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6019e4b0 con 0x7fca6010b470 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca5e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca5e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34150/0 (socket says 172.21.15.31:34150) 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca5e59c700 1 -- 172.21.15.31:0/3170834673 learned_addr learned my addr 172.21.15.31:0/3170834673 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.104+0000 7fca5dd9b700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 0x7fca601996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5ed9d700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 0x7fca6019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 -- 172.21.15.31:0/3170834673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 msgr2=0x7fca6019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 0x7fca6019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 -- 172.21.15.31:0/3170834673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 msgr2=0x7fca601996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 0x7fca601996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 -- 172.21.15.31:0/3170834673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca58009d00 con 0x7fca60102050 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5dd9b700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 0x7fca601996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5ed9d700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 0x7fca6019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:46:21.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca5e59c700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9a0 tx=0x7fca5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50009790 con 0x7fca60102050 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca5000cbd0 con 0x7fca60102050 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6019e790 con 0x7fca60102050 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50018680 con 0x7fca60102050 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.105+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca601a5fd0 con 0x7fca60102050 2023-10-06T21:46:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.106+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca600673d0 con 0x7fca60102050 2023-10-06T21:46:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.110+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fca5000cd40 con 0x7fca60102050 2023-10-06T21:46:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.111+0000 7fca4f7fe700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 0x7fca480a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.111+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fca50014070 con 0x7fca60102050 2023-10-06T21:46:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.112+0000 7fca5dd9b700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 0x7fca480a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.112+0000 7fca5dd9b700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 0x7fca480a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca54005e10 tx=0x7fca54005d60 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.113+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fca500968b0 con 0x7fca60102050 2023-10-06T21:46:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.329+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca60110ea0 con 0x7fca60102050 2023-10-06T21:46:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.330+0000 7fca4f7fe700 1 -- 172.21.15.31:0/3170834673 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca500260a0 con 0x7fca60102050 2023-10-06T21:46:21.333 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.332+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 msgr2=0x7fca480a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.332+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 0x7fca480a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca54005e10 tx=0x7fca54005d60 comp rx=0 tx=0).stop 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.332+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 msgr2=0x7fca601991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.332+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9a0 tx=0x7fca5000ed60 comp rx=0 tx=0).stop 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 shutdown_connections 2023-10-06T21:46:21.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca6010b470 0x7fca6019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fca480a1670 0x7fca480a3b30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca60102050 0x7fca601991a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 --2- 172.21.15.31:0/3170834673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60102970 0x7fca601996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 >> 172.21.15.31:0/3170834673 conn(0x7fca600fb890 msgr2=0x7fca600fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 shutdown_connections 2023-10-06T21:46:21.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.333+0000 7fca64dff700 1 -- 172.21.15.31:0/3170834673 wait complete. 2023-10-06T21:46:21.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T21:46:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 -- 172.21.15.31:0/2749678039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1061f0 msgr2=0x7f8fac10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/2749678039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1061f0 0x7f8fac10dd70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b0a0 tx=0x7f8fa400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 -- 172.21.15.31:0/2749678039 shutdown_connections 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/2749678039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1061f0 0x7f8fac10dd70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/2749678039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1010d0 0x7f8fac1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/2749678039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac101a80 0x7f8fac105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 -- 172.21.15.31:0/2749678039 >> 172.21.15.31:0/2749678039 conn(0x7f8fac0fc990 msgr2=0x7f8fac0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 -- 172.21.15.31:0/2749678039 shutdown_connections 2023-10-06T21:46:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.555+0000 7f8fb25ff700 1 -- 172.21.15.31:0/2749678039 wait complete. 2023-10-06T21:46:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.556+0000 7f8fb25ff700 1 Processor -- start 2023-10-06T21:46:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 -- start start 2023-10-06T21:46:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 0x7f8fac197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 0x7f8fac19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac19c400 con 0x7f8fac1010d0 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac19c570 con 0x7f8fac1061f0 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fb25ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac19c730 con 0x7f8fac19b980 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34168/0 (socket says 172.21.15.31:34168) 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.557+0000 7f8fabfff700 1 -- 172.21.15.31:0/326846109 learned_addr learned my addr 172.21.15.31:0/326846109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 0x7f8fac197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fb0b9c700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 0x7f8fac19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 -- 172.21.15.31:0/326846109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 msgr2=0x7f8fac19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 0x7f8fac19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 -- 172.21.15.31:0/326846109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 msgr2=0x7f8fac197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 0x7f8fac197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 -- 172.21.15.31:0/326846109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4009d00 con 0x7f8fac1010d0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fb0b9c700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 0x7f8fac19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 0x7f8fac197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fabfff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00d950 tx=0x7f8f9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c009b40 con 0x7f8fac1010d0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f9c009ca0 con 0x7f8fac1010d0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00f620 con 0x7f8fac1010d0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.558+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1a3c90 con 0x7f8fac1010d0 2023-10-06T21:46:21.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.559+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1a4170 con 0x7f8fac1010d0 2023-10-06T21:46:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.560+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac067cd0 con 0x7f8fac1010d0 2023-10-06T21:46:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.562+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8f9c010460 con 0x7f8fac1010d0 2023-10-06T21:46:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.563+0000 7f8fa9ffb700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 0x7f8f980a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:46:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.563+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f8f9c0cd450 con 0x7f8fac1010d0 2023-10-06T21:46:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.563+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 0x7f8f980a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:46:21.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.564+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 0x7f8f980a3b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fac1985f0 tx=0x7f8f94005d80 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:46:21.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.567+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f9c096700 con 0x7f8fac1010d0 2023-10-06T21:46:21.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:21 smithi142 ceph-mon[102951]: pgmap v806: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:21.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:21 smithi142 ceph-mon[102951]: from='client.18123 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:21.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:21 smithi142 ceph-mon[102951]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:21.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:21 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/383497705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:21 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2760215095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:21 smithi031 ceph-mon[98539]: pgmap v806: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:21 smithi031 ceph-mon[98539]: from='client.18123 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:21 smithi031 ceph-mon[98539]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:21 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/383497705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:21 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2760215095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:21 smithi146 ceph-mon[102809]: pgmap v806: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:21 smithi146 ceph-mon[102809]: from='client.18123 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:46:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:21 smithi146 ceph-mon[102809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T21:46:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:21 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/383497705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:21 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2760215095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:46:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.840+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f8fac198330 con 0x7f8fac1010d0 2023-10-06T21:46:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.851+0000 7f8fa9ffb700 1 -- 172.21.15.31:0/326846109 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+734409 (secure 0 0 0) 0x7f8f9c099d20 con 0x7f8fac1010d0 2023-10-06T21:46:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 25, 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 15570, 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2023-10-06T21:46:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2497223303 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2497223303 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/2497223303", 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2023-10-06T21:20:56.899865+0000", 2023-10-06T21:46:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540138322906710015, 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 25300, 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138322906710015, 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:46:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T21:46:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T21:46:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T21:46:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:46:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T21:46:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:46:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T21:46:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:46:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T21:46:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:46:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-10-06T21:46:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:46:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:46:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:46:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T21:46:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2023-10-06T21:46:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:46:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T21:46:21.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T21:46:21.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:46:21.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.952 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:21.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.956 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:21.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:46:21.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T21:46:21.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.963 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.965 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T21:46:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.968 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:46:21.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T21:46:21.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:46:21.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T21:46:21.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:21.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:21.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.993 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:46:21.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:21.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:46:22.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:46:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:46:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:46:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:46:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:46:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:46:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:46:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T21:46:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T21:46:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T21:46:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:46:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T21:46:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.106 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T21:46:22.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T21:46:22.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.120 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.124 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.127 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.133 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.136 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T21:46:22.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.141 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T21:46:22.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.148 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-10-06T21:46:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T21:46:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T21:46:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T21:46:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:46:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T21:46:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T21:46:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:46:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:46:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T21:46:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T21:46:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2023-10-06T21:46:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T21:46:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T21:46:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 25334, 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138322906710015, 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T21:46:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T21:46:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T21:46:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T21:46:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T21:46:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T21:46:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:46:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T21:46:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:46:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T21:46:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T21:46:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:46:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2023-10-06T21:46:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T21:46:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:46:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-10-06T21:46:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-10-06T21:46:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T21:46:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T21:46:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T21:46:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:46:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:46:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:46:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T21:46:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T21:46:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T21:46:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:46:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T21:46:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:46:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T21:46:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:46:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:46:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:46:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:46:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:46:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:46:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:46:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T21:46:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:46:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T21:46:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T21:46:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T21:46:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:46:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T21:46:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T21:46:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T21:46:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-10-06T21:46:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:46:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T21:46:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:46:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:46:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T21:46:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T21:46:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T21:46:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T21:46:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T21:46:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T21:46:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T21:46:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T21:46:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T21:46:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T21:46:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T21:46:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T21:46:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T21:46:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T21:46:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T21:46:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T21:46:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T21:46:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T21:46:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T21:46:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T21:46:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:46:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T21:46:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:46:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T21:46:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:46:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T21:46:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:46:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T21:46:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T21:46:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T21:46:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:46:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T21:46:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-10-06T21:46:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T21:46:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:46:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:46:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:46:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:46:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:46:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:46:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:46:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:46:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:46:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:46:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T21:46:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:46:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:46:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T21:46:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T21:46:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T21:46:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T21:46:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T21:46:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T21:46:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T21:46:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T21:46:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-10-06T21:46:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T21:46:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T21:46:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T21:46:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T21:46:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T21:46:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T21:46:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T21:46:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T21:46:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T21:46:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T21:46:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T21:46:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-10-06T21:46:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:46:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T21:46:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T21:46:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T21:46:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T21:46:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T21:46:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T21:46:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T21:46:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T21:46:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T21:46:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T21:46:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T21:46:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T21:46:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 91, 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3443401625 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2353374026 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1233158868 2023-10-06T21:46:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 903547883 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2586626840 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1574470222 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T21:46:23.408 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:46:23.410 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:22 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3170834673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:46:23.410 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:22 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/326846109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:46:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:22 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3170834673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:46:23.410 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:22 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/326846109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 msgr2=0x7f8f980a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 0x7f8f980a3b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fac1985f0 tx=0x7f8f94005d80 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 msgr2=0x7f8fac196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00d950 tx=0x7f8f9c00dd10 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 shutdown_connections 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac19b980 0x7f8fac19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f8f980a16b0 0x7f8f980a3b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac1010d0 0x7f8fac196fe0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 --2- 172.21.15.31:0/326846109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fac1061f0 0x7f8fac197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 >> 172.21.15.31:0/326846109 conn(0x7f8fac0fc990 msgr2=0x7f8fac0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 shutdown_connections 2023-10-06T21:46:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:46:21.858+0000 7f8fb25ff700 1 -- 172.21.15.31:0/326846109 wait complete. 2023-10-06T21:46:23.418 DEBUG:tasks.check_counter:Getting stats from a 2023-10-06T21:46:23.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-06T21:46:23.419 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:22 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3170834673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:46:23.419 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:22 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/326846109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:46:23.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:23 smithi142 ceph-mon[102951]: pgmap v807: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:23.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:23 smithi031 ceph-mon[98539]: pgmap v807: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:23.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:23 smithi146 ceph-mon[102809]: pgmap v807: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2023-10-06T21:46:24.165 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:46:25.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:25 smithi031 ceph-mon[98539]: pgmap v808: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 KiB/s wr, 157 op/s 2023-10-06T21:46:25.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:25 smithi142 ceph-mon[102951]: pgmap v808: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 KiB/s wr, 157 op/s 2023-10-06T21:46:25.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:25 smithi146 ceph-mon[102809]: pgmap v808: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 KiB/s wr, 157 op/s 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 3353, 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 3353, 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 76497996, 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 1041988, 2023-10-06T21:46:25.957 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 8, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.453672914, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.110646190, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.207341388, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.099294199, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3352, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.159121864, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000047470 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 832, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1166, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 71657549, 2023-10-06T21:46:25.959 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 1006454, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 11, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 7, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.238859492, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.046336049, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.155168095, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.025120296, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1163, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.074811643, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000064326 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.960 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 748256 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1928, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1890, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 56610211, 2023-10-06T21:46:25.961 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 710097, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.326516697, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.078026129, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.185535076, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.057086089, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1889, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.095906335, 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000050770 2023-10-06T21:46:25.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1728816, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 275008 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-10-06T21:46:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 9174, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.267143288, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000029119 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-10-06T21:46:25.964 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 8289, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 53, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 368, 2023-10-06T21:46:25.966 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 7868, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 8212, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:25.967 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2023-10-06T21:46:25.968 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:25.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2023-10-06T21:46:25.970 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2023-10-06T21:46:25.971 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2023-10-06T21:46:25.972 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 44214, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 67, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 5, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 205663901, 2023-10-06T21:46:25.973 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 205655732, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 44214 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1500, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 8193, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 6693, 2023-10-06T21:46:25.974 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 59, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 80, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 21, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 8289, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 8551, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 262, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 63260, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 207152 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2023-10-06T21:46:25.976 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.977 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.978 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.979 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2023-10-06T21:46:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2023-10-06T21:46:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 2, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 3, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 1, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 3, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2023-10-06T21:46:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 4134, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 241009, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 83, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 176, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 9135288, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 40803, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 5578, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 5578, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 5578, 2023-10-06T21:46:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 5.812912289, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001042114 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 5578, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 5579 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 5578, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 5578, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2023-10-06T21:46:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 3182, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 2394, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2023-10-06T21:46:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2023-10-06T21:46:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 132, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 46, 2023-10-06T21:46:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 11, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 14, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 3, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2023-10-06T21:46:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 6109, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 204186210, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 6109, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 6109, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 204186210, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 5578, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 201482390, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 5578, 2023-10-06T21:46:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 201482390, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 5578, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 5578, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 5578, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 5578, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55ff8801ca00": { 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:26.011 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:46:26.466 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-06T21:46:26.466 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-06T21:46:26.466 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-10-06T21:46:26.466 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-10-06T21:46:26.466 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-10-06T21:46:26.467 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:26.467 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:26.467 DEBUG:tasks.check_counter:Getting stats from d 2023-10-06T21:46:26.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-06T21:46:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:46:27.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:27 smithi031 ceph-mon[98539]: pgmap v809: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 141 op/s 2023-10-06T21:46:27.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:27 smithi142 ceph-mon[102951]: pgmap v809: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 141 op/s 2023-10-06T21:46:27.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:27 smithi146 ceph-mon[102809]: pgmap v809: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 141 op/s 2023-10-06T21:46:28.806 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:46:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 457, 2023-10-06T21:46:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 447, 2023-10-06T21:46:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1492048, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 269984, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.083912025, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.032998895, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.042636573, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.010364720, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 446, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.021651727, 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000048546 2023-10-06T21:46:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1492048, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 269984 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 7, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 344, 2023-10-06T21:46:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3472, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 290656, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.033294510, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.023679635, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.004589079, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 342, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.026535435, 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000077588 2023-10-06T21:46:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3472, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 290656 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2023-10-06T21:46:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000977250, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000823379, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2023-10-06T21:46:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148714, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 460, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1427455, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 460, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 460, 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1427455, 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:28.819 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-10-06T21:46:29.253 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-10-06T21:46:29.253 DEBUG:tasks.check_counter:Getting stats from g 2023-10-06T21:46:29.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-06T21:46:29.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:29 smithi031 ceph-mon[98539]: pgmap v810: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 605 KiB/s wr, 130 op/s 2023-10-06T21:46:29.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:29 smithi142 ceph-mon[102951]: pgmap v810: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 605 KiB/s wr, 130 op/s 2023-10-06T21:46:29.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:29 smithi146 ceph-mon[102809]: pgmap v810: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 605 KiB/s wr, 130 op/s 2023-10-06T21:46:30.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:46:31.568 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:46:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2429, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2429, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 84647556, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 754624, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 9, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.410711963, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.109007670, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.199392984, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.073973408, 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2428, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.284409173, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000117137 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2532, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2868, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 68798646, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 1624368, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 10, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 7, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.339313813, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.083943527, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.166455869, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.067186146, 2023-10-06T21:46:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2865, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.133264431, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000046514 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:46:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 837792 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1202, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1164, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 77410023, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 484948, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.284077770, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.054955142, 2023-10-06T21:46:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.199298400, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.030368735, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1163, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.070672575, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000060767 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1729360, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 276256 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 9487, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.258877159, 2023-10-06T21:46:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000027287 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2023-10-06T21:46:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2023-10-06T21:46:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 8923, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 8923, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 8923, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2023-10-06T21:46:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2023-10-06T21:46:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2023-10-06T21:46:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2023-10-06T21:46:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 50590, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2023-10-06T21:46:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 75, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 6, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 231729842, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 231729243, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 50590 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 1807, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 9058, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 7251, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 65, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 100, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 35, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 8923, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 9432, 2023-10-06T21:46:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 509, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 65852, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 207152 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2023-10-06T21:46:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2023-10-06T21:46:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2023-10-06T21:46:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 2, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 2, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 0, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 2, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 3266, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 240265, 2023-10-06T21:46:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 82, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 176, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 10260223, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 44635, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2023-10-06T21:46:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 5625, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 5625, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 5625, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 5.569916335, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000990207 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 5625, 2023-10-06T21:46:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 5626 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 5625, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 5625, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 3220, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 2403, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2023-10-06T21:46:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2023-10-06T21:46:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2023-10-06T21:46:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 132, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 46, 2023-10-06T21:46:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 11, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 13, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 2, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2023-10-06T21:46:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 6159, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 230272032, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 6159, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 6159, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 230272032, 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 5625, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 227558879, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 5625, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 227558879, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 5625, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 5625, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 5625, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 5625, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55713ab16a00": { 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:31.621 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:46:31.621 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:31 smithi031 ceph-mon[98539]: pgmap v811: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 612 KiB/s wr, 140 op/s 2023-10-06T21:46:31.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:31 smithi142 ceph-mon[102951]: pgmap v811: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 612 KiB/s wr, 140 op/s 2023-10-06T21:46:31.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:31 smithi146 ceph-mon[102809]: pgmap v811: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 612 KiB/s wr, 140 op/s 2023-10-06T21:46:32.005 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-10-06T21:46:32.005 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-10-06T21:46:32.005 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-10-06T21:46:32.005 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-10-06T21:46:32.006 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-10-06T21:46:32.006 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:32.006 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:32.006 DEBUG:tasks.check_counter:Getting stats from j 2023-10-06T21:46:32.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-06T21:46:32.746 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:46:33.615 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:33 smithi031 ceph-mon[98539]: pgmap v812: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 27 op/s 2023-10-06T21:46:33.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:33 smithi142 ceph-mon[102951]: pgmap v812: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 27 op/s 2023-10-06T21:46:33.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:33 smithi146 ceph-mon[102809]: pgmap v812: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 27 op/s 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.004255677, 2023-10-06T21:46:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000068257, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001327673, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.001201252, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000400417 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 459, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 449, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1492592, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 271232, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.080114575, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.031547794, 2023-10-06T21:46:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.043917580, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.010433841, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 448, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.023936077, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000053428 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1492592, 2023-10-06T21:46:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 271232 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 3, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 344, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 480, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 290560, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.030581159, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.022369687, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.003230129, 2023-10-06T21:46:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 341, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.032348013, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000094862 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 290560 2023-10-06T21:46:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-10-06T21:46:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148714, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 462, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1427715, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 462, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 462, 2023-10-06T21:46:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1427715, 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T21:46:34.324 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T21:46:34.846 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-06T21:46:34.847 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-06T21:46:34.847 DEBUG:tasks.check_counter:Getting stats from b 2023-10-06T21:46:34.847 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-06T21:46:35.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:35 smithi142 ceph-mon[102951]: pgmap v813: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 32 op/s 2023-10-06T21:46:35.703 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:46:35.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:35 smithi031 ceph-mon[98539]: pgmap v813: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 32 op/s 2023-10-06T21:46:35.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:35 smithi146 ceph-mon[102809]: pgmap v813: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 32 op/s 2023-10-06T21:46:36.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:36 smithi142 ceph-mon[102951]: pgmap v814: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:36.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:36 smithi031 ceph-mon[98539]: pgmap v814: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:36.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:36 smithi146 ceph-mon[102809]: pgmap v814: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 294925, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 344985, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 102052535, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 216139121, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 11, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:37.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 28.868892842, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 12.993741873, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 16.507579795, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 2.185126444, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 344989, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 13.880754752, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000040235 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 599246, 2023-10-06T21:46:37.681 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.121134238, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000202 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 602528 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 801460, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 815330, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 235667031, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 608538758, 2023-10-06T21:46:37.682 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 19, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 61.833941260, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 24.675989210, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 40.520761550, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 4.261889575, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 815328, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 26.268712236, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000032218 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.683 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1562772, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.276129575, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000176 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 924464, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 400288 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 292866, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 329597, 2023-10-06T21:46:37.684 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 91619096, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 143503531, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 268, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 10, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 27.641163995, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 12.690479943, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 16.995651865, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.387664837, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 329590, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 12.694414855, 2023-10-06T21:46:37.685 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038515 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 599734, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.115301973, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000192 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1817744, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 296432 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-10-06T21:46:37.686 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 62259, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 35.440326154, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000569240 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.687 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8910, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.597573316, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000067067 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 741973, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 708426, 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-10-06T21:46:37.688 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 708426, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 610.147551899, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000861272 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 33547, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 4, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 1, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 2031, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 1, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 2039, 2023-10-06T21:46:37.689 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 480, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1549, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 10, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1264, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 2254, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 3, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 3, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 6, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 1440506, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 1424265, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 7565, 2023-10-06T21:46:37.690 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 188, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 150, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 207, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 42, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 1829, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 75, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 2263, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:37.691 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 65, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 246, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 65, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 44, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 4, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 40, 2023-10-06T21:46:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 21, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 27, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 31172, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 10518, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 36678, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 4311, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 614, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 12540 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.693 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 507, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 351, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 225, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 319, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 9040, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 33, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 3146, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 3113, 2023-10-06T21:46:37.694 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 4, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 49, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 1, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:37.695 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 46061, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 45530, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 45530, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 531, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 346, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 179, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 177, 2023-10-06T21:46:37.696 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 177, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 1, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 1, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 267793584, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 271263230, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 31601, 2023-10-06T21:46:37.697 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 124.844646700, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003950654 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1679, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 6559, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 4880, 2023-10-06T21:46:37.698 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 92, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 534, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 442, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 2039, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 10854, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 8815, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 3, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 3986, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 3983, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 73940, 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207152 2023-10-06T21:46:37.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 780732, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 21701, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 741973, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 459, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 44812, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3741, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 10.755464857, 2023-10-06T21:46:37.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002875024 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 666152, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 264.417760924, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000396933 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26438, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 179.496999265, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006789356 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.703 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.007599282, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000292280 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.706 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 119, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.369736286, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003107027 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-10-06T21:46:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 16.073477450, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.618210671 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1895, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.991466684, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001050905 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5947, 2023-10-06T21:46:37.708 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 31.471211858, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005291947 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 786, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 96.179961857, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.122366363 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.709 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 44, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.412854314, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.009383052 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:37.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.260723463, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013036173 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5, 2023-10-06T21:46:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.036428664, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007285732 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 108, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.041428836, 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000383600 2023-10-06T21:46:37.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.018221283, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003036880 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-10-06T21:46:37.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3113, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 8.614216876, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002767175 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:37.714 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 5, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2023-10-06T21:46:37.715 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 39130, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 19565, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1544, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 240, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 240, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:37.716 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:37.717 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:37.718 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 584112, 2023-10-06T21:46:37.719 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 193, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:37.720 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 5305684, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 22657, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-10-06T21:46:37.721 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 42413, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 279267779, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 37, 2023-10-06T21:46:37.722 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 42413, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 42413, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 266.370341533, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006280393 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 42413, 2023-10-06T21:46:37.723 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 50118 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 42413, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1077, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 41336, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1905, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 1549, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1003, 2023-10-06T21:46:37.724 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 32600, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 1001, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 1, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 3750, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 74, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 3098, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-10-06T21:46:37.726 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-10-06T21:46:37.727 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 132, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 47, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-10-06T21:46:37.728 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 4548, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 10, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 579 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-10-06T21:46:37.729 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 1 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 3113, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:37.730 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1389247, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 304172014, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1389247, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1389247, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 304172014, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.731 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:37.732 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 42413, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 279585388, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 42413, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 279585388, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:37.733 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 42413, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 42413, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 42413, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 42413, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.734 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 3113, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 317677, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 3113, 2023-10-06T21:46:37.735 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 999, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 317677, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55c75b2106e0": { 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:37.736 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 46061, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 267068926, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 46061, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 31601, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 267068926, 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:37.737 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:37.738 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:37.738 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:37.738 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.738 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:37.738 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-10-06T21:46:38.075 INFO:tasks.check_counter:Daemon mds.b mds.exported=42 2023-10-06T21:46:38.075 INFO:tasks.check_counter:Daemon mds.b mds.imported=75 2023-10-06T21:46:38.075 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=507 2023-10-06T21:46:38.075 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=351 2023-10-06T21:46:38.076 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-10-06T21:46:38.076 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2023-10-06T21:46:38.076 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=20 2023-10-06T21:46:38.076 DEBUG:tasks.check_counter:Getting stats from e 2023-10-06T21:46:38.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-06T21:46:38.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:38 smithi142 ceph-mon[102951]: pgmap v815: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:38.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:38 smithi031 ceph-mon[98539]: pgmap v815: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:38.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:38 smithi146 ceph-mon[102809]: pgmap v815: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-10-06T21:46:38.835 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 436441, 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 449529, 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 149178848, 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 336795629, 2023-10-06T21:46:41.538 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 20, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 34.525234723, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 14.461219781, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 23.456041953, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.905133573, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 449528, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 15.202234546, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000033818 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 848092, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.163828636, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000193 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 91422, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 87914, 2023-10-06T21:46:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 33015471, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 99048713, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 11, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 7.518551542, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.139731721, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 4.571399665, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.498178800, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 87913, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.089414307, 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000046516 2023-10-06T21:46:41.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 163025, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.032187395, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000197 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1753264, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 883968 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:41.542 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 245484, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 207894, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 74337379, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 107697286, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 19, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 23.691204629, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 12.099123252, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 14.476336468, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.396751701, 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 207891, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.957989761, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038279 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 434182, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.087592190, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000201 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 4224, 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 112176 2023-10-06T21:46:41.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 28652, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 22.588471605, 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000788373 2023-10-06T21:46:41.545 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 14810, 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.074027026, 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000072520 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-10-06T21:46:41.546 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 291488, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 285953, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 285953, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 188.232366847, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000658263 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 5535, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 35, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 3, 2023-10-06T21:46:41.547 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1743, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 212, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1432, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 99, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1440, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 451, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 6, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 566650, 2023-10-06T21:46:41.548 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 540879, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 1650, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 302, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 278, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 5, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 242, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 14, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 565, 2023-10-06T21:46:41.549 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 246, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:41.550 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 15262, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 19373, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 25371, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 7505, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 695, 2023-10-06T21:46:41.551 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 23294 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 365, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 5281, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 2696, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 5525, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1529, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 20, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 6518, 2023-10-06T21:46:41.552 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 6498, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 5, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 3, 2023-10-06T21:46:41.553 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 44219, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 44219, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 67, 2023-10-06T21:46:41.554 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 67, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 5, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 205677416, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 14441, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 54.601500476, 2023-10-06T21:46:41.555 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003781005 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1457, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 8283, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 6826, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 77, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 3005, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 2928, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1778, 2023-10-06T21:46:41.556 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 11984, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 10206, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 6525, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 6525, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 68636, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198960 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 318646, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 3155, 2023-10-06T21:46:41.557 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 291488, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 510, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 9059, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6380, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 22.350575285, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003503224 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 246641, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 84.772656927, 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000343708 2023-10-06T21:46:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-10-06T21:46:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8404, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5.647636494, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000672017 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.560 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.561 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-10-06T21:46:41.563 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3800, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.492844538, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000919169 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 12820, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 49.302417239, 2023-10-06T21:46:41.564 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003845742 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1290, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.393941353, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003406156 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:41.565 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.154939772, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007746988 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.566 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 100, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.034422715, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000344227 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-10-06T21:46:41.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6498, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 18.082932524, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002782845 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.570 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 33643, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 16821, 2023-10-06T21:46:41.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1548, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:41.572 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:41.573 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:41.574 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 466824, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 159, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:41.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 4566939, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 19913, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-10-06T21:46:41.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 24511, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 202714630, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 27, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 24511, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 24511, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 93.359399251, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003808877 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.577 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 24511, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 24845 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 24511, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1300, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 23211, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2023-10-06T21:46:41.578 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1300, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 15741, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 717, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 3, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6558, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2023-10-06T21:46:41.579 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-10-06T21:46:41.580 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-10-06T21:46:41.581 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 132, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 46, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 16, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 4, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 348, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 148 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.582 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T21:46:41.583 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 6498, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 773343, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 186898344, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:41.584 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 773343, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 773343, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 186898344, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:41.585 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 24511, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 203377624, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 24511, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 203377624, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 24511, 2023-10-06T21:46:41.587 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 24511, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 24511, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 24511, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:41.588 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 6498, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 662994, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 6498, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1300, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 662994, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55ac6dff46e0": { 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 44219, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 201483112, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 44219, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:41.590 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 14441, 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 201483112, 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:41.591 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-10-06T21:46:41.592 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:41 smithi146 ceph-mon[102809]: pgmap v816: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2023-10-06T21:46:41.592 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:41 smithi142 ceph-mon[102951]: pgmap v816: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2023-10-06T21:46:41.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:41 smithi031 ceph-mon[98539]: pgmap v816: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2023-10-06T21:46:42.032 INFO:tasks.check_counter:Daemon mds.e mds.exported=5 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds.imported=14 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=365 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=5281 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:42.033 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:42.033 DEBUG:tasks.check_counter:Getting stats from h 2023-10-06T21:46:42.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-06T21:46:42.762 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:46:43.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:43 smithi146 ceph-mon[102809]: pgmap v817: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-10-06T21:46:43.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:43 smithi142 ceph-mon[102951]: pgmap v817: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-10-06T21:46:43.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:43 smithi031 ceph-mon[98539]: pgmap v817: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-10-06T21:46:44.314 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2839, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2841, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 97305873, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 882939, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.384864433, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.096890753, 2023-10-06T21:46:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.188557039, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.070871810, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2839, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.110734776, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000039004 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.316 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2086, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2062, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 97280040, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 742825, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.306925801, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.065971645, 2023-10-06T21:46:44.317 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.178893424, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.047983676, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2061, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.078321918, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038001 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.318 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 451872, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 229920 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1028, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1306, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 66048507, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 918227, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 7, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.193656886, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.034672315, 2023-10-06T21:46:44.319 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.122608440, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.025156045, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1305, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.053931616, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000041326 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.320 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 600384 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 9431, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.313709135, 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000033263 2023-10-06T21:46:44.321 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2023-10-06T21:46:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2023-10-06T21:46:44.323 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 4940, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 2524, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 2392, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 24, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1334, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 170, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2023-10-06T21:46:44.324 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:44.325 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 246, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:44.326 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2023-10-06T21:46:44.327 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:44.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-10-06T21:46:44.329 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 43014, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 3, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 102, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-10-06T21:46:44.330 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 267793584, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 271307255, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 271294048, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 43014 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-10-06T21:46:44.331 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1665, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 6157, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 4492, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 78, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 178, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 100, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 4940, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 6886, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 1946, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2023-10-06T21:46:44.332 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 60560, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207128 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-10-06T21:46:44.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.336 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.337 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-10-06T21:46:44.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.341 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.342 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-10-06T21:46:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:44.345 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 0, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-10-06T21:46:44.346 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 3284, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:44.347 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:44.348 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:44.349 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 240199, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 79, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-10-06T21:46:44.350 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-10-06T21:46:44.351 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 7069932, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 27906, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 5510, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2023-10-06T21:46:44.352 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 5510, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5510, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5.056911633, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000917769 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5510, 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5511 2023-10-06T21:46:44.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 5510, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 5510, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2759, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 2748, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-10-06T21:46:44.354 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:44.355 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-10-06T21:46:44.356 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 132, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-10-06T21:46:44.357 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 40, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 11, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 11, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:44.358 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2023-10-06T21:46:44.359 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 5949, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 260073941, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 5949, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:44.360 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5949, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 260073941, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:44.361 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5510, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 258667564, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5510, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 258667564, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.362 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5510, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 5510, 2023-10-06T21:46:44.363 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5510, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 5510, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x555de876e8c0": { 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-10-06T21:46:44.366 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:44.367 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-10-06T21:46:44.839 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-10-06T21:46:44.839 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-10-06T21:46:44.839 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-10-06T21:46:44.839 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-10-06T21:46:44.839 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-10-06T21:46:44.840 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:44.840 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:44.840 DEBUG:tasks.check_counter:Getting stats from k 2023-10-06T21:46:44.840 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-06T21:46:45.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:45 smithi146 ceph-mon[102809]: pgmap v818: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2023-10-06T21:46:45.568 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:46:45.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:45 smithi142 ceph-mon[102951]: pgmap v818: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2023-10-06T21:46:45.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:45 smithi031 ceph-mon[98539]: pgmap v818: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2023-10-06T21:46:47.124 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-10-06T21:46:47.124 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 341294, 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 373363, 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 104113839, 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 286815616, 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 27, 2023-10-06T21:46:47.125 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 28.225188644, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 12.039250279, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 19.199500197, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.426492055, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 373355, 2023-10-06T21:46:47.126 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 12.263107698, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000032845 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 683731, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.131286328, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000192 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1746096, 2023-10-06T21:46:47.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 888224 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 268489, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 226307, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 94975612, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 133506242, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 12, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 22.897008511, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 11.077433878, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 14.741876688, 2023-10-06T21:46:47.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.467968565, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 226309, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 9.467639211, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000041835 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 476306, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.090179958, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000189 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 5232, 2023-10-06T21:46:47.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 126192 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 176377, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 178769, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 56890669, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 186490940, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 16.047787562, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 7.621122478, 2023-10-06T21:46:47.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 9.405285930, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.942821022, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 178770, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.939547581, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000044412 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 330434, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.059322973, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000179 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26901, 2023-10-06T21:46:47.132 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 24.943978610, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000927250 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 15351, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.155747506, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000075288 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.133 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 302857, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 301427, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 301427, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 212.818140426, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000706035 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 1430, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:47.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 60, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 6, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 1, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 2041, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 278, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1677, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 86, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1643, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 745, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 1, 2023-10-06T21:46:47.135 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 1, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 6, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 582704, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 557952, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 1427, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 318, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 278, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 13, 2023-10-06T21:46:47.136 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 485, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 6, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 210, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 246, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:47.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 16010, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 21693, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 1, 2023-10-06T21:46:47.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 29384, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 9626, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 705, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 24528 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 1565, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 5444, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 2777, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 5723, 2023-10-06T21:46:47.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1694, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 20, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 6841, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 6821, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-10-06T21:46:47.140 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 13, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 7, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 50591, 2023-10-06T21:46:47.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 50591, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 75, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 75, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 6, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-10-06T21:46:47.142 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 231742853, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 13507, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 53.574491358, 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003966424 2023-10-06T21:46:47.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1752, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 9149, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 7397, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 82, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 3099, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 3017, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 2045, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 13520, 2023-10-06T21:46:47.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 11475, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 1, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 6850, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 6849, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 70880, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198960 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 330537, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 25, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 302857, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 645, 2023-10-06T21:46:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 1707, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6670, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 23.388036333, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003506452 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 252209, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 89.974990646, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000356747 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-10-06T21:46:47.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 9649, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.105942684, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000736443 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4902, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.267572517, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000870577 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 19572, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 62.245101463, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003180313 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-10-06T21:46:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1392, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.567129017, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003280983 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.116507202, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005825360 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:47.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 192, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.070723171, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000368349 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.157 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6821, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 21.082137393, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003090769 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:47.158 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.159 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 35294, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 17647, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1558, 2023-10-06T21:46:47.160 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:47.161 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:47.162 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:47.163 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:47.164 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:47.165 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 449809, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 148, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:47.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 5380931, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 22790, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-10-06T21:46:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-10-06T21:46:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 23992, 2023-10-06T21:46:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 229122234, 2023-10-06T21:46:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 27, 2023-10-06T21:46:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 23992, 2023-10-06T21:46:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-10-06T21:46:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23992, 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 95.744361548, 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003990678 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-10-06T21:46:47.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23992, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 24352 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 23992, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1324, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 22668, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1324, 2023-10-06T21:46:47.172 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 14831, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 725, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 6, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6889, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:47.173 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-10-06T21:46:47.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-10-06T21:46:47.175 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-10-06T21:46:47.176 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 132, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 46, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 16, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 4, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 396, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2023-10-06T21:46:47.177 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 151 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.178 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 6821, 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:47.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 786156, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 185193371, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 786156, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 786156, 2023-10-06T21:46:47.180 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 185193371, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:47.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 23992, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 229822843, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 23992, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 229822843, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 23992, 2023-10-06T21:46:47.183 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 23992, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 23992, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 23992, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:47.184 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 6821, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 700609, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 6821, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1324, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 700609, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55900390e6e0": { 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 50591, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 227548549, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 50591, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-10-06T21:46:47.186 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 13507, 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 227548549, 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-10-06T21:46:47.187 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-10-06T21:46:47.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:47 smithi142 ceph-mon[102951]: pgmap v819: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:47.742 INFO:tasks.check_counter:Daemon mds.k mds.exported=13 2023-10-06T21:46:47.742 INFO:tasks.check_counter:Daemon mds.k mds.imported=6 2023-10-06T21:46:47.742 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=1565 2023-10-06T21:46:47.742 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=5444 2023-10-06T21:46:47.742 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2023-10-06T21:46:47.743 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:47.743 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:47.743 DEBUG:tasks.check_counter:Getting stats from c 2023-10-06T21:46:47.743 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-06T21:46:47.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:47 smithi031 ceph-mon[98539]: pgmap v819: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:47.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:47 smithi146 ceph-mon[102809]: pgmap v819: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:48.602 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:46:49.440 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:49 smithi146 ceph-mon[102809]: pgmap v820: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:49.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:49 smithi142 ceph-mon[102951]: pgmap v820: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:49 smithi031 ceph-mon[98539]: pgmap v820: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 3124, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 3079, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 107406902, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1082693, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2023-10-06T21:46:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.493905991, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.108572617, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.273325328, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.087473925, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3078, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.149624757, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000048611 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1759008, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 280624 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1258, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1598, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 93378976, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1087674, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 9, 2023-10-06T21:46:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 7, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.258220749, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.044885755, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.166255092, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.031435670, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1596, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.077180317, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000048358 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 698240 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 2321, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 2323, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 126463416, 2023-10-06T21:46:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 721838, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.325807958, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.071652900, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.183482828, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.052934345, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2321, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.094141431, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000040560 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 10488, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.283697258, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000027049 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 11248, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 310, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 9202, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 1736, 2023-10-06T21:46:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 10805, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 1, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2023-10-06T21:46:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2023-10-06T21:46:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2023-10-06T21:46:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2023-10-06T21:46:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2023-10-06T21:46:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 79633, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 114, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 8, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 327938672, 2023-10-06T21:46:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 327925232, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 79633 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 2495, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 11875, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 9380, 2023-10-06T21:46:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 102, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 192, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 90, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 11248, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 12291, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 1043, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 70964, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 207152 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2023-10-06T21:46:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 1, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 0, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 3280, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 241009, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 83, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 176, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 13368246, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 54616, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2023-10-06T21:46:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 6152, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 6152, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6152, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 6.069910098, 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000986656 2023-10-06T21:46:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6152, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 6153 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 6152, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 6152, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 3282, 2023-10-06T21:46:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 2868, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2023-10-06T21:46:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 132, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 46, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 11, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 1, 2023-10-06T21:46:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2023-10-06T21:46:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 6699, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 326615825, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 6699, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6699, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 326615825, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.406 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 6152, 2023-10-06T21:46:50.407 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 323788522, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6152, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 323788522, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2023-10-06T21:46:50.408 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 6152, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 6152, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6152, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 6152, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.409 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:50.410 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x556c1bedaa00": { 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:50.411 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:50.412 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:50.413 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.413 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:50.413 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-10-06T21:46:50.879 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-06T21:46:50.879 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-10-06T21:46:50.880 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-10-06T21:46:50.880 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-10-06T21:46:50.880 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-10-06T21:46:50.880 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:50.880 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:50.880 DEBUG:tasks.check_counter:Getting stats from f 2023-10-06T21:46:50.880 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-06T21:46:51.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:51 smithi142 ceph-mon[102951]: pgmap v821: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2023-10-06T21:46:51.662 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:46:51.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:51 smithi031 ceph-mon[98539]: pgmap v821: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2023-10-06T21:46:51.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:51 smithi146 ceph-mon[102809]: pgmap v821: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2023-10-06T21:46:53.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:53 smithi146 ceph-mon[102809]: pgmap v822: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-10-06T21:46:53.626 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-10-06T21:46:53.626 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 556848, 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 607990, 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 166185529, 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 412287681, 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 14, 2023-10-06T21:46:53.627 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:53.628 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 45.459017201, 2023-10-06T21:46:53.628 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 19.163065385, 2023-10-06T21:46:53.628 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 31.743810982, 2023-10-06T21:46:53.628 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.801565437, 2023-10-06T21:46:53.628 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 607990, 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 16.643572995, 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000027374 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:53.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1110287, 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.204075463, 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000183 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1763728, 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 309104 2023-10-06T21:46:53.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.631 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:53.631 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 101865, 2023-10-06T21:46:53.631 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 102908, 2023-10-06T21:46:53.631 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 32651278, 2023-10-06T21:46:53.631 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 152947275, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 16, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 8, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 7.913734792, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 3.467319767, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 4.624179775, 2023-10-06T21:46:53.632 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.398078801, 2023-10-06T21:46:53.633 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:53.633 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 102903, 2023-10-06T21:46:53.633 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 3.377166727, 2023-10-06T21:46:53.633 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000032818 2023-10-06T21:46:53.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.634 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:53.634 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 171320, 2023-10-06T21:46:53.634 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.030761397, 2023-10-06T21:46:53.634 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000179 2023-10-06T21:46:53.634 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2160, 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 702944 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 222150, 2023-10-06T21:46:53.635 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 211172, 2023-10-06T21:46:53.636 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 84687882, 2023-10-06T21:46:53.636 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 204502021, 2023-10-06T21:46:53.636 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 16, 2023-10-06T21:46:53.636 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2023-10-06T21:46:53.636 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 19.463401773, 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 9.012747601, 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 11.181385717, 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 1.556891476, 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 211172, 2023-10-06T21:46:53.637 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 7.426475352, 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000035167 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 391164, 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.065739372, 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000168 2023-10-06T21:46:53.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 912, 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 720 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2023-10-06T21:46:53.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 54920, 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 31.460804858, 2023-10-06T21:46:53.640 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000572847 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 27141, 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2.048290236, 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000075468 2023-10-06T21:46:53.641 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 332079, 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 330502, 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 330502, 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 243.558835445, 2023-10-06T21:46:53.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000736936 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 1577, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 59, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 6, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 1850, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 224, 2023-10-06T21:46:53.643 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 1536, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 90, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 1529, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 1373, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 1, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 1, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 6, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 656486, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 595657, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 1571, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 321, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:53.644 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 207, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 1000, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 1, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 25, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 925, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 11, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 270, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 49, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 246, 2023-10-06T21:46:53.645 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 6, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 49, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 49, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 6, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 24703, 2023-10-06T21:46:53.646 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 37853, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 42645, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 12778, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 1222, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 43140 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 3326, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 4307, 2023-10-06T21:46:53.647 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 2245, 2023-10-06T21:46:53.649 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 4401, 2023-10-06T21:46:53.649 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 3763, 2023-10-06T21:46:53.649 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 20, 2023-10-06T21:46:53.649 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 11946, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 11926, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2023-10-06T21:46:53.650 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 25, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 6, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 92014, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 1025, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 1, 2023-10-06T21:46:53.651 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 92013, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 1024, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 130, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 2, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 1, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 129, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 8, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 1, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194352, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 324755395, 2023-10-06T21:46:53.652 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 27391, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 76.657283068, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002798630 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 1559, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 14593, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 13034, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 73, 2023-10-06T21:46:53.653 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 2443, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 2370, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 1850, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 21393, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 19543, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 1, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 12036, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 12035, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 73052, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 198960 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2023-10-06T21:46:53.654 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 396747, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 452, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 332079, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 1026, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 13688, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 11772, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 33.124559335, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002813842 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 254037, 2023-10-06T21:46:53.655 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 89.478440959, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000352226 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 15143, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 10.816719757, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000714304 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.657 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.658 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.659 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.660 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7429, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 5.740845191, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000772761 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 27183, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 64.695589509, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002380001 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.661 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2495, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 7.160098210, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002869778 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.662 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 20, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.115895263, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005794763 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.663 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 5, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.010107261, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002021452 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.664 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 492, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.166037123, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000337473 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2023-10-06T21:46:53.666 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 11926, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 32.250542837, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002704221 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.667 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 35125, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 35125, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1573, 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:53.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:53.669 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:53.670 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:53.671 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 441970, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 145, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:53.672 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 176, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 4879563, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 20904, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2023-10-06T21:46:53.673 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 45664, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 322923035, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 27, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 45664, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 45664, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 135.154994622, 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002959771 2023-10-06T21:46:53.674 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 45664, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 46316 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 45664, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 2328, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 43336, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2023-10-06T21:46:53.675 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 12, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 2279, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 29670, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 1282, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 6, 2023-10-06T21:46:53.676 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 12014, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 49, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 24, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:53.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2023-10-06T21:46:53.678 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 132, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 46, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2023-10-06T21:46:53.679 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 685, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 295 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2023-10-06T21:46:53.680 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 6, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 11926, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.681 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 880859, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 204215109, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 880859, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 880859, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 204215109, 2023-10-06T21:46:53.682 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:53.683 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 45664, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 324139553, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 45664, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 324139553, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 45664, 2023-10-06T21:46:53.685 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 45664, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 45664, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 45664, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:53.686 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 11926, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1216518, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 11926, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2279, 2023-10-06T21:46:53.687 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1216518, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55c14821c6e0": { 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 92014, 2023-10-06T21:46:53.688 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 320561091, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 92014, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 27391, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 320561091, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-10-06T21:46:53.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:53 smithi142 ceph-mon[102951]: pgmap v822: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-10-06T21:46:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:53 smithi031 ceph-mon[98539]: pgmap v822: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-10-06T21:46:54.064 INFO:tasks.check_counter:Daemon mds.f mds.exported=25 2023-10-06T21:46:54.065 INFO:tasks.check_counter:Daemon mds.f mds.imported=11 2023-10-06T21:46:54.065 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=3326 2023-10-06T21:46:54.065 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=4307 2023-10-06T21:46:54.065 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-10-06T21:46:54.065 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:54.066 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:54.066 DEBUG:tasks.check_counter:Getting stats from i 2023-10-06T21:46:54.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-06T21:46:54.799 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:46:55.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:55 smithi142 ceph-mon[102951]: pgmap v823: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-10-06T21:46:55.724 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:55 smithi146 ceph-mon[102809]: pgmap v823: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-10-06T21:46:55.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:55 smithi031 ceph-mon[98539]: pgmap v823: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 281022, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 277183, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 81360037, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 221926458, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 10, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:56.410 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 21.538896863, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 8.939606038, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 13.794267042, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 1.224520663, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 277182, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 8.316520068, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000030003 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 525001, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.092969250, 2023-10-06T21:46:56.411 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000177 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 479871, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 534677, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 146708921, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 454880334, 2023-10-06T21:46:56.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 15, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 42.526040183, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 19.016141596, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 28.466856585, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.771779421, 2023-10-06T21:46:56.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 534676, 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 15.105269109, 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000028251 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 974117, 2023-10-06T21:46:56.414 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.182807301, 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000187 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1765040, 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 287024 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:56.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 104469, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 121742, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 30687140, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 158747125, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 19, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 9, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 9.112572372, 2023-10-06T21:46:56.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 3.767861015, 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 5.099449973, 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.800383366, 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 121739, 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 4.110014058, 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000033760 2023-10-06T21:46:56.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 195676, 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.037496587, 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000191 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T21:46:56.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 699264 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.419 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 42788, 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 27.323116050, 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000638569 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:56.420 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 20824, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.168561962, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000056116 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 322297, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 321307, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 321307, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 263.064621520, 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000818732 2023-10-06T21:46:56.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 990, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 9, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 114, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 12, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 6, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 2758, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 739, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 1968, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 51, 2023-10-06T21:46:56.422 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 1699, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 1291, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 2, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 2, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 6, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 609786, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 570832, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 976, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 295, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 253, 2023-10-06T21:46:56.423 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 2000, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 2, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 41, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 984, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 20, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 1157, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 31, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 246, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 6, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 31, 2023-10-06T21:46:56.424 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 31, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 1769, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 18836, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 28525, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:56.425 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 38128, 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 13757, 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 1087, 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 30342 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 3368, 2023-10-06T21:46:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 3498, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 1807, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 3700, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 3642, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 20, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:56.427 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 8478, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 8458, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2023-10-06T21:46:56.428 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 48, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 18, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2023-10-06T21:46:56.429 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 79633, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 79633, 2023-10-06T21:46:56.430 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 114, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 114, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 8, 2023-10-06T21:46:56.431 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 327938672, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 21344, 2023-10-06T21:46:56.432 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 68.934306504, 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003229680 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 2466, 2023-10-06T21:46:56.433 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 12016, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 9550, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 121, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 2108, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 1987, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 2758, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 17756, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 14998, 2023-10-06T21:46:56.434 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 2, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 8507, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 8505, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 74892, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 198960 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 367332, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 665, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 322297, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 742, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 6604, 2023-10-06T21:46:56.435 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 8257, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 29.981344130, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003631021 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 244188, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 87.378412045, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000357832 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:56.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 12593, 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 9.625343645, 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000764340 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:56.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:56.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.443 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2023-10-06T21:46:56.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6909, 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 5.664980123, 2023-10-06T21:46:56.445 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000819942 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 38848, 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 71.583959662, 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001842667 2023-10-06T21:46:56.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1686, 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 28.398235141, 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.016843555 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:56.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 20, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.120976524, 2023-10-06T21:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.006048826 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 72, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.345782927, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.004802540 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 276, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.158928383, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000575827 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 8458, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 29.806658940, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003524078 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.452 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 1, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 0, 2023-10-06T21:46:56.453 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 36379, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 36379, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1577, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:56.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:56.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:56.456 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 472793, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 156, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:56.457 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 176, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2023-10-06T21:46:56.458 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 7408539, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 29651, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2023-10-06T21:46:56.459 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 35461, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 326080313, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 27, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 35461, 2023-10-06T21:46:56.460 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 35461, 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 119.921237422, 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003381778 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2023-10-06T21:46:56.461 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 35461, 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 36077 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 35461, 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 2018, 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 33443, 2023-10-06T21:46:56.462 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 1978, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 23322, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 1146, 2023-10-06T21:46:56.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 10, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 8561, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2023-10-06T21:46:56.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 32, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:56.465 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:56.466 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2023-10-06T21:46:56.467 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 132, 2023-10-06T21:46:56.468 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 46, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 718, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 18, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 262 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2023-10-06T21:46:56.469 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 1 2023-10-06T21:46:56.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 6, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 8458, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:56.471 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 865358, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 180842084, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 865358, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 865358, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 180842084, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.472 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:56.473 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 35461, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 326947227, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 35461, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 326947227, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:56.474 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 35461, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 35461, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 35461, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 35461, 2023-10-06T21:46:56.475 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:56.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 8458, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 867058, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 8458, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1978, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 867058, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.477 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x559724d4e6e0": { 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 79633, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 323744368, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 79633, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:56.478 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 21344, 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 323744368, 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:56.479 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-10-06T21:46:56.877 INFO:tasks.check_counter:Daemon mds.i mds.exported=41 2023-10-06T21:46:56.877 INFO:tasks.check_counter:Daemon mds.i mds.imported=20 2023-10-06T21:46:56.878 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=3368 2023-10-06T21:46:56.878 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=3498 2023-10-06T21:46:56.878 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-10-06T21:46:56.878 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:56.878 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:56.878 DEBUG:tasks.check_counter:Getting stats from l 2023-10-06T21:46:56.878 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-06T21:46:57.609 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:46:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:57 smithi142 ceph-mon[102951]: pgmap v824: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:57 smithi031 ceph-mon[98539]: pgmap v824: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:57.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:57 smithi146 ceph-mon[102809]: pgmap v824: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:59.177 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-10-06T21:46:59.178 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:46:59.178 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1318, 2023-10-06T21:46:59.178 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1280, 2023-10-06T21:46:59.178 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 77426772, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 525250, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.245992098, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.058600611, 2023-10-06T21:46:59.179 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.156177862, 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.028837215, 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1279, 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.058668259, 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000045870 2023-10-06T21:46:59.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.181 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1733504, 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 285152 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1926, 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 2273, 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 117929919, 2023-10-06T21:46:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1298625, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 9, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 7, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.327292411, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.065423796, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.193827062, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.047514821, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2270, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.098054444, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000043195 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.183 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 700256 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 3590, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 3590, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 128683538, 2023-10-06T21:46:59.184 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1115752, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.494756695, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.105048771, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.253091216, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.100387479, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3589, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.152679777, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000042541 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2023-10-06T21:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 10741, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.292187771, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000027203 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.187 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 13824, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2023-10-06T21:46:59.189 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 13824, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 13824, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 1, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2023-10-06T21:46:59.190 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2023-10-06T21:46:59.191 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:46:59.192 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2023-10-06T21:46:59.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2023-10-06T21:46:59.194 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2023-10-06T21:46:59.195 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2023-10-06T21:46:59.196 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 92014, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 129, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 9, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2023-10-06T21:46:59.197 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194352, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 324755395, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 324741009, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 92014 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.198 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 1603, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 14387, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 12784, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 59, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 112, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 53, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 13824, 2023-10-06T21:46:59.199 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 14783, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 959, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 69308, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 207152 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2023-10-06T21:46:59.200 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2023-10-06T21:46:59.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2023-10-06T21:46:59.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.204 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.210 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.211 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.212 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.213 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.214 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2023-10-06T21:46:59.217 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 5022, 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2023-10-06T21:46:59.218 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:46:59.219 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:46:59.220 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2023-10-06T21:46:59.221 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 241049, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 84, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:46:59.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 8424, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 176, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2023-10-06T21:46:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 13300918, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 63224, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 6282, 2023-10-06T21:46:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 6282, 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6282, 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 6.213907426, 2023-10-06T21:46:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000989160 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6282, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 6283 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 6282, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 6282, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2023-10-06T21:46:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 3285, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 2995, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2023-10-06T21:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2023-10-06T21:46:59.228 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2023-10-06T21:46:59.229 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2023-10-06T21:46:59.230 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 132, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 46, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 11, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 11, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2023-10-06T21:46:59.231 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:46:59.232 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2023-10-06T21:46:59.233 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 6830, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 323394909, 2023-10-06T21:46:59.234 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 6830, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6830, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 323394909, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 6282, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 320572569, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6282, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 320572569, 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:59.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 6282, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 6282, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6282, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 6282, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.238 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2023-10-06T21:46:59.239 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55d28e388a00": { 2023-10-06T21:46:59.240 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2023-10-06T21:46:59.241 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-10-06T21:46:59.242 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-10-06T21:46:59.549 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:46:59 smithi146 ceph-mon[102809]: pgmap v825: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:59.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:46:59 smithi142 ceph-mon[102951]: pgmap v825: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:46:59.685 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:46:59.685 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T21:46:59.697 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T21:46:59.697 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:46:59.697 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:46:59.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:46:59 smithi031 ceph-mon[98539]: pgmap v825: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-10-06T21:46:59.767 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:46:59.767 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:46:59.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:47:00.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 -- 172.21.15.31:0/2342354705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 msgr2=0x7fee7410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2342354705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee7410adf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fee64009990 tx=0x7fee64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 -- 172.21.15.31:0/2342354705 shutdown_connections 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2342354705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 0x7fee7410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2342354705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 0x7fee741023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2342354705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee7410adf0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 -- 172.21.15.31:0/2342354705 >> 172.21.15.31:0/2342354705 conn(0x7fee740fb7f0 msgr2=0x7fee740fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 -- 172.21.15.31:0/2342354705 shutdown_connections 2023-10-06T21:47:00.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.073+0000 7fee7bbff700 1 -- 172.21.15.31:0/2342354705 wait complete. 2023-10-06T21:47:00.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.074+0000 7fee7bbff700 1 Processor -- start 2023-10-06T21:47:00.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.074+0000 7fee7bbff700 1 -- start start 2023-10-06T21:47:00.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 0x7fee741990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 0x7fee7419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e0a0 con 0x7fee74102910 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e210 con 0x7fee74101ff0 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7bbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e3d0 con 0x7fee7410b410 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38914/0 (socket says 172.21.15.31:38914) 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7919a700 1 -- 172.21.15.31:0/2646767538 learned_addr learned my addr 172.21.15.31:0/2646767538 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7999b700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 0x7fee741990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.075+0000 7fee7a19c700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 0x7fee7419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 -- 172.21.15.31:0/2646767538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 msgr2=0x7fee7419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 0x7fee7419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 -- 172.21.15.31:0/2646767538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 msgr2=0x7fee741990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 0x7fee741990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 -- 172.21.15.31:0/2646767538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640095f0 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7919a700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fee64005740 tx=0x7fee64005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6401d070 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee6400bea0 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.076+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7419e650 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.077+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64022d00 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.077+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee741a5ef0 con 0x7fee74102910 2023-10-06T21:47:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.078+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74067cd0 con 0x7fee74102910 2023-10-06T21:47:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.080+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fee64022500 con 0x7fee74102910 2023-10-06T21:47:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.082+0000 7fee6affd700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 0x7fee600a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.083+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fee640ce570 con 0x7fee74102910 2023-10-06T21:47:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.083+0000 7fee7999b700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 0x7fee600a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.084+0000 7fee7999b700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 0x7fee600a3af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee70009c60 tx=0x7fee70009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:00.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.087+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fee64097970 con 0x7fee74102910 2023-10-06T21:47:00.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.311+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee74110c70 con 0x7fee74102910 2023-10-06T21:47:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.313+0000 7fee6affd700 1 -- 172.21.15.31:0/2646767538 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+23280 (secure 0 0 0) 0x7fee64027020 con 0x7fee74102910 2023-10-06T21:47:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:47:00.315 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:46:14.268743+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1070417538":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/126831737":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1598815521":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/751435865":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.315+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 msgr2=0x7fee600a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.315+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 0x7fee600a3af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee70009c60 tx=0x7fee70009260 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 msgr2=0x7fee74199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fee64005740 tx=0x7fee64005770 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 shutdown_connections 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee7410b410 0x7fee7419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee74101ff0 0x7fee741990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fee600a1630 0x7fee600a3af0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 --2- 172.21.15.31:0/2646767538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74102910 0x7fee74199600 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 >> 172.21.15.31:0/2646767538 conn(0x7fee740fb7f0 msgr2=0x7fee7410d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 shutdown_connections 2023-10-06T21:47:00.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.316+0000 7fee7bbff700 1 -- 172.21.15.31:0/2646767538 wait complete. 2023-10-06T21:47:00.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:47:00.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 -- 172.21.15.31:0/2742824257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1061d0 msgr2=0x7f3eac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/2742824257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1061d0 0x7f3eac10dd50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400b0a0 tx=0x7f3ea400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 -- 172.21.15.31:0/2742824257 shutdown_connections 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/2742824257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1061d0 0x7f3eac10dd50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/2742824257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac1010b0 0x7f3eac101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/2742824257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac101a60 0x7f3eac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 -- 172.21.15.31:0/2742824257 >> 172.21.15.31:0/2742824257 conn(0x7f3eac0fc950 msgr2=0x7f3eac0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:00.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 -- 172.21.15.31:0/2742824257 shutdown_connections 2023-10-06T21:47:00.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.543+0000 7f3eb21ff700 1 -- 172.21.15.31:0/2742824257 wait complete. 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 Processor -- start 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 -- start start 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 0x7f3eac0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac073c30 0x7f3eac1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a5850 con 0x7f3eac1010b0 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a59c0 con 0x7f3eac1061d0 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eb21ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a5b80 con 0x7f3eac073c30 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.544+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38924/0 (socket says 172.21.15.31:38924) 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 -- 172.21.15.31:0/4130957481 learned_addr learned my addr 172.21.15.31:0/4130957481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eaaffd700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 0x7f3eac0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eabfff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac073c30 0x7f3eac1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 -- 172.21.15.31:0/4130957481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac073c30 msgr2=0x7f3eac1a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac073c30 0x7f3eac1a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 -- 172.21.15.31:0/4130957481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 msgr2=0x7f3eac0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 0x7f3eac0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 -- 172.21.15.31:0/4130957481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009d00 con 0x7f3eac1010b0 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eaaffd700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 0x7f3eac0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eab7fe700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d980 tx=0x7f3e9c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:00.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00bb60 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e9c007640 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.545+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1a5e60 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.546+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004d60 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.546+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1a6330 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.547+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac067c60 con 0x7f3eac1010b0 2023-10-06T21:47:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.549+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3e9c004020 con 0x7f3eac1010b0 2023-10-06T21:47:00.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.550+0000 7f3ea8ff9700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 0x7f3e940a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:00.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.550+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f3e9c0ccff0 con 0x7f3eac1010b0 2023-10-06T21:47:00.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.550+0000 7f3eaaffd700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 0x7f3e940a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.551+0000 7f3eaaffd700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 0x7f3e940a3b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0749a0 tx=0x7f3ea0009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:00.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.554+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e9c096240 con 0x7f3eac1010b0 2023-10-06T21:47:00.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:00 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2646767538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:00.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:00 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2646767538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.766+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eac101cb0 con 0x7f3eac1010b0 2023-10-06T21:47:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.767+0000 7f3ea8ff9700 1 -- 172.21.15.31:0/4130957481 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3eac101cb0 con 0x7f3eac1010b0 2023-10-06T21:47:00.768 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:47:00.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 msgr2=0x7f3e940a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 0x7f3e940a3b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0749a0 tx=0x7f3ea0009260 comp rx=0 tx=0).stop 2023-10-06T21:47:00.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 msgr2=0x7f3eac0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d980 tx=0x7f3e9c00dd40 comp rx=0 tx=0).stop 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 shutdown_connections 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eac073c30 0x7f3eac1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f3e940a1670 0x7f3e940a3b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eac1010b0 0x7f3eac0731b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 --2- 172.21.15.31:0/4130957481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac1061d0 0x7f3eac0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 >> 172.21.15.31:0/4130957481 conn(0x7f3eac0fc950 msgr2=0x7f3eac0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 shutdown_connections 2023-10-06T21:47:00.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.769+0000 7f3eb21ff700 1 -- 172.21.15.31:0/4130957481 wait complete. 2023-10-06T21:47:00.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:47:00.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:00 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2646767538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:00.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/88436692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c181061d0 msgr2=0x7f5c1810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/88436692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c181061d0 0x7f5c1810dd50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000b0a0 tx=0x7f5c1000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/88436692 shutdown_connections 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/88436692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c181061d0 0x7f5c1810dd50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/88436692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181010b0 0x7f5c18101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/88436692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c18101a60 0x7f5c18105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/88436692 >> 172.21.15.31:0/88436692 conn(0x7f5c180fc950 msgr2=0x7f5c180fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/88436692 shutdown_connections 2023-10-06T21:47:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.995+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/88436692 wait complete. 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.997+0000 7f5c1e7ff700 1 Processor -- start 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.997+0000 7f5c1e7ff700 1 -- start start 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 0x7f5c18197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 0x7f5c1819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1819c410 con 0x7f5c1819b990 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1819c580 con 0x7f5c181061d0 2023-10-06T21:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1e7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1819c740 con 0x7f5c181010b0 2023-10-06T21:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c1cd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 0x7f5c1819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 0x7f5c18197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.31:58346/0 (socket says 172.21.15.31:58346) 2023-10-06T21:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 -- 172.21.15.31:0/2903559001 learned_addr learned my addr 172.21.15.31:0/2903559001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 -- 172.21.15.31:0/2903559001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 msgr2=0x7f5c18197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 0x7f5c18197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 -- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 msgr2=0x7f5c1819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 0x7f5c1819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.998+0000 7f5c17fff700 1 -- 172.21.15.31:0/2903559001 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10009d00 con 0x7f5c181010b0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c1cd9c700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 0x7f5c1819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c177fe700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 0x7f5c18197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c17fff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9a0 tx=0x7f5c0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08009790 con 0x7f5c181010b0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c181a3ca0 con 0x7f5c181010b0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c08010430 con 0x7f5c181010b0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800cea0 con 0x7f5c181010b0 2023-10-06T21:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:00.999+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c181a41d0 con 0x7f5c181010b0 2023-10-06T21:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.000+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c18067c60 con 0x7f5c181010b0 2023-10-06T21:47:01.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.003+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5c080105a0 con 0x7f5c181010b0 2023-10-06T21:47:01.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.004+0000 7f5c157fa700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 0x7f5c000a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.005+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f5c08014070 con 0x7f5c181010b0 2023-10-06T21:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.005+0000 7f5c177fe700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 0x7f5c000a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.005+0000 7f5c177fe700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 0x7f5c000a3b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c18198600 tx=0x7f5c0c009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.007+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5c080964b0 con 0x7f5c181010b0 2023-10-06T21:47:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.218+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c18101c90 con 0x7f5c181010b0 2023-10-06T21:47:01.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.219+0000 7f5c157fa700 1 -- 172.21.15.31:0/2903559001 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c08099ad0 con 0x7f5c181010b0 2023-10-06T21:47:01.221 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 msgr2=0x7f5c000a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 0x7f5c000a3b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c18198600 tx=0x7f5c0c009260 comp rx=0 tx=0).stop 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 msgr2=0x7f5c18196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9a0 tx=0x7f5c0800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 shutdown_connections 2023-10-06T21:47:01.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f5c000a1670 0x7f5c000a3b30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c1819b990 0x7f5c1819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c181010b0 0x7f5c18196ff0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 --2- 172.21.15.31:0/2903559001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c181061d0 0x7f5c18197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 >> 172.21.15.31:0/2903559001 conn(0x7f5c180fc950 msgr2=0x7f5c180fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 shutdown_connections 2023-10-06T21:47:01.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.222+0000 7f5c1e7ff700 1 -- 172.21.15.31:0/2903559001 wait complete. 2023-10-06T21:47:01.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:47:01.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/2484738352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c4101a60 msgr2=0x7fe8c4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/2484738352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c4101a60 0x7fe8c4105aa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009990 tx=0x7fe8b8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/2484738352 shutdown_connections 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/2484738352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 0x7fe8c410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/2484738352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 0x7fe8c4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/2484738352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c4101a60 0x7fe8c4105aa0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/2484738352 >> 172.21.15.31:0/2484738352 conn(0x7fe8c40fc950 msgr2=0x7fe8c40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/2484738352 shutdown_connections 2023-10-06T21:47:01.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.448+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/2484738352 wait complete. 2023-10-06T21:47:01.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.449+0000 7fe8ca5ff700 1 Processor -- start 2023-10-06T21:47:01.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.449+0000 7fe8ca5ff700 1 -- start start 2023-10-06T21:47:01.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 0x7fe8c4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 0x7fe8c4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419e560 con 0x7fe8c419dae0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419e6d0 con 0x7fe8c41061d0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8ca5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419e890 con 0x7fe8c41010b0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8c8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8c8b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38954/0 (socket says 172.21.15.31:38954) 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8c8b9c700 1 -- 172.21.15.31:0/3124248060 learned_addr learned my addr 172.21.15.31:0/3124248060 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.450+0000 7fe8c37fe700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 0x7fe8c4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c3fff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 0x7fe8c4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 -- 172.21.15.31:0/3124248060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 msgr2=0x7fe8c4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 0x7fe8c4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 -- 172.21.15.31:0/3124248060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 msgr2=0x7fe8c4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 0x7fe8c4199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 -- 172.21.15.31:0/3124248060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80095f0 con 0x7fe8c419dae0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c8b9c700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007d70 tx=0x7fe8bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00ad40 con 0x7fe8c419dae0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8bc0128b0 con 0x7fe8c419dae0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c41a5df0 con 0x7fe8c419dae0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0115a0 con 0x7fe8c419dae0 2023-10-06T21:47:01.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.451+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c41a62d0 con 0x7fe8c419dae0 2023-10-06T21:47:01.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.452+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4067c60 con 0x7fe8c419dae0 2023-10-06T21:47:01.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.454+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8bc012a20 con 0x7fe8c419dae0 2023-10-06T21:47:01.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.456+0000 7fe8c17fa700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 0x7fe8ac0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.456+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fe8bc0cd2b0 con 0x7fe8c419dae0 2023-10-06T21:47:01.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.456+0000 7fe8c3fff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 0x7fe8ac0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.457+0000 7fe8c3fff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 0x7fe8ac0a3b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4005950 tx=0x7fe8b40058e0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.460+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe8bc096610 con 0x7fe8c419dae0 2023-10-06T21:47:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:01 smithi031 ceph-mon[98539]: pgmap v826: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-10-06T21:47:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/4130957481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:01 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/2903559001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:01 smithi142 ceph-mon[102951]: pgmap v826: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-10-06T21:47:01.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/4130957481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:01 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/2903559001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.671+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8c4101b00 con 0x7fe8c419dae0 2023-10-06T21:47:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.672+0000 7fe8c17fa700 1 -- 172.21.15.31:0/3124248060 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8bc099c30 con 0x7fe8c419dae0 2023-10-06T21:47:01.674 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:47:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 msgr2=0x7fe8ac0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 0x7fe8ac0a3b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4005950 tx=0x7fe8b40058e0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 msgr2=0x7fe8c419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007d70 tx=0x7fe8bc00a340 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 shutdown_connections 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fe8ac0a1670 0x7fe8ac0a3b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8c419dae0 0x7fe8c419df60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8c41010b0 0x7fe8c4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 --2- 172.21.15.31:0/3124248060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c41061d0 0x7fe8c4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.674+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 >> 172.21.15.31:0/3124248060 conn(0x7fe8c40fc950 msgr2=0x7fe8c40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.675+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 shutdown_connections 2023-10-06T21:47:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.675+0000 7fe8ca5ff700 1 -- 172.21.15.31:0/3124248060 wait complete. 2023-10-06T21:47:01.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-06T21:47:01.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:01 smithi146 ceph-mon[102809]: pgmap v826: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-10-06T21:47:01.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/4130957481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:01 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/2903559001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:01.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 -- 172.21.15.31:0/3427948680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc101a20 msgr2=0x7fc2fc105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 --2- 172.21.15.31:0/3427948680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc101a20 0x7fc2fc105a60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 -- 172.21.15.31:0/3427948680 shutdown_connections 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 --2- 172.21.15.31:0/3427948680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 0x7fc2fc10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 --2- 172.21.15.31:0/3427948680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 0x7fc2fc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 --2- 172.21.15.31:0/3427948680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc101a20 0x7fc2fc105a60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 -- 172.21.15.31:0/3427948680 >> 172.21.15.31:0/3427948680 conn(0x7fc2fc0fc910 msgr2=0x7fc2fc0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 -- 172.21.15.31:0/3427948680 shutdown_connections 2023-10-06T21:47:01.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.899+0000 7fc301fff700 1 -- 172.21.15.31:0/3427948680 wait complete. 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.900+0000 7fc301fff700 1 Processor -- start 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.900+0000 7fc301fff700 1 -- start start 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 0x7fc2fc0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 0x7fc2fc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1a56b0 con 0x7fc2fc073c30 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1a5820 con 0x7fc2fc106190 2023-10-06T21:47:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc301fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1a59e0 con 0x7fc2fc101070 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc2fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc2fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38962/0 (socket says 172.21.15.31:38962) 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc2fbfff700 1 -- 172.21.15.31:0/3513262609 learned_addr learned my addr 172.21.15.31:0/3513262609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc2fb7fe700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 0x7fc2fc0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.901+0000 7fc2faffd700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 0x7fc2fc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 -- 172.21.15.31:0/3513262609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 msgr2=0x7fc2fc0731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 0x7fc2fc0731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 -- 172.21.15.31:0/3513262609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 msgr2=0x7fc2fc0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 0x7fc2fc0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 -- 172.21.15.31:0/3513262609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00095f0 con 0x7fc2fc073c30 2023-10-06T21:47:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2fbfff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4007d70 tx=0x7fc2f400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400ac90 con 0x7fc2fc073c30 2023-10-06T21:47:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.902+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc1a5cc0 con 0x7fc2fc073c30 2023-10-06T21:47:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.903+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2f400adf0 con 0x7fc2fc073c30 2023-10-06T21:47:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.903+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40115e0 con 0x7fc2fc073c30 2023-10-06T21:47:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.903+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1a61a0 con 0x7fc2fc073c30 2023-10-06T21:47:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.904+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc067bf0 con 0x7fc2fc073c30 2023-10-06T21:47:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.905+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc2f4012420 con 0x7fc2fc073c30 2023-10-06T21:47:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.907+0000 7fc2f8ff9700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 0x7fc2e40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.907+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7fc2f40cd1b0 con 0x7fc2fc073c30 2023-10-06T21:47:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.907+0000 7fc2fb7fe700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 0x7fc2e40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:01.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.908+0000 7fc2fb7fe700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 0x7fc2e40a3b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c60 tx=0x7fc2ec009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:01.911+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc2f4096510 con 0x7fc2fc073c30 2023-10-06T21:47:02.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.141+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc2fc101c50 con 0x7fc2fc073c30 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.142+0000 7fc2f8ff9700 1 -- 172.21.15.31:0/3513262609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v132) v1 ==== 81+0+938 (secure 0 0 0) 0x7fc2f4099b30 con 0x7fc2fc073c30 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/964869793 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1070417538 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/126831737 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3876681444 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/484008040 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/964869793 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.144 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2656198457 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/461858081 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3144929895 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/461858081 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/484008040 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1560768070 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1598815521 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2794001065 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/751435865 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1457841953 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 msgr2=0x7fc2e40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 0x7fc2e40a3b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c60 tx=0x7fc2ec009260 comp rx=0 tx=0).stop 2023-10-06T21:47:02.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 msgr2=0x7fc2fc1a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4007d70 tx=0x7fc2f400a340 comp rx=0 tx=0).stop 2023-10-06T21:47:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 shutdown_connections 2023-10-06T21:47:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7fc2e40a1670 0x7fc2e40a3b30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2fc073c30 0x7fc2fc1a5020 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2fc101070 0x7fc2fc0731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 --2- 172.21.15.31:0/3513262609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106190 0x7fc2fc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.144+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 >> 172.21.15.31:0/3513262609 conn(0x7fc2fc0fc910 msgr2=0x7fc2fc0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.145+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 shutdown_connections 2023-10-06T21:47:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.145+0000 7fc301fff700 1 -- 172.21.15.31:0/3513262609 wait complete. 2023-10-06T21:47:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:listed 16 entries 2023-10-06T21:47:02.165 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T21:47:02.165 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:47:02.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:47:02.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 -- 172.21.15.31:0/2989531765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 msgr2=0x7f2ae81014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 --2- 172.21.15.31:0/2989531765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae81014b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009990 tx=0x7f2ad8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 -- 172.21.15.31:0/2989531765 shutdown_connections 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 --2- 172.21.15.31:0/2989531765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 --2- 172.21.15.31:0/2989531765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae81014b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 --2- 172.21.15.31:0/2989531765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae8101a80 0x7f2ae8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 -- 172.21.15.31:0/2989531765 >> 172.21.15.31:0/2989531765 conn(0x7f2ae80fc990 msgr2=0x7f2ae80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 -- 172.21.15.31:0/2989531765 shutdown_connections 2023-10-06T21:47:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.454+0000 7f2aecbff700 1 -- 172.21.15.31:0/2989531765 wait complete. 2023-10-06T21:47:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.455+0000 7f2aecbff700 1 Processor -- start 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 -- start start 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae819dae0 0x7f2ae819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819e560 con 0x7f2ae81010d0 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819e6d0 con 0x7f2ae81061f0 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2aecbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819e890 con 0x7f2ae819dae0 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2ae659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2ae659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38976/0 (socket says 172.21.15.31:38976) 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2ae659c700 1 -- 172.21.15.31:0/463586106 learned_addr learned my addr 172.21.15.31:0/463586106 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.456+0000 7f2ae5d9b700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae6d9d700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae819dae0 0x7f2ae819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 -- 172.21.15.31:0/463586106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae819dae0 msgr2=0x7f2ae819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae819dae0 0x7f2ae819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 -- 172.21.15.31:0/463586106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 msgr2=0x7f2ae8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 -- 172.21.15.31:0/463586106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80095f0 con 0x7f2ae81010d0 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae5d9b700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ae659c700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad801d070 con 0x7f2ae81010d0 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae81a5d90 con 0x7f2ae81010d0 2023-10-06T21:47:02.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ad8003f40 con 0x7f2ae81010d0 2023-10-06T21:47:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8022bc0 con 0x7f2ae81010d0 2023-10-06T21:47:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.457+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae81a61c0 con 0x7f2ae81010d0 2023-10-06T21:47:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.458+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8067cd0 con 0x7f2ae81010d0 2023-10-06T21:47:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.460+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2ad8003ab0 con 0x7f2ae81010d0 2023-10-06T21:47:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.462+0000 7f2ad77fe700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 0x7f2ad00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.462+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f2ad80ceb90 con 0x7f2ae81010d0 2023-10-06T21:47:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.462+0000 7f2ae5d9b700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 0x7f2ad00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.463+0000 7f2ae5d9b700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 0x7f2ad00a3af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2adc005950 tx=0x7f2adc0058e0 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.464+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ad8097e70 con 0x7f2ae81010d0 2023-10-06T21:47:02.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3124248060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:02.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:02 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3513262609' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:02.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:02 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3124248060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:02.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:02 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3513262609' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:02.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.677+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ae8101b50 con 0x7f2ae81010d0 2023-10-06T21:47:02.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.679+0000 7f2ad77fe700 1 -- 172.21.15.31:0/463586106 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+23280 (secure 0 0 0) 0x7f2ad809b490 con 0x7f2ae81010d0 2023-10-06T21:47:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T21:47:02.681 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":132,"fsid":"9b5d7b0a-648c-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:10:28.425727+0000","modified":"2023-10-06T21:46:14.268743+0000","last_up_change":"2023-10-06T21:17:57.773939+0000","last_in_change":"2023-10-06T21:17:37.275263+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:14:04.332397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:18:52.619751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:18:53.803685+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e79b590-506f-4710-912e-2de1d329bdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6803","nonce":811549482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6805","nonce":811549482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6809","nonce":811549482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":811549482},{"type":"v1","addr":"172.21.15.31:6807","nonce":811549482}]},"public_addr":"172.21.15.31:6803/811549482","cluster_addr":"172.21.15.31:6805/811549482","heartbeat_back_addr":"172.21.15.31:6809/811549482","heartbeat_front_addr":"172.21.15.31:6807/811549482","state":["exists","up"]},{"osd":1,"uuid":"0ed69e82-9719-4e1c-9a4c-3c78add6dd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6801","nonce":2309964911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6803","nonce":2309964911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6807","nonce":2309964911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2309964911},{"type":"v1","addr":"172.21.15.142:6805","nonce":2309964911}]},"public_addr":"172.21.15.142:6801/2309964911","cluster_addr":"172.21.15.142:6803/2309964911","heartbeat_back_addr":"172.21.15.142:6807/2309964911","heartbeat_front_addr":"172.21.15.142:6805/2309964911","state":["exists","up"]},{"osd":2,"uuid":"7b2cda23-4d75-4d4a-9cbc-fb219d2ab1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6801","nonce":1261175101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6803","nonce":1261175101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6807","nonce":1261175101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1261175101},{"type":"v1","addr":"172.21.15.146:6805","nonce":1261175101}]},"public_addr":"172.21.15.146:6801/1261175101","cluster_addr":"172.21.15.146:6803/1261175101","heartbeat_back_addr":"172.21.15.146:6807/1261175101","heartbeat_front_addr":"172.21.15.146:6805/1261175101","state":["exists","up"]},{"osd":3,"uuid":"53868b14-df36-4751-89c7-54721dec7417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6811","nonce":3470883528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6813","nonce":3470883528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6817","nonce":3470883528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3470883528},{"type":"v1","addr":"172.21.15.31:6815","nonce":3470883528}]},"public_addr":"172.21.15.31:6811/3470883528","cluster_addr":"172.21.15.31:6813/3470883528","heartbeat_back_addr":"172.21.15.31:6817/3470883528","heartbeat_front_addr":"172.21.15.31:6815/3470883528","state":["exists","up"]},{"osd":4,"uuid":"95502ff0-ad06-45c6-8fe0-de7706d59da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6809","nonce":3887215066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6811","nonce":3887215066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6815","nonce":3887215066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3887215066},{"type":"v1","addr":"172.21.15.142:6813","nonce":3887215066}]},"public_addr":"172.21.15.142:6809/3887215066","cluster_addr":"172.21.15.142:6811/3887215066","heartbeat_back_addr":"172.21.15.142:6815/3887215066","heartbeat_front_addr":"172.21.15.142:6813/3887215066","state":["exists","up"]},{"osd":5,"uuid":"c6d0247c-93bc-4b4c-8855-087be5ba9a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6809","nonce":4108707603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6811","nonce":4108707603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6815","nonce":4108707603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4108707603},{"type":"v1","addr":"172.21.15.146:6813","nonce":4108707603}]},"public_addr":"172.21.15.146:6809/4108707603","cluster_addr":"172.21.15.146:6811/4108707603","heartbeat_back_addr":"172.21.15.146:6815/4108707603","heartbeat_front_addr":"172.21.15.146:6813/4108707603","state":["exists","up"]},{"osd":6,"uuid":"2eb7cdf0-f30d-4a86-bb67-fab477c7ff6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6819","nonce":1906076219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6821","nonce":1906076219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6825","nonce":1906076219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1906076219},{"type":"v1","addr":"172.21.15.31:6823","nonce":1906076219}]},"public_addr":"172.21.15.31:6819/1906076219","cluster_addr":"172.21.15.31:6821/1906076219","heartbeat_back_addr":"172.21.15.31:6825/1906076219","heartbeat_front_addr":"172.21.15.31:6823/1906076219","state":["exists","up"]},{"osd":7,"uuid":"5705f243-b05f-4005-90f0-b62dc8b0ffbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6817","nonce":722083176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6819","nonce":722083176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6823","nonce":722083176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":722083176},{"type":"v1","addr":"172.21.15.142:6821","nonce":722083176}]},"public_addr":"172.21.15.142:6817/722083176","cluster_addr":"172.21.15.142:6819/722083176","heartbeat_back_addr":"172.21.15.142:6823/722083176","heartbeat_front_addr":"172.21.15.142:6821/722083176","state":["exists","up"]},{"osd":8,"uuid":"0cc710f2-d725-47da-9841-e1c75d64bd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6817","nonce":1953163251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6819","nonce":1953163251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6823","nonce":1953163251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1953163251},{"type":"v1","addr":"172.21.15.146:6821","nonce":1953163251}]},"public_addr":"172.21.15.146:6817/1953163251","cluster_addr":"172.21.15.146:6819/1953163251","heartbeat_back_addr":"172.21.15.146:6823/1953163251","heartbeat_front_addr":"172.21.15.146:6821/1953163251","state":["exists","up"]},{"osd":9,"uuid":"50342566-6d06-4bdd-9336-075df7afc340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6827","nonce":3073626060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6829","nonce":3073626060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6833","nonce":3073626060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3073626060},{"type":"v1","addr":"172.21.15.31:6831","nonce":3073626060}]},"public_addr":"172.21.15.31:6827/3073626060","cluster_addr":"172.21.15.31:6829/3073626060","heartbeat_back_addr":"172.21.15.31:6833/3073626060","heartbeat_front_addr":"172.21.15.31:6831/3073626060","state":["exists","up"]},{"osd":10,"uuid":"d033e7e5-3d15-45be-8b09-0fb8f7187f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6825","nonce":1169171647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6827","nonce":1169171647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6831","nonce":1169171647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1169171647},{"type":"v1","addr":"172.21.15.142:6829","nonce":1169171647}]},"public_addr":"172.21.15.142:6825/1169171647","cluster_addr":"172.21.15.142:6827/1169171647","heartbeat_back_addr":"172.21.15.142:6831/1169171647","heartbeat_front_addr":"172.21.15.142:6829/1169171647","state":["exists","up"]},{"osd":11,"uuid":"6a0ed91b-ecfa-499d-bb72-ed9b4b847dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6825","nonce":740640181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6827","nonce":740640181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6831","nonce":740640181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":740640181},{"type":"v1","addr":"172.21.15.146:6829","nonce":740640181}]},"public_addr":"172.21.15.146:6825/740640181","cluster_addr":"172.21.15.146:6827/740640181","heartbeat_back_addr":"172.21.15.146:6831/740640181","heartbeat_front_addr":"172.21.15.146:6829/740640181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:08.930611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:13:34.394085+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:00.507895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:27.226096+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:14:53.548340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:19.356534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:15:45.959848+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:11.815463+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:16:37.527881+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:04.071497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:29.732815+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:17:56.024557+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1070417538":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/126831737":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/3876681444":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6800/964869793":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2656198457":"2023-10-07T21:11:25.716471+0000","172.21.15.31:6801/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/3144929895":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6800/461858081":"2023-10-07T21:10:58.882715+0000","172.21.15.31:6801/484008040":"2023-10-07T21:11:25.716471+0000","172.21.15.31:0/1560768070":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/1598815521":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/2794001065":"2023-10-07T21:10:58.882715+0000","172.21.15.31:0/751435865":"2023-10-07T21:20:56.899536+0000","172.21.15.31:0/1457841953":"2023-10-07T21:11:25.716471+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 msgr2=0x7f2ad00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 0x7f2ad00a3af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2adc005950 tx=0x7f2adc0058e0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 msgr2=0x7f2ae8199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8009510 comp rx=0 tx=0).stop 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 shutdown_connections 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ae819dae0 0x7f2ae819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f2ad00a1630 0x7f2ad00a3af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae81010d0 0x7f2ae8199140 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 --2- 172.21.15.31:0/463586106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ae81061f0 0x7f2ae8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 >> 172.21.15.31:0/463586106 conn(0x7f2ae80fc990 msgr2=0x7f2ae8102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 shutdown_connections 2023-10-06T21:47:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.682+0000 7f2aecbff700 1 -- 172.21.15.31:0/463586106 wait complete. 2023-10-06T21:47:02.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:14:04.332397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:18:52.619751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:18:53.803685+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:47:02.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:47:02.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:02 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3124248060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:02.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:02 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3513262609' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.911+0000 7f879639d700 1 -- 172.21.15.31:0/2621258306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790106dd0 msgr2=0x7f87901071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.911+0000 7f879639d700 1 --2- 172.21.15.31:0/2621258306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790106dd0 0x7f87901071b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f878c0099c0 tx=0x7f878c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 -- 172.21.15.31:0/2621258306 shutdown_connections 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 --2- 172.21.15.31:0/2621258306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f879010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 --2- 172.21.15.31:0/2621258306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790106dd0 0x7f87901071b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 --2- 172.21.15.31:0/2621258306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790068040 0x7f87900684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 -- 172.21.15.31:0/2621258306 >> 172.21.15.31:0/2621258306 conn(0x7f87900769d0 msgr2=0x7f8790076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:02.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 -- 172.21.15.31:0/2621258306 shutdown_connections 2023-10-06T21:47:02.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.912+0000 7f879639d700 1 -- 172.21.15.31:0/2621258306 wait complete. 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 Processor -- start 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 -- start start 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f8790199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790106dd0 0x7f879019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879019ddf0 con 0x7f8790068040 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879019df60 con 0x7f8790106dd0 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.913+0000 7f879639d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879019e120 con 0x7f8790068a00 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38990/0 (socket says 172.21.15.31:38990) 2023-10-06T21:47:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 -- 172.21.15.31:0/356027119 learned_addr learned my addr 172.21.15.31:0/356027119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f8795b9c700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790106dd0 0x7f879019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f8794b9a700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f8790199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 -- 172.21.15.31:0/356027119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 msgr2=0x7f8790199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f8790199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 -- 172.21.15.31:0/356027119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790106dd0 msgr2=0x7f879019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790106dd0 0x7f879019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 -- 172.21.15.31:0/356027119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0095f0 con 0x7f8790068040 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f8794b9a700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f8790199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f879539b700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f878c005910 tx=0x7f878c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c01d070 con 0x7f8790068040 2023-10-06T21:47:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.914+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f878c0054e0 con 0x7f8790068040 2023-10-06T21:47:02.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.915+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c004a90 con 0x7f8790068040 2023-10-06T21:47:02.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.915+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879019e3a0 con 0x7f8790068040 2023-10-06T21:47:02.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.915+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879019e620 con 0x7f8790068040 2023-10-06T21:47:02.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.916+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8790067bf0 con 0x7f8790068040 2023-10-06T21:47:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.919+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f878c00fb80 con 0x7f8790068040 2023-10-06T21:47:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.921+0000 7f87867fc700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 0x7f877c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.921+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f878c0ce1f0 con 0x7f8790068040 2023-10-06T21:47:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.921+0000 7f8794b9a700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 0x7f877c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.921+0000 7f8794b9a700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 0x7f877c0a3b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f879019a420 tx=0x7f8780009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:02.923+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f878c0974d0 con 0x7f8790068040 2023-10-06T21:47:03.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.136+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f879019e620 con 0x7f8790068040 2023-10-06T21:47:03.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.137+0000 7f87867fc700 1 -- 172.21.15.31:0/356027119 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f879019e620 con 0x7f8790068040 2023-10-06T21:47:03.139 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T21:47:03.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 msgr2=0x7f877c0a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 0x7f877c0a3b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f879019a420 tx=0x7f8780009260 comp rx=0 tx=0).stop 2023-10-06T21:47:03.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 msgr2=0x7f8790198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f878c005910 tx=0x7f878c00f690 comp rx=0 tx=0).stop 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 shutdown_connections 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8790106dd0 0x7f879019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f877c0a1690 0x7f877c0a3b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8790068040 0x7f8790198e10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 --2- 172.21.15.31:0/356027119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8790068a00 0x7f8790199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 >> 172.21.15.31:0/356027119 conn(0x7f87900769d0 msgr2=0x7f879010bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.139+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 shutdown_connections 2023-10-06T21:47:03.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.140+0000 7f879639d700 1 -- 172.21.15.31:0/356027119 wait complete. 2023-10-06T21:47:03.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:47:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 -- 172.21.15.31:0/337080295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f58101a60 msgr2=0x7f1f58105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 --2- 172.21.15.31:0/337080295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f58101a60 0x7f1f58105aa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009990 tx=0x7f1f48009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 -- 172.21.15.31:0/337080295 shutdown_connections 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 --2- 172.21.15.31:0/337080295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f581061d0 0x7f1f5810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 --2- 172.21.15.31:0/337080295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 --2- 172.21.15.31:0/337080295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f58101a60 0x7f1f58105aa0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 -- 172.21.15.31:0/337080295 >> 172.21.15.31:0/337080295 conn(0x7f1f580fc950 msgr2=0x7f1f580fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 -- 172.21.15.31:0/337080295 shutdown_connections 2023-10-06T21:47:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.366+0000 7f1f607ff700 1 -- 172.21.15.31:0/337080295 wait complete. 2023-10-06T21:47:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.367+0000 7f1f607ff700 1 Processor -- start 2023-10-06T21:47:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.367+0000 7f1f607ff700 1 -- start start 2023-10-06T21:47:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f5819dbd0 0x7f1f5819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5819e650 con 0x7f1f581061d0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5819e7c0 con 0x7f1f581010b0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f607ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5819e980 con 0x7f1f5819dbd0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51614/0 (socket says 172.21.15.31:51614) 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 -- 172.21.15.31:0/233268153 learned_addr learned my addr 172.21.15.31:0/233268153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5ed9c700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f5819dbd0 0x7f1f5819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5e59b700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 -- 172.21.15.31:0/233268153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f5819dbd0 msgr2=0x7f1f5819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f5819dbd0 0x7f1f5819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 -- 172.21.15.31:0/233268153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 msgr2=0x7f1f58199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5dd9a700 1 -- 172.21.15.31:0/233268153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f480095f0 con 0x7f1f581061d0 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.368+0000 7f1f5e59b700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f5dd9a700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800c010 tx=0x7f1f48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4801d070 con 0x7f1f581061d0 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f48022470 con 0x7f1f581061d0 2023-10-06T21:47:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f581a5e80 con 0x7f1f581061d0 2023-10-06T21:47:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017650 con 0x7f1f581061d0 2023-10-06T21:47:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.369+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f581a6240 con 0x7f1f581061d0 2023-10-06T21:47:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.370+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f58067c60 con 0x7f1f581061d0 2023-10-06T21:47:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.372+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1f480225e0 con 0x7f1f581061d0 2023-10-06T21:47:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.373+0000 7f1f4f7fe700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 0x7f1f440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.373+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f1f480ce660 con 0x7f1f581061d0 2023-10-06T21:47:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.374+0000 7f1f5e59b700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 0x7f1f440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.374+0000 7f1f5e59b700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 0x7f1f440a3ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005d90 tx=0x7f1f54005d00 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.377+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f48097940 con 0x7f1f581061d0 2023-10-06T21:47:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:03 smithi031 ceph-mon[98539]: pgmap v827: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-10-06T21:47:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:03 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/463586106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:03.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:03 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/356027119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.589+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f58101b00 con 0x7f1f581061d0 2023-10-06T21:47:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.589+0000 7f1f4f7fe700 1 -- 172.21.15.31:0/233268153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f4809af60 con 0x7f1f581061d0 2023-10-06T21:47:03.591 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T21:47:03.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.591+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 msgr2=0x7f1f440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.591+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 0x7f1f440a3ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005d90 tx=0x7f1f54005d00 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.591+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 msgr2=0x7f1f58199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.591+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800c010 tx=0x7f1f48004730 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 shutdown_connections 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f5819dbd0 0x7f1f5819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f581010b0 0x7f1f58199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f1f440a15f0 0x7f1f440a3ab0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 --2- 172.21.15.31:0/233268153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f581061d0 0x7f1f58199770 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 >> 172.21.15.31:0/233268153 conn(0x7f1f580fc950 msgr2=0x7f1f580fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:03.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 shutdown_connections 2023-10-06T21:47:03.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.592+0000 7f1f607ff700 1 -- 172.21.15.31:0/233268153 wait complete. 2023-10-06T21:47:03.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:47:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:03 smithi142 ceph-mon[102951]: pgmap v827: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-10-06T21:47:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:03 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/463586106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:03.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:03 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/356027119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:03.796 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:03 smithi146 ceph-mon[102809]: pgmap v827: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-10-06T21:47:03.796 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:03 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/463586106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:47:03.796 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:03 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/356027119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:47:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 -- 172.21.15.31:0/506623304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f00732e0 msgr2=0x7f63f00736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 --2- 172.21.15.31:0/506623304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f00732e0 0x7f63f00736c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009990 tx=0x7f63e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 -- 172.21.15.31:0/506623304 shutdown_connections 2023-10-06T21:47:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 --2- 172.21.15.31:0/506623304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f010d680 0x7f63f010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 --2- 172.21.15.31:0/506623304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f00732e0 0x7f63f00736c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 --2- 172.21.15.31:0/506623304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f0073c00 0x7f63f010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.816+0000 7f63f5fff700 1 -- 172.21.15.31:0/506623304 >> 172.21.15.31:0/506623304 conn(0x7f63f00fc950 msgr2=0x7f63f00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.817+0000 7f63f5fff700 1 -- 172.21.15.31:0/506623304 shutdown_connections 2023-10-06T21:47:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.817+0000 7f63f5fff700 1 -- 172.21.15.31:0/506623304 wait complete. 2023-10-06T21:47:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.817+0000 7f63f5fff700 1 Processor -- start 2023-10-06T21:47:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 -- start start 2023-10-06T21:47:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f00732e0 0x7f63f01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f0073c00 0x7f63f0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f019e0c0 con 0x7f63f010d680 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f019e230 con 0x7f63f0073c00 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63f5fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f019e3f0 con 0x7f63f00732e0 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51628/0 (socket says 172.21.15.31:51628) 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63effff700 1 -- 172.21.15.31:0/584024833 learned_addr learned my addr 172.21.15.31:0/584024833 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.818+0000 7f63ef7fe700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f00732e0 0x7f63f01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63eeffd700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f0073c00 0x7f63f0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 -- 172.21.15.31:0/584024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f00732e0 msgr2=0x7f63f01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f00732e0 0x7f63f01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 -- 172.21.15.31:0/584024833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f0073c00 msgr2=0x7f63f0199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f0073c00 0x7f63f0199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 -- 172.21.15.31:0/584024833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e00095f0 con 0x7f63f010d680 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63effff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007ee0 tx=0x7f63e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8017070 con 0x7f63f010d680 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e800f040 con 0x7f63f010d680 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8005710 con 0x7f63f010d680 2023-10-06T21:47:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f019e6d0 con 0x7f63f010d680 2023-10-06T21:47:03.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.819+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f01a5f10 con 0x7f63f010d680 2023-10-06T21:47:03.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.820+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0108890 con 0x7f63f010d680 2023-10-06T21:47:03.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.823+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63e8005870 con 0x7f63f010d680 2023-10-06T21:47:03.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.824+0000 7f63ecff9700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 0x7f63d80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.824+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f63e80ccea0 con 0x7f63f010d680 2023-10-06T21:47:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.825+0000 7f63ef7fe700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 0x7f63d80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.825+0000 7f63ef7fe700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 0x7f63d80a3b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006010 tx=0x7f63e000b260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:03.827+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f63e8096150 con 0x7f63f010d680 2023-10-06T21:47:04.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.042+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63f019a430 con 0x7f63f010d680 2023-10-06T21:47:04.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.042+0000 7f63ecff9700 1 -- 172.21.15.31:0/584024833 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63e8099770 con 0x7f63f010d680 2023-10-06T21:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 msgr2=0x7f63d80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 0x7f63d80a3b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006010 tx=0x7f63e000b260 comp rx=0 tx=0).stop 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 msgr2=0x7f63f019da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007ee0 tx=0x7f63e800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 shutdown_connections 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f63d80a1670 0x7f63d80a3b30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f010d680 0x7f63f019da80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63f00732e0 0x7f63f01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 --2- 172.21.15.31:0/584024833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63f0073c00 0x7f63f0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 >> 172.21.15.31:0/584024833 conn(0x7f63f00fc950 msgr2=0x7f63f00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 shutdown_connections 2023-10-06T21:47:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.045+0000 7f63f5fff700 1 -- 172.21.15.31:0/584024833 wait complete. 2023-10-06T21:47:04.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-06T21:47:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 -- 172.21.15.31:0/4122328346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4101a60 msgr2=0x7f6af4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 --2- 172.21.15.31:0/4122328346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4101a60 0x7f6af4105aa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009990 tx=0x7f6ae8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 -- 172.21.15.31:0/4122328346 shutdown_connections 2023-10-06T21:47:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 --2- 172.21.15.31:0/4122328346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af41061d0 0x7f6af410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 --2- 172.21.15.31:0/4122328346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 --2- 172.21.15.31:0/4122328346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af4101a60 0x7f6af4105aa0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 -- 172.21.15.31:0/4122328346 >> 172.21.15.31:0/4122328346 conn(0x7f6af40fc930 msgr2=0x7f6af40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 -- 172.21.15.31:0/4122328346 shutdown_connections 2023-10-06T21:47:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.270+0000 7f6af97ff700 1 -- 172.21.15.31:0/4122328346 wait complete. 2023-10-06T21:47:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 Processor -- start 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 -- start start 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af40731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af4073c30 0x7f6af41a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41a57a0 con 0x7f6af41061d0 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41a5910 con 0x7f6af41010b0 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.271+0000 7f6af97ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41a5ad0 con 0x7f6af4073c30 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51654/0 (socket says 172.21.15.31:51654) 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 -- 172.21.15.31:0/3782555950 learned_addr learned my addr 172.21.15.31:0/3782555950 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af37fe700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af4073c30 0x7f6af41a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af40731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 -- 172.21.15.31:0/3782555950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af4073c30 msgr2=0x7f6af41a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af4073c30 0x7f6af41a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 -- 172.21.15.31:0/3782555950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 msgr2=0x7f6af40731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af40731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 -- 172.21.15.31:0/3782555950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80095f0 con 0x7f6af41061d0 2023-10-06T21:47:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af40731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:47:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.272+0000 7f6af27fc700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005910 tx=0x7f6ae8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.273+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae801d070 con 0x7f6af41061d0 2023-10-06T21:47:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.273+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ae8022470 con 0x7f6af41061d0 2023-10-06T21:47:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.273+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8017650 con 0x7f6af41061d0 2023-10-06T21:47:04.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.273+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af41a5d50 con 0x7f6af41061d0 2023-10-06T21:47:04.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.273+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af41a6120 con 0x7f6af41061d0 2023-10-06T21:47:04.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.274+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af4067c60 con 0x7f6af41061d0 2023-10-06T21:47:04.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.276+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6ae80225e0 con 0x7f6af41061d0 2023-10-06T21:47:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.278+0000 7f6adbfff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 0x7f6adc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:47:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.279+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6ae80ce5c0 con 0x7f6af41061d0 2023-10-06T21:47:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.279+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 0x7f6adc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:47:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.279+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 0x7f6adc0a3b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009cb0 tx=0x7f6ae4009260 comp rx=0 tx=0).ready entity=mgr.15570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:47:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.282+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6ae80978a0 con 0x7f6af41061d0 2023-10-06T21:47:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.520+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6af40746e0 con 0x7f6af41061d0 2023-10-06T21:47:04.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.520+0000 7f6adbfff700 1 -- 172.21.15.31:0/3782555950 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v132) v1 ==== 81+0+938 (secure 0 0 0) 0x7f6ae80273c0 con 0x7f6af41061d0 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/964869793 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1070417538 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/126831737 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3876681444 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/484008040 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/964869793 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2656198457 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/461858081 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3144929895 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/461858081 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/484008040 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1560768070 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1598815521 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2794001065 2023-10-07T21:10:58.882715+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/751435865 2023-10-07T21:20:56.899536+0000 2023-10-06T21:47:04.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1457841953 2023-10-07T21:11:25.716471+0000 2023-10-06T21:47:04.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 msgr2=0x7f6adc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 0x7f6adc0a3b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009cb0 tx=0x7f6ae4009260 comp rx=0 tx=0).stop 2023-10-06T21:47:04.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 msgr2=0x7f6af40736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005910 tx=0x7f6ae8005740 comp rx=0 tx=0).stop 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 shutdown_connections 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af4073c30 0x7f6af41a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af41010b0 0x7f6af40731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:6800/2497223303,v1:172.21.15.31:6801/2497223303] conn(0x7f6adc0a1670 0x7f6adc0a3b30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 --2- 172.21.15.31:0/3782555950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af41061d0 0x7f6af40736f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 >> 172.21.15.31:0/3782555950 conn(0x7f6af40fc930 msgr2=0x7f6af40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 shutdown_connections 2023-10-06T21:47:04.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T21:47:04.523+0000 7f6af97ff700 1 -- 172.21.15.31:0/3782555950 wait complete. 2023-10-06T21:47:04.527 INFO:teuthology.orchestra.run.smithi031.stderr:listed 16 entries 2023-10-06T21:47:04.544 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2023-10-06T21:47:04.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:47:04.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T21:47:04.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:04 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/233268153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:04.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:04 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/584024833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:04.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[160790]: fuse finished with error 0 and tester_r 0 2023-10-06T21:47:04.636 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T21:47:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:04 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/233268153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:04.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:04 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/584024833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:04.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:04 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/233268153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:47:04.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:04 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/584024833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:47:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:05 smithi031 ceph-mon[98539]: pgmap v828: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 27 op/s 2023-10-06T21:47:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:05 smithi031 ceph-mon[98539]: from='client.? 172.21.15.31:0/3782555950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:05.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:05 smithi146 ceph-mon[102809]: pgmap v828: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 27 op/s 2023-10-06T21:47:05.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:05 smithi146 ceph-mon[102809]: from='client.? 172.21.15.31:0/3782555950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:05.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:05 smithi142 ceph-mon[102951]: pgmap v828: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 27 op/s 2023-10-06T21:47:05.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:05 smithi142 ceph-mon[102951]: from='client.? 172.21.15.31:0/3782555950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:47:06.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:06 smithi031 ceph-mon[98539]: pgmap v829: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:06.798 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:06 smithi146 ceph-mon[102809]: pgmap v829: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:06.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:06 smithi142 ceph-mon[102951]: pgmap v829: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:08.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:08 smithi031 ceph-mon[98539]: pgmap v830: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:08.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:08 smithi146 ceph-mon[102809]: pgmap v830: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:08.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:08 smithi142 ceph-mon[102951]: pgmap v830: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-10-06T21:47:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:09 smithi031 ceph-mon[98539]: osdmap e133: 12 total, 12 up, 12 in 2023-10-06T21:47:09.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:09 smithi146 ceph-mon[102809]: osdmap e133: 12 total, 12 up, 12 in 2023-10-06T21:47:09.901 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:09 smithi142 ceph-mon[102951]: osdmap e133: 12 total, 12 up, 12 in 2023-10-06T21:47:10.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:47:10.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:47:10.738 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T21:47:10.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:47:10.814 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T21:47:10.886 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T21:47:10.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:47:10.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T21:47:10.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:47:10.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T21:47:11.084 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-06T21:47:11.095 INFO:tasks.cephadm:Teardown begin 2023-10-06T21:47:11.096 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephadm.py", line 1756, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-06T21:47:11.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:47:11.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:47:11.268 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:47:11.372 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-06T21:47:11.372 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:47:11.416 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:47:11.461 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:47:11.527 INFO:tasks.cephadm:Stopping all daemons... 2023-10-06T21:47:11.527 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-06T21:47:11.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T21:47:11.548 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:11 smithi146 ceph-mon[102809]: pgmap v832: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 30 op/s 2023-10-06T21:47:11.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:11 smithi031 ceph-mon[98539]: pgmap v832: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 30 op/s 2023-10-06T21:47:11.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:11 smithi142 ceph-mon[102951]: pgmap v832: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 30 op/s 2023-10-06T21:47:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:11 smithi031 systemd[1]: Stopping Ceph mon.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:47:12.052+0000 7f51d8f28700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:12.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a[98514]: 2023-10-06T21:47:12.052+0000 7f51d8f28700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-06T21:47:13.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Oct 06 21:47:12 smithi031 bash[182002]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T21:47:13.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-06T21:47:13.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:13.150 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-06T21:47:13.150 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-06T21:47:13.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.b 2023-10-06T21:47:13.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:13 smithi142 systemd[1]: Stopping Ceph mon.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:14.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-b[102927]: 2023-10-06T21:47:13.772+0000 7f669aa68700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:14.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-b[102927]: 2023-10-06T21:47:13.772+0000 7f669aa68700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-06T21:47:14.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 06 21:47:14 smithi142 bash[149507]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-b 2023-10-06T21:47:14.715 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-06T21:47:14.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:14.813 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-06T21:47:14.813 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-06T21:47:14.813 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.c 2023-10-06T21:47:15.298 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:14 smithi146 systemd[1]: Stopping Ceph mon.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:15.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-c[102785]: 2023-10-06T21:47:15.368+0000 7f96ad4be700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:15.799 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-c[102785]: 2023-10-06T21:47:15.368+0000 7f96ad4be700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-06T21:47:16.299 INFO:journalctl@ceph.mon.c.smithi146.stdout:Oct 06 21:47:15 smithi146 bash[150638]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mon-c 2023-10-06T21:47:16.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-06T21:47:16.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:16.502 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-06T21:47:16.502 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-06T21:47:16.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T21:47:16.761 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:47:16 smithi031 systemd[1]: Stopping Ceph mgr.x for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:17.299 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:47:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:47:17.019+0000 7fc3f056c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:17.299 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:47:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x[98786]: 2023-10-06T21:47:17.019+0000 7fc3f056c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-06T21:47:17.760 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Oct 06 21:47:17 smithi031 bash[186133]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-x 2023-10-06T21:47:17.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-06T21:47:18.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:18.064 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-06T21:47:18.064 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-06T21:47:18.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.y 2023-10-06T21:47:18.401 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:47:18 smithi142 systemd[1]: Stopping Ceph mgr.y for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:19.106 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:47:18 smithi142 bash[149656]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-y 2023-10-06T21:47:19.336 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-06T21:47:19.396 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:47:19 smithi142 systemd[1]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-06T21:47:19.396 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:47:19 smithi142 systemd[1]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.y.service: Failed with result 'exit-code'. 2023-10-06T21:47:19.397 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Oct 06 21:47:19 smithi142 systemd[1]: Stopped Ceph mgr.y for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:47:19.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:19.422 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-06T21:47:19.422 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-06T21:47:19.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.z 2023-10-06T21:47:19.799 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:47:19 smithi146 systemd[1]: Stopping Ceph mgr.z for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:20.548 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Oct 06 21:47:20 smithi146 bash[150789]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mgr-z 2023-10-06T21:47:20.711 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-06T21:47:20.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:20.799 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-06T21:47:20.799 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-06T21:47:20.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.0 2023-10-06T21:47:21.220 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:47:20 smithi031 systemd[1]: Stopping Ceph osd.0 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:21.510 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:47:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:47:21.309+0000 7f85ddcf4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:21.510 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:47:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:47:21.309+0000 7f85ddcf4700 -1 osd.0 133 *** Got signal Terminated *** 2023-10-06T21:47:21.510 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:47:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0[113768]: 2023-10-06T21:47:21.309+0000 7f85ddcf4700 -1 osd.0 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:47:27.205 INFO:journalctl@ceph.osd.0.smithi031.stdout:Oct 06 21:47:27 smithi031 bash[186279]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-0 2023-10-06T21:47:29.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-06T21:47:29.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:29.157 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-06T21:47:29.157 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-06T21:47:29.158 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.1 2023-10-06T21:47:29.652 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:47:29 smithi142 systemd[1]: Stopping Ceph osd.1 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:30.153 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:47:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1[107995]: 2023-10-06T21:47:29.661+0000 7fdcadbd6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:30.165 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:47:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1[107995]: 2023-10-06T21:47:29.661+0000 7fdcadbd6700 -1 osd.1 133 *** Got signal Terminated *** 2023-10-06T21:47:30.165 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:47:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1[107995]: 2023-10-06T21:47:29.661+0000 7fdcadbd6700 -1 osd.1 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:47:35.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Oct 06 21:47:35 smithi142 bash[149809]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-1 2023-10-06T21:47:37.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-06T21:47:37.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:37.597 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-06T21:47:37.597 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-06T21:47:37.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.2 2023-10-06T21:47:38.050 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:47:37 smithi146 systemd[1]: Stopping Ceph osd.2 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:38.549 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:47:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2[108686]: 2023-10-06T21:47:38.108+0000 7fbdb1cfe700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:38.550 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:47:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2[108686]: 2023-10-06T21:47:38.108+0000 7fbdb1cfe700 -1 osd.2 133 *** Got signal Terminated *** 2023-10-06T21:47:38.550 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:47:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2[108686]: 2023-10-06T21:47:38.108+0000 7fbdb1cfe700 -1 osd.2 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:47:44.298 INFO:journalctl@ceph.osd.2.smithi146.stdout:Oct 06 21:47:43 smithi146 bash[150950]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-2 2023-10-06T21:47:45.829 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-06T21:47:45.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:45.921 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-06T21:47:45.921 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-06T21:47:45.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.3 2023-10-06T21:47:46.260 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:46 smithi031 systemd[1]: Stopping Ceph osd.3 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:46.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:46 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:46.438+0000 7ff1e0247700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:46.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:46 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:46.438+0000 7ff1e0247700 -1 osd.3 133 *** Got signal Terminated *** 2023-10-06T21:47:46.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:46 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:46.438+0000 7ff1e0247700 -1 osd.3 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:47:47.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:47 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:47.262+0000 7ff1da839700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.917285+0000 front 2023-10-06T21:47:25.917340+0000 (oldest deadline 2023-10-06T21:47:47.017514+0000) 2023-10-06T21:47:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:47.426+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:48.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:48 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:48.268+0000 7ff1da839700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.917285+0000 front 2023-10-06T21:47:25.917340+0000 (oldest deadline 2023-10-06T21:47:47.017514+0000) 2023-10-06T21:47:48.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:48.406+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:49.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:49.260+0000 7ff1da839700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.917285+0000 front 2023-10-06T21:47:25.917340+0000 (oldest deadline 2023-10-06T21:47:47.017514+0000) 2023-10-06T21:47:49.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:49 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:49.374+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:49.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:49.361+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:50.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:50.320+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:50.549 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:50.127+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:50.549 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:50.163+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:50.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:50.259+0000 7ff1da839700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.917285+0000 front 2023-10-06T21:47:25.917340+0000 (oldest deadline 2023-10-06T21:47:47.017514+0000) 2023-10-06T21:47:50.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:50.354+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:50.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:50 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:50.476+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:51.353 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:51.082+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:51.353 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:51.180+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:51.760 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3[122335]: 2023-10-06T21:47:51.279+0000 7ff1da839700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.917285+0000 front 2023-10-06T21:47:25.917340+0000 (oldest deadline 2023-10-06T21:47:47.017514+0000) 2023-10-06T21:47:51.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:51.435+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:51.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:51 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:51.364+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:51.350+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:52.333 INFO:journalctl@ceph.osd.3.smithi031.stdout:Oct 06 21:47:52 smithi031 bash[186550]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-3 2023-10-06T21:47:52.360 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:52.101+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:52.360 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:52.207+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:52.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:51 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:51.919+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:52.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:52.392+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:52.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:52 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:52.405+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:52.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:52.356+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:53.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:52 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:52.954+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:53.457 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:53.386+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:53.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:53.361+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:53.549 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:53.121+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:53.549 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:53.171+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:53.746 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:53 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:53.454+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:54.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-06T21:47:54.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:47:54.170 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-06T21:47:54.170 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-06T21:47:54.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.4 2023-10-06T21:47:54.237 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:53 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:53.973+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:54.365 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:54.079+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:54.366 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:54.135+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:54.651 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:47:54 smithi142 systemd[1]: Stopping Ceph osd.4 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:47:54.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:54.344+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:54.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:54 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:54.465+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:54.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:54.362+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:54.956 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:47:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:47:54.687+0000 7f53065d5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:47:54.956 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:47:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:47:54.687+0000 7f53065d5700 -1 osd.4 133 *** Got signal Terminated *** 2023-10-06T21:47:54.956 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:47:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:47:54.687+0000 7f53065d5700 -1 osd.4 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:47:55.360 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:55.144+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:55.361 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:55.084+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:55.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:54 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:54.954+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:55.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:55.370+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:55.761 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:55 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:55.444+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:55.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:55.358+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:56.350 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:56.081+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:56.350 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:56.170+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:56.350 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:56.170+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:47:56.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:56 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:56.001+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:56.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:56 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:56.001+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:47:56.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:56 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:56.383+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:56.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:56 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:56.478+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:56.347+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:56.347+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:47:57.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:57 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:57.005+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:57.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:57 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:57.005+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:47:57.549 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:57.089+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:57.549 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:57.130+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:57.549 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:57.130+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:47:57.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:57.320+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:57.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:57.320+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:47:57.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:57.399+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:57.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:57.399+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:47:57.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:57.492+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:57.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:57 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:57.492+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:47:58.364 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:58.146+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:58.364 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:58.146+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:47:58.365 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:58.071+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:58.365 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:58.071+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:47:58.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:58 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:58.021+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:58.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:58 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:58.021+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:47:58.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:58 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:58.407+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:58.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:58 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:58.407+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:47:58.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:58 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:58.511+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:58.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:58 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:58.511+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:47:58.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:58.361+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:58.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:58 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:58.361+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:47:59.151 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:47:58 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4[116508]: 2023-10-06T21:47:58.889+0000 7f5301bc9700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.693585+0000 front 2023-10-06T21:47:32.693611+0000 (oldest deadline 2023-10-06T21:47:57.993874+0000) 2023-10-06T21:47:59.152 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:59 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:59.035+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:47:59.152 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:47:59 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:47:59.035+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:47:59.370 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:59.152+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:47:59.370 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:47:59.152+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:47:59.370 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:59.102+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:47:59.370 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:47:59.102+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:47:59.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:59 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:59.438+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:47:59.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:47:59 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:47:59.438+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:47:59.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:59 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:59.526+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:47:59.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:47:59 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:47:59.526+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:47:59.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:59.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:47:59.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:47:59 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:47:59.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:00.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:00 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:00.287+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:00.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:00 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:00.062+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:00.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:00 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:00.062+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:00.416 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:00.159+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:00.417 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:00.159+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:00.417 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:00.099+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:00.417 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:00.099+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:00.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:00.456+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:00.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:00.456+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:00.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:00.539+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:00.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:00 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:00.539+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:00.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:00.413+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:00.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:00 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:00.413+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:00.901 INFO:journalctl@ceph.osd.4.smithi142.stdout:Oct 06 21:48:00 smithi142 bash[150076]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-4 2023-10-06T21:48:01.304 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:01 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:01.043+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:01.304 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:01 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:01.043+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:01.383 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:01.112+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:01.383 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:01.112+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:01.383 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:01.050+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:01.384 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:01.050+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:01.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:01 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:01.301+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:01.759 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:01.445+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:01.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:01.445+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:01.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:01.517+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:01.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:01.517+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:01.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:01.380+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:01.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:01 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:01.380+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:02.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:02 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:02.086+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:02.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:02 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:02.086+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:02.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:02.024+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:02.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:02.024+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:02.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:02.067+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:02.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:02.067+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:02.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-06T21:48:02.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:02.448 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-06T21:48:02.448 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-06T21:48:02.448 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.5 2023-10-06T21:48:02.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:02 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:02.342+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:02.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:02 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:02.478+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:02.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:02 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:02.478+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:02.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:02 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:02.482+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:02.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:02 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:02.482+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:02.798 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 systemd[1]: Stopping Ceph osd.5 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:02.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:02.349+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:02.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:02.349+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:03.086+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:03.086+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:02.952+0000 7fe263316700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:02.952+0000 7fe263316700 -1 osd.5 133 *** Got signal Terminated *** 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:02 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:02.952+0000 7fe263316700 -1 osd.5 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:03.016+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:03.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:03.016+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:03.402 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:03 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:03.308+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:03.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:03 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:03.063+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:03.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:03 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:03.063+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:03.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:03.380+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:03.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:03.380+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:04.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:03.508+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:04.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:03.508+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:04.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:03.515+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:04.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:03 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:03.515+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:04.294 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:04 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:04.016+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:04.294 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:04 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:04.016+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:04.298 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:03.973+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:04.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:03 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:03.973+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:04.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:04.065+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:04.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:04.065+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:04.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:04 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:04.292+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:04.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:04.337+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:04.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:04.337+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:05.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:04 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:04.544+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:05.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:04 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:04.544+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:05.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:04 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:04.512+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:05.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:04 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:04.512+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:05.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:04.952+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:05.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:04 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:04.952+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:05.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:05.039+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:05.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:05.039+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:05.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:05 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:05.252+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:05.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:05 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:05.032+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:05.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:05 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:05.032+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:05.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:05 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:05.032+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:05.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:05.574+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:05.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:05.574+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:05.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:05.488+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:05.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:05 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:05.488+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:05.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:05.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:05.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:05.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:06.298 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:05.922+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:06.299 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:05 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:05.922+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:06.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:06.002+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:06.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:06.002+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:06.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:06 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:06.284+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:06.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:06 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:06.284+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:06.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:06 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:06.050+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:06.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:06 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:06.050+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:06.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:06 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:06.050+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:06.593+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:06.593+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:06.593+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:06.459+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:06.459+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:06.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:06.459+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:06.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:06.332+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:06.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:06.332+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:07.295 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:07 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:07.009+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:07.295 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:07 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:07.009+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:07.295 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:07 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:07.009+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:07.295 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:06.940+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.146604+0000 front 2023-10-06T21:47:24.146656+0000 (oldest deadline 2023-10-06T21:47:49.446462+0000) 2023-10-06T21:48:07.295 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:06 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5[117227]: 2023-10-06T21:48:06.940+0000 7fe25d107700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.948299+0000 front 2023-10-06T21:47:32.948497+0000 (oldest deadline 2023-10-06T21:47:57.648400+0000) 2023-10-06T21:48:07.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:07 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:07.276+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:07.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:07 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:07.276+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:07.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:07 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:07.068+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:07.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:07 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:07.068+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:07.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:07 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:07.068+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:07.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:07 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:07.291+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:07.549 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:07 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:07.292+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:07.629+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:07.629+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:07.629+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:07.447+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:07.447+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:07.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:07 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:07.447+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:08.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:08.029+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:08.370 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:08.029+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:08.370 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:08.029+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:08.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:08 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:08.084+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:08.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:08 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:08.084+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:08.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:08 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:08.084+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:08.402 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:08 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:08.320+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:08.402 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:08 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:08.320+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:08.693 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:08.337+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:08.694 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:08.337+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:08.694 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:08 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:08.337+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:08.663+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:08.663+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:08.663+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:08.432+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:08.432+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:08.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:08 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:08.432+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:09.029 INFO:journalctl@ceph.osd.5.smithi146.stdout:Oct 06 21:48:08 smithi146 bash[151221]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-5 2023-10-06T21:48:09.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:09.026+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:09.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:09.026+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:09.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:09.026+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:09.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:09 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:09.287+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:09.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:09 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:09.287+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:09.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:09 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:09.039+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:09.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:09 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:09.039+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:09.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:09 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:09.039+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:09.691 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:09.399+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:09.691 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:09.399+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:09.691 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:09.399+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:09.750 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:09.382+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:09.751 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:09.382+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:09.751 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:09 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:09.382+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:10.009 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:09.689+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:10.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:09.689+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:10.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:09 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:09.689+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:10.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:10.036+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:10.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:10.036+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:10.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:10.036+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:10.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:10.083+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:10.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:10.083+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:10.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:10.083+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:10.402 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:10.287+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:10.402 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:10 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:10.287+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:10.646 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:10.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:10.646 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:10.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:10.646 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:10 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:10.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:10.670 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-06T21:48:10.734 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:10.377+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:10.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:10.377+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:10.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:10.377+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:10.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:10.759 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-06T21:48:10.759 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-06T21:48:10.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.6 2023-10-06T21:48:11.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:10.733+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:11.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:10.733+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:11.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:10 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:10.733+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:11.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:10 smithi031 systemd[1]: Stopping Ceph osd.6 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:11.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.038+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:11.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.038+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:11.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.038+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:11.339 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:11 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:11.081+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:11.339 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:11 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:11.081+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:11.339 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:11 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:11.081+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:11.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:11 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:11.336+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:11.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:11 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:11.336+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:11.728 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.293+0000 7f9cef46c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:11.728 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.293+0000 7f9cef46c700 -1 osd.6 133 *** Got signal Terminated *** 2023-10-06T21:48:11.729 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.293+0000 7f9cef46c700 -1 osd.6 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:11.729 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.726+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:11.729 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:11.414+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:11.729 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:11.414+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:11.729 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:11.414+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:11.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:11.341+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:11.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:11.341+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:11.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:11.341+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:12.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.726+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:12.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:11 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:11.726+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:12.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.992+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:12.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.992+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:12.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:11 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:11.992+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:12.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:12.291+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:12.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:12.291+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:12.401 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:12.119+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:12.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:12.119+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:12.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:12 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:12.119+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:12.738 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:12.378+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:12.738 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:12.378+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:12.739 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:12.378+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:12.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:12.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:12.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:12.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:12.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:12.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:13.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:12.736+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:13.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:12.736+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:13.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:12.736+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:13.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:12.964+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:13.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:12.964+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:13.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:12.964+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:13.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:12 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:12.964+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:13.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:13.340+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:13.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:13.340+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:13.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:13.102+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:13.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:13.102+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:13.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:13 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:13.102+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:13.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:13.329+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:13.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:13.329+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:13.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:13.329+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:13.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:13.375+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:13.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:13.375+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:13.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:13.375+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:14.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:13.771+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:14.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:13.771+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:14.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:13 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:13.771+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:14.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:13.923+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:14.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:13.923+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:14.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:13.923+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:14.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:13 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:13.923+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:14.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:14.292+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:14.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:14.292+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:14.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:14.292+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:14.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:14.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:14.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:14.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:14.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:14 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:14.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:14.752 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:14.366+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:14.752 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:14.366+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:14.752 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:14.366+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:14.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:14.366+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:14.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:14.366+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:14.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:14.366+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:15.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:14.750+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:15.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:14.750+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:15.010 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:14 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:14.750+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:15.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:14.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:15.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:14.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:15.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:14.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:15.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:14 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:14.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:15.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:15.309+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:15.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:15.309+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:15.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:15.309+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:15.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:15.199+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:15.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:15.199+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:15.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:15.199+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:15.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:15 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:15.199+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:15.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:15.330+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:15.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:15.330+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:15.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:15.330+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:15.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:15.333+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:15.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:15.333+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:15.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:15.333+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:16.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:15.775+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:26.251400+0000 front 2023-10-06T21:47:26.251436+0000 (oldest deadline 2023-10-06T21:47:49.751594+0000) 2023-10-06T21:48:16.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:15.775+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:31.453609+0000 front 2023-10-06T21:47:31.453358+0000 (oldest deadline 2023-10-06T21:47:57.353485+0000) 2023-10-06T21:48:16.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:15.775+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.856680+0000 front 2023-10-06T21:47:41.856605+0000 (oldest deadline 2023-10-06T21:48:05.956245+0000) 2023-10-06T21:48:16.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6[130743]: 2023-10-06T21:48:15.775+0000 7f9cea25f700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.457211+0000 front 2023-10-06T21:47:49.457273+0000 (oldest deadline 2023-10-06T21:48:14.757370+0000) 2023-10-06T21:48:16.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:15.871+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:16.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:15.871+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:16.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:15.871+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:16.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:15 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:15.871+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:16.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:16.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:16.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:16.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:16.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:16.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:16.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:16.152+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:16.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:16.279+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:16.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:16.279+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:16.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:16 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:16.279+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:16.759 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:16 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:16.351+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:16.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:16 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:16.351+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:16.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:16 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:16.351+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:16.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:16.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:16.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:16.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:16.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:16.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:17.260 INFO:journalctl@ceph.osd.6.smithi031.stdout:Oct 06 21:48:17 smithi031 bash[186816]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-6 2023-10-06T21:48:17.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:16.854+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:17.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:16.854+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:17.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:16.854+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:17.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:16 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:16.854+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:17.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:17.300+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:17.300+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:17.300+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:17.158+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:17.158+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:17.158+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:17.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:17 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:17.158+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:17.759 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:17.396+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:17.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:17.396+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:17.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:17 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:17.396+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:17.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:17.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:17.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:17.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:17.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:17.376+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:18.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:17.897+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:18.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:17.897+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:18.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:17.897+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:18.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:17 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:17.897+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:18.308+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:18.308+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:18.308+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:18.184+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:18.184+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:18.184+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:18.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:18 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:18.184+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:18.761 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:18.438+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:18.761 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:18.438+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:18.763 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:18.438+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:18.763 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:18 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:18.438+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:18.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:18.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:18.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:18.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:18.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:18.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:18.909 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:18.899+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:18.909 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:18.899+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:19.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-06T21:48:19.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:19.120 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-06T21:48:19.120 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-06T21:48:19.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.7 2023-10-06T21:48:19.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:18.899+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:19.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:18 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:18.899+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:19.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 systemd[1]: Stopping Ceph osd.7 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:19.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.357+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:19.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.357+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:19.401 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.357+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:19.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:19.143+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:19.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:19.143+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:19.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:19.143+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:19.402 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:19.143+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:19.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:19.455+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:19.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:19.455+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:19.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:19.455+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:19.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:19 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:19.455+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:19.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:19.381+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:19.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:19.381+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:19.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:19.381+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:19.901 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.629+0000 7f9330427700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:19.901 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.629+0000 7f9330427700 -1 osd.7 133 *** Got signal Terminated *** 2023-10-06T21:48:19.901 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:19 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:19.629+0000 7f9330427700 -1 osd.7 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:20.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:19.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:20.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:19.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:20.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:19.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:20.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:19 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:19.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:20.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:20.404+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:20.404+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:20.404+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:20.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:20.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:20.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:20.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:20.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:20.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:20.423+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:20.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:20.423+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:20.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:20.423+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:20.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:20 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:20.423+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:20.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:20.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:20.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:20.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:20.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:20.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:20.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:20.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:21.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:20.884+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:21.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:20.884+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:21.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:20.884+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:21.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:20 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:20.884+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:21.448+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:21.448+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:21.448+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:21.214+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:21.214+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:21.214+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:21.654 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:21 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:21.214+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:21.759 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:21.376+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:21.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:21.376+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:21.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:21.376+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:21.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:21 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:21.376+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:21.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:21.372+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:21.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:21.372+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:21.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:21.372+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:21.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:21.372+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:22.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:21.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:22.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:21.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:22.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:21.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:22.300 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:21.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:22.300 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:21 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:21.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:22.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:22.430+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:22.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:22.430+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:22.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:22.430+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:22.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:22.238+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:22.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:22.238+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:22.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:22.238+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:22.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:22 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:22.238+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:22.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:22.418+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:22.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:22.418+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:22.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:22.418+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:22.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:22 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:22.418+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:22.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:22.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:22.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:22.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:22.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:22.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:22.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:22.390+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:23.298 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:22.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:23.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:22.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:23.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:22.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:23.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:22.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:23.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:22 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:22.886+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:23.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:23.412+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:23.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:23.412+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:23.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:23.412+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:23.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:23.232+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:23.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:23.232+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:23.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:23.232+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:23.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:23 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:23.232+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:23.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:23.451+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:23.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:23.451+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:23.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:23.451+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:23.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:23.451+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:23.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:23 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:23.451+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:23.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:23.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:23.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:23.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:23.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:23.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:23.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:23.370+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:24.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:23.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:24.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:23.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:24.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:23.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:24.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:23.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:24.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:23 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:23.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:24.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:24.445+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:34.302397+0000 front 2023-10-06T21:47:34.302474+0000 (oldest deadline 2023-10-06T21:47:59.602520+0000) 2023-10-06T21:48:24.651 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:24.445+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:39.603156+0000 front 2023-10-06T21:47:39.603140+0000 (oldest deadline 2023-10-06T21:48:05.503208+0000) 2023-10-06T21:48:24.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:24.445+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:48.404748+0000 front 2023-10-06T21:47:48.404443+0000 (oldest deadline 2023-10-06T21:48:13.704396+0000) 2023-10-06T21:48:24.652 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7[125095]: 2023-10-06T21:48:24.445+0000 7f932aa19700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.406673+0000 front 2023-10-06T21:47:59.406700+0000 (oldest deadline 2023-10-06T21:48:24.106630+0000) 2023-10-06T21:48:24.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:24.225+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:24.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:24.225+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:24.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:24.225+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:24.653 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:24 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:24.225+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:24.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:24.417+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:24.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:24.417+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:24.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:24.417+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:24.761 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:24.417+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:24.761 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:24 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:24.417+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:24.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:24.406+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:24.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:24.406+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:24.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:24.406+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:24.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:24.406+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:25.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:24.808+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:25.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:24.808+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:25.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:24.808+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:25.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:24.808+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:25.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:24 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:24.808+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:25.451 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:25 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:25.191+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:25.452 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:25 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:25.191+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:25.452 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:25 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:25.191+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:25.452 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:25 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:25.191+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:25.452 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:25 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:25.191+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:25.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:25.407+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:25.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:25.407+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:25.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:25.407+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:25.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:25.407+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:25.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:25 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:25.407+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:25.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:25.442+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:25.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:25.442+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:25.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:25.442+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:25.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:25.442+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:25.901 INFO:journalctl@ceph.osd.7.smithi142.stdout:Oct 06 21:48:25 smithi142 bash[150356]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-7 2023-10-06T21:48:26.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:25.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:26.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:25.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:26.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:25.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:26.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:25.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:26.300 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:25 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:25.851+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:26.429 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:26.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:26.429 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:26.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:26.429 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:26.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:26.430 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:26.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:26.430 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:26 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:26.164+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:26.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:26.401+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:26.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:26.401+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:26.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:26.401+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:26.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:26.401+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:26.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:26 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:26.401+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:26.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:26.398+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:26.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:26.398+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:26.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:26.398+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:26.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:26.398+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:27.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:26.846+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:27.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:26.846+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:27.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:26.846+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:27.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:26.846+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:27.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:26 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:26.846+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:27.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-06T21:48:27.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:27.398 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-06T21:48:27.398 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-06T21:48:27.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.8 2023-10-06T21:48:27.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:27.198+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:27.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:27.198+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:27.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:27.198+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:27.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:27.198+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:27.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:27.198+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:27.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:27 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:27.439+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:27.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:27 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:27.439+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:27.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:27 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:27.439+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:27.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:27 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:27.439+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:27.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:27 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:27.439+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:27.806 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 systemd[1]: Stopping Ceph osd.8 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:27.806 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:27.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:27.807 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:27.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:27.807 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:27.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:27.807 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:27.367+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.895+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.895+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.895+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.895+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.895+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.915+0000 7fb42d8c8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:28.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.915+0000 7fb42d8c8700 -1 osd.8 133 *** Got signal Terminated *** 2023-10-06T21:48:28.300 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:27 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:27.915+0000 7fb42d8c8700 -1 osd.8 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:28.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:28.241+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:28.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:28.241+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:28.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:28.241+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:28.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:28.241+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:28.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:28 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:28.241+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:28.759 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:28.463+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:28.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:28.463+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:28.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:28.463+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:28.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:28.463+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:28.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:28 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:28.463+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:28.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:28.412+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:28.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:28.412+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:28.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:28.412+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:28.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:28.412+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:29.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:28.904+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:29.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:28.904+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:29.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:28.904+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:29.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:28.904+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:29.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:28 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:28.904+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:29.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:29.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:29.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:29.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:29.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:29.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:29.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:29.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:29.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:29 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:29.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:29.759 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:29 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:29.504+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:29.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:29 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:29.504+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:29.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:29 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:29.504+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:29.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:29 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:29.504+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:29.760 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:29 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:29.504+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:29.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:29.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:29.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:29.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:29.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:29.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:29.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:29.377+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:30.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:29.903+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:30.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:29.903+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:30.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:29.903+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:30.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:29.903+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:30.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:29 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:29.903+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:30.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:30.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:30.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:30.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:30.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:30.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:30 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:30.264+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:30.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:30.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:30.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:30.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:30.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:30.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:30.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:30.369+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:31.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:30 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:30.533+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:31.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:30 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:30.533+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:31.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:30 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:30.533+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:31.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:30 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:30.533+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:31.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:30 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:30.533+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:31.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:30.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:31.301 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:30.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:31.301 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:30.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:31.301 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:30.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:31.301 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:30 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:30.870+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:31.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:31.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:31.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:31.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:31.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:31.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:31 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:31.260+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:31.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:31.407+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:31.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:31.407+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:31.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:31.407+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:31.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:31.407+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:32.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:31 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:31.567+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:32.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:31 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:31.567+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:32.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:31 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:31.567+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:32.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:31 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:31.567+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:32.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:31 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:31.567+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:32.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:31.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:32.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:31.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:32.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:31.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:32.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:31.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:32.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:31 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:31.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:32.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:32.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:32.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:32.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:32.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:32.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:32 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:32.303+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:32.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:32.445+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:32.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:32.445+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:32.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:32.445+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:32.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:32.445+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:32.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:32.445+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:33.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:32 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:32.519+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:33.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:32.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:24.157452+0000 front 2023-10-06T21:47:24.157400+0000 (oldest deadline 2023-10-06T21:47:49.457306+0000) 2023-10-06T21:48:33.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:32.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.358502+0000 front 2023-10-06T21:47:32.358716+0000 (oldest deadline 2023-10-06T21:47:55.258560+0000) 2023-10-06T21:48:33.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:32.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.260551+0000 front 2023-10-06T21:47:42.260582+0000 (oldest deadline 2023-10-06T21:48:06.960841+0000) 2023-10-06T21:48:33.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:32.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.961630+0000 front 2023-10-06T21:47:46.961741+0000 (oldest deadline 2023-10-06T21:48:12.861833+0000) 2023-10-06T21:48:33.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:32 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8[125807]: 2023-10-06T21:48:32.911+0000 7fb425eb6700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:59.066105+0000 front 2023-10-06T21:47:59.066263+0000 (oldest deadline 2023-10-06T21:48:21.366241+0000) 2023-10-06T21:48:33.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:33.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:33.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:33.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:33.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:33.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:33 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:33.267+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:33.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:33 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:33.495+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:33.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:33 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:33.495+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:33.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:33 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:33.495+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:33.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:33 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:33.495+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:33.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:33 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:33.495+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:34.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:33 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:33.548+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:34.299 INFO:journalctl@ceph.osd.8.smithi146.stdout:Oct 06 21:48:33 smithi146 bash[151497]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-8 2023-10-06T21:48:34.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:34.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:34.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:34.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:34.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:34.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:34.301+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:34.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:34 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:34.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:34.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:34 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:34.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:34.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:34 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:34.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:34.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:34 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:34.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:34.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:34 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:34.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:35.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:34 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:34.590+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:35.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:35.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:35.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:35.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:35.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:35.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:35 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:35.329+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:35.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:35 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:35.532+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:35.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:35 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:35.532+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:35.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:35 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:35.532+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:35.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:35 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:35.532+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:35.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:35 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:35.532+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:35.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:36.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:35 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:35.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:36.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:36.035 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-06T21:48:36.035 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-06T21:48:36.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.9 2023-10-06T21:48:36.510 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 systemd[1]: Stopping Ceph osd.9 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:36.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:36.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:36.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:36.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:36.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:36.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:36 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:36.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:36.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:36.510+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:36.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:36.510+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:36.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:36.510+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:36.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:36.510+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:36.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:36 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:36.510+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.563+0000 7f24653f4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.563+0000 7f24653f4700 -1 osd.9 133 *** Got signal Terminated *** 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.563+0000 7f24653f4700 -1 osd.9 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:37.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:37.011 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:36 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:36.604+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:37.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:37.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:37.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:37.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:37.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:37.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:37 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:37.382+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:37.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:37.483+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:37.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:37.483+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:37.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:37.483+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:37.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:37.483+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:37.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:37 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:37.483+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:38.009 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:38.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:37 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:37.597+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.876881+0000 front 2023-10-06T21:48:14.876897+0000 (oldest deadline 2023-10-06T21:48:37.177053+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.349+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.349+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.349+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.349+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.349+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:38.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:38 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:38.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:38.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:38.521+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:38.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:38.521+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:38.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:38.521+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:38.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:38.521+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:38.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:38 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:38.521+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:39.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:38 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:38.563+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.876881+0000 front 2023-10-06T21:48:14.876897+0000 (oldest deadline 2023-10-06T21:48:37.177053+0000) 2023-10-06T21:48:39.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:39.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:39.333+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:39.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:39 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:39.479+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:39.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:39 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:39.479+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:39.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:39 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:39.479+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:39.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:39 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:39.479+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:39.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:39 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:39.479+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:40.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:40.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:40.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:40.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:40.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:40.011 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:40.011 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:39 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:39.522+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.876881+0000 front 2023-10-06T21:48:14.876897+0000 (oldest deadline 2023-10-06T21:48:37.177053+0000) 2023-10-06T21:48:40.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:40.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:40.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:40.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:40.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:40.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:40.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:40 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:40.294+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:40.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:40 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:40.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:40.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:40 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:40.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:40.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:40 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:40.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:40.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:40 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:40.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:40.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:40 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:40.499+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.867154+0000 front 2023-10-06T21:47:23.867187+0000 (oldest deadline 2023-10-06T21:47:49.167415+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.068720+0000 front 2023-10-06T21:47:32.068874+0000 (oldest deadline 2023-10-06T21:47:57.368790+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:40.870596+0000 front 2023-10-06T21:47:40.870728+0000 (oldest deadline 2023-10-06T21:48:06.170812+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:46.672615+0000 front 2023-10-06T21:47:46.672640+0000 (oldest deadline 2023-10-06T21:48:17.873784+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:57.874485+0000 front 2023-10-06T21:47:57.874670+0000 (oldest deadline 2023-10-06T21:48:22.574446+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:07.875752+0000 front 2023-10-06T21:48:07.875848+0000 (oldest deadline 2023-10-06T21:48:31.975852+0000) 2023-10-06T21:48:41.010 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:40 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9[139313]: 2023-10-06T21:48:40.558+0000 7f24619ea700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.876881+0000 front 2023-10-06T21:48:14.876897+0000 (oldest deadline 2023-10-06T21:48:37.177053+0000) 2023-10-06T21:48:41.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:41.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:41.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:41.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:41.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:41.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:41.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:41 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:41.286+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:41.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:41 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:41.455+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:42.449 INFO:journalctl@ceph.osd.9.smithi031.stdout:Oct 06 21:48:42 smithi031 bash[187093]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-9 2023-10-06T21:48:42.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.289+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:42.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.289+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:42.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.289+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:42.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.289+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:42.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:42.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:42.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:42 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:42.290+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:42.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:42 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:42.409+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:43.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:43.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:43.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:43.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:43.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:43.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:43.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:43 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:43.317+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:43.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:43.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:43.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:43.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:43.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:43.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:43.387+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:44.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-06T21:48:44.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:44.275 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-06T21:48:44.275 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-06T21:48:44.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.10 2023-10-06T21:48:44.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.272+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:44.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 systemd[1]: Stopping Ceph osd.10 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:44.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:44 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:44.418+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:45.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.783+0000 7f5b1efe6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:45.151 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.783+0000 7f5b1efe6700 -1 osd.10 133 *** Got signal Terminated *** 2023-10-06T21:48:45.152 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:44 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:44.783+0000 7f5b1efe6700 -1 osd.10 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:45.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:45.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:45.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:45.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:45.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:45.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:45.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:45 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:45.293+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:45.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:45.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:45.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:45.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:45.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:45.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:45 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:45.457+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:46.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:46.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:46.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:46.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:46.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:46.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:46.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:46 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:46.309+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:46.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:46.474+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:47.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.324+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:47.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.324+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:47.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.324+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:47.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.324+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:47.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.325+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:47.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.325+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:47.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:47 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:47.325+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:47.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:47 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:47.493+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:48.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:48.651 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:48 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:48.327+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:48.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:48.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:48 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:48.498+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:25.928075+0000 front 2023-10-06T21:47:25.928139+0000 (oldest deadline 2023-10-06T21:47:51.828025+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.329264+0000 front 2023-10-06T21:47:32.329292+0000 (oldest deadline 2023-10-06T21:47:55.829417+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:42.132017+0000 front 2023-10-06T21:47:42.131894+0000 (oldest deadline 2023-10-06T21:48:04.432033+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-10-06T21:47:49.634038+0000 front 2023-10-06T21:47:49.633917+0000 (oldest deadline 2023-10-06T21:48:14.333890+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:58.435015+0000 front 2023-10-06T21:47:58.435038+0000 (oldest deadline 2023-10-06T21:48:24.835864+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.537464+0000 front 2023-10-06T21:48:06.537414+0000 (oldest deadline 2023-10-06T21:48:29.937980+0000) 2023-10-06T21:48:49.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:49 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10[133765]: 2023-10-06T21:48:49.350+0000 7f5b1addb700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:14.539743+0000 front 2023-10-06T21:48:14.539772+0000 (oldest deadline 2023-10-06T21:48:38.039612+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:49.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:49 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:49.524+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:50.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:50.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:50.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:50.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:50.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:50.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:50.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:50 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:50.485+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:50.901 INFO:journalctl@ceph.osd.10.smithi142.stdout:Oct 06 21:48:50 smithi142 bash[150623]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-10 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:51.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:51.469+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:52.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-06T21:48:52.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:48:52.573 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-06T21:48:52.573 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-06T21:48:52.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.11 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:52.506+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:52.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:52 smithi146 systemd[1]: Stopping Ceph osd.11 for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:48:53.473 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.087+0000 7fa2b60cd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:48:53.474 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.087+0000 7fa2b60cd700 -1 osd.11 133 *** Got signal Terminated *** 2023-10-06T21:48:53.474 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.087+0000 7fa2b60cd700 -1 osd.11 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:48:53.474 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:53.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:53.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:53.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:53.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:53.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:53.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:53 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:53.471+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:54.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:54 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:54.501+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:55.798 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:55.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:55 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:55.459+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6820 osd.8 since back 2023-10-06T21:48:31.002877+0000 front 2023-10-06T21:48:31.002892+0000 (oldest deadline 2023-10-06T21:48:55.102883+0000) 2023-10-06T21:48:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:56.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:56.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:56.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:56.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:56 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:56.507+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6820 osd.8 since back 2023-10-06T21:48:31.002877+0000 front 2023-10-06T21:48:31.002892+0000 (oldest deadline 2023-10-06T21:48:55.102883+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-10-06T21:47:23.388888+0000 front 2023-10-06T21:47:23.388936+0000 (oldest deadline 2023-10-06T21:47:46.888741+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-10-06T21:47:32.691010+0000 front 2023-10-06T21:47:32.690809+0000 (oldest deadline 2023-10-06T21:47:55.590773+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2023-10-06T21:47:41.993491+0000 front 2023-10-06T21:47:41.993757+0000 (oldest deadline 2023-10-06T21:48:07.892885+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-10-06T21:47:54.295125+0000 front 2023-10-06T21:47:54.295143+0000 (oldest deadline 2023-10-06T21:48:20.195222+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2023-10-06T21:48:06.498179+0000 front 2023-10-06T21:48:06.498209+0000 (oldest deadline 2023-10-06T21:48:32.398421+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-10-06T21:48:15.899519+0000 front 2023-10-06T21:48:15.899634+0000 (oldest deadline 2023-10-06T21:48:41.199702+0000) 2023-10-06T21:48:57.799 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-10-06T21:48:23.401892+0000 front 2023-10-06T21:48:23.401955+0000 (oldest deadline 2023-10-06T21:48:45.701636+0000) 2023-10-06T21:48:57.800 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11[133753]: 2023-10-06T21:48:57.478+0000 7fa2b06bf700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.146:6820 osd.8 since back 2023-10-06T21:48:31.002877+0000 front 2023-10-06T21:48:31.002892+0000 (oldest deadline 2023-10-06T21:48:55.102883+0000) 2023-10-06T21:48:59.049 INFO:journalctl@ceph.osd.11.smithi146.stdout:Oct 06 21:48:58 smithi146 bash[151764]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-osd-11 2023-10-06T21:49:00.642 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-06T21:49:00.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:00.735 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-06T21:49:00.736 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-06T21:49:00.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.a 2023-10-06T21:49:01.254 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:49:00 smithi031 systemd[1]: Stopping Ceph mds.a for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:01.509 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:49:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-a[147392]: 2023-10-06T21:49:01.252+0000 7f998102b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:01.510 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:49:01 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-a[147392]: 2023-10-06T21:49:01.252+0000 7f998102b700 -1 mds.a *** got signal Terminated *** 2023-10-06T21:49:06.010 INFO:journalctl@ceph.mds.a.smithi031.stdout:Oct 06 21:49:05 smithi031 bash[187363]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-a 2023-10-06T21:49:06.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-06T21:49:06.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:06.294 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-06T21:49:06.294 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-06T21:49:06.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.d 2023-10-06T21:49:06.509 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:49:06 smithi031 systemd[1]: Stopping Ceph mds.d for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:07.260 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:49:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-d[146303]: 2023-10-06T21:49:06.793+0000 7fa291bef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:07.260 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:49:06 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-d[146303]: 2023-10-06T21:49:06.793+0000 7fa291bef700 -1 mds.d *** got signal Terminated *** 2023-10-06T21:49:11.509 INFO:journalctl@ceph.mds.d.smithi031.stdout:Oct 06 21:49:11 smithi031 bash[187511]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-d 2023-10-06T21:49:11.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-06T21:49:11.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:11.632 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-06T21:49:11.632 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-06T21:49:11.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.g 2023-10-06T21:49:12.010 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:49:11 smithi031 systemd[1]: Stopping Ceph mds.g for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:12.510 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:49:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-g[146962]: 2023-10-06T21:49:12.133+0000 7fb151260700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:12.510 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:49:12 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-g[146962]: 2023-10-06T21:49:12.133+0000 7fb151260700 -1 mds.g *** got signal Terminated *** 2023-10-06T21:49:14.401 INFO:journalctl@ceph.mds.g.smithi031.stdout:Oct 06 21:49:14 smithi031 bash[187656]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-g 2023-10-06T21:49:14.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-06T21:49:14.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:14.798 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-06T21:49:14.799 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-06T21:49:14.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.j 2023-10-06T21:49:15.009 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:14 smithi031 systemd[1]: Stopping Ceph mds.j for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:15.760 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-j[145917]: 2023-10-06T21:49:15.299+0000 7f541255f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:15.760 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:15 smithi031 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-j[145917]: 2023-10-06T21:49:15.299+0000 7f541255f700 -1 mds.j *** got signal Terminated *** 2023-10-06T21:49:19.760 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:19 smithi031 bash[187796]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-j 2023-10-06T21:49:20.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-06T21:49:20.072 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:19 smithi031 systemd[1]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.j.service: Succeeded. 2023-10-06T21:49:20.072 INFO:journalctl@ceph.mds.j.smithi031.stdout:Oct 06 21:49:20 smithi031 systemd[1]: Stopped Ceph mds.j for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7. 2023-10-06T21:49:20.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:20.097 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-06T21:49:20.097 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-06T21:49:20.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.b 2023-10-06T21:49:20.606 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:49:20 smithi142 systemd[1]: Stopping Ceph mds.b for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:20.901 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:49:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b[140597]: 2023-10-06T21:49:20.602+0000 7fa27f690700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:20.901 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:49:20 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b[140597]: 2023-10-06T21:49:20.602+0000 7fa27f690700 -1 mds.b *** got signal Terminated *** 2023-10-06T21:49:27.151 INFO:journalctl@ceph.mds.b.smithi142.stdout:Oct 06 21:49:26 smithi142 bash[150901]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-b 2023-10-06T21:49:27.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-06T21:49:27.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:27.270 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-06T21:49:27.270 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-06T21:49:27.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.e 2023-10-06T21:49:27.651 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:49:27 smithi142 systemd[1]: Stopping Ceph mds.e for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:28.151 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:49:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-e[139912]: 2023-10-06T21:49:27.750+0000 7f0f479e5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:28.151 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:49:27 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-e[139912]: 2023-10-06T21:49:27.750+0000 7f0f479e5700 -1 mds.e *** got signal Terminated *** 2023-10-06T21:49:33.651 INFO:journalctl@ceph.mds.e.smithi142.stdout:Oct 06 21:49:33 smithi142 bash[151044]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-e 2023-10-06T21:49:33.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-06T21:49:33.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:33.797 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-06T21:49:33.797 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-06T21:49:33.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.h 2023-10-06T21:49:34.151 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:49:33 smithi142 systemd[1]: Stopping Ceph mds.h for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:34.651 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:49:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: 2023-10-06T21:49:34.310+0000 7fe158a85700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:34.651 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:49:34 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h[140254]: 2023-10-06T21:49:34.310+0000 7fe158a85700 -1 mds.h *** got signal Terminated *** 2023-10-06T21:49:38.901 INFO:journalctl@ceph.mds.h.smithi142.stdout:Oct 06 21:49:38 smithi142 bash[151190]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-h 2023-10-06T21:49:39.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-06T21:49:39.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:39.249 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-06T21:49:39.250 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-06T21:49:39.250 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.k 2023-10-06T21:49:39.651 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:49:39 smithi142 systemd[1]: Stopping Ceph mds.k for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:40.151 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:49:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-k[139571]: 2023-10-06T21:49:39.724+0000 7f5292fb6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:40.151 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:49:39 smithi142 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-k[139571]: 2023-10-06T21:49:39.724+0000 7f5292fb6700 -1 mds.k *** got signal Terminated *** 2023-10-06T21:49:42.901 INFO:journalctl@ceph.mds.k.smithi142.stdout:Oct 06 21:49:42 smithi142 bash[151330]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-k 2023-10-06T21:49:43.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-06T21:49:43.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:43.153 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-06T21:49:43.154 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-06T21:49:43.154 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.c 2023-10-06T21:49:43.549 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:49:43 smithi146 systemd[1]: Stopping Ceph mds.c for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:44.049 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:49:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-c[139835]: 2023-10-06T21:49:43.679+0000 7fc3cd018700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:44.049 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:49:43 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-c[139835]: 2023-10-06T21:49:43.679+0000 7fc3cd018700 -1 mds.c *** got signal Terminated *** 2023-10-06T21:49:45.490 INFO:journalctl@ceph.mds.c.smithi146.stdout:Oct 06 21:49:45 smithi146 bash[152044]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-c 2023-10-06T21:49:45.522 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-06T21:49:45.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:45.607 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-06T21:49:45.607 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-06T21:49:45.607 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.f 2023-10-06T21:49:45.798 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:49:45 smithi146 systemd[1]: Stopping Ceph mds.f for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:46.549 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:49:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:49:46.112+0000 7f36f5eac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:46.549 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:49:46 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f[139487]: 2023-10-06T21:49:46.112+0000 7f36f5eac700 -1 mds.f *** got signal Terminated *** 2023-10-06T21:49:51.257 INFO:journalctl@ceph.mds.f.smithi146.stdout:Oct 06 21:49:50 smithi146 bash[152190]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-f 2023-10-06T21:49:51.282 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-06T21:49:51.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:51.363 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-06T21:49:51.364 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-06T21:49:51.364 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.i 2023-10-06T21:49:51.548 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:49:51 smithi146 systemd[1]: Stopping Ceph mds.i for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:52.298 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:49:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-i[140181]: 2023-10-06T21:49:51.860+0000 7f1201c2b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:52.299 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:49:51 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-i[140181]: 2023-10-06T21:49:51.860+0000 7f1201c2b700 -1 mds.i *** got signal Terminated *** 2023-10-06T21:49:56.548 INFO:journalctl@ceph.mds.i.smithi146.stdout:Oct 06 21:49:56 smithi146 bash[152336]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-i 2023-10-06T21:49:56.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-06T21:49:56.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:49:56.715 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-06T21:49:56.716 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-06T21:49:56.716 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.l 2023-10-06T21:49:57.048 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:49:56 smithi146 systemd[1]: Stopping Ceph mds.l for 9b5d7b0a-648c-11ee-8db5-212e2dc638e7... 2023-10-06T21:49:57.548 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:49:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-l[139131]: 2023-10-06T21:49:57.209+0000 7fc623ae8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:49:57.549 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:49:57 smithi146 ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-l[139131]: 2023-10-06T21:49:57.210+0000 7fc623ae8700 -1 mds.l *** got signal Terminated *** 2023-10-06T21:50:02.049 INFO:journalctl@ceph.mds.l.smithi146.stdout:Oct 06 21:50:01 smithi146 bash[152475]: ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7-mds-l 2023-10-06T21:50:02.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b5d7b0a-648c-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-06T21:50:02.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:50:02.175 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-06T21:50:02.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:50:02.587 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:50:06.014 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:50:06.419 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:50:09.637 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:50:10.036 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:50:13.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:50:13.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:50:13.502 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:50:13.573 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-06T21:50:13.575 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi031/crash 2023-10-06T21:50:13.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:50:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:50:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:50:13.673 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi142/crash 2023-10-06T21:50:13.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:50:13.774 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:50:13.774 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:50:13.778 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi146/crash 2023-10-06T21:50:13.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:50:13.910 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:50:13.910 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:50:13.912 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-06T21:50:13.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-06T21:50:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:grep: /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-06T21:50:14.001 INFO:tasks.cephadm:Compressing logs... 2023-10-06T21:50:14.002 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:50:14.007 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:50:14.012 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:50:14.078 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T21:50:14.078 INFO:teuthology.orchestra.run.smithi031.stderr:: No such file or directory 2023-10-06T21:50:14.081 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-06T21:50:14.082 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-06T21:50:14.083 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-06T21:50:14.084 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-06T21:50:14.085 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-06T21:50:14.086 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-06T21:50:14.087 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T21:50:14.087 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2023-10-06T21:50:14.088 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-06T21:50:14.088 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-06T21:50:14.089 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:50:14.090 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-06T21:50:14.092 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-06T21:50:14.093 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-10-06T21:50:14.093 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T21:50:14.093 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2023-10-06T21:50:14.094 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-06T21:50:14.098 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-06T21:50:14.099 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.log: 94.6% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-06T21:50:14.099 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-06T21:50:14.099 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-06T21:50:14.100 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-06T21:50:14.100 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-06T21:50:14.101 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-06T21:50:14.101 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-06T21:50:14.101 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-06T21:50:14.102 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-06T21:50:14.102 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-06T21:50:14.103 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-06T21:50:14.103 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-06T21:50:14.104 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-06T21:50:14.104 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:50:14.105 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-06T21:50:14.105 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-06T21:50:14.106 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-06T21:50:14.106 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-06T21:50:14.107 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-06T21:50:14.107 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:50:14.108 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-06T21:50:14.108 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-06T21:50:14.111 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log: /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.l.log: 92.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-06T21:50:14.118 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-06T21:50:14.151 INFO:teuthology.orchestra.run.smithi031.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:50:14.269 INFO:teuthology.orchestra.run.smithi146.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:50:14.295 INFO:teuthology.orchestra.run.smithi031.stderr: 91.0% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:50:14.942 INFO:teuthology.orchestra.run.smithi142.stderr: 90.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:50:15.145 INFO:teuthology.orchestra.run.smithi146.stderr: 90.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:50:15.149 INFO:teuthology.orchestra.run.smithi146.stderr: 91.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-06T21:50:15.350 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:50:15.640 INFO:teuthology.orchestra.run.smithi031.stderr: 89.7% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-06T21:50:16.891 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-06T21:50:20.468 INFO:teuthology.orchestra.run.smithi031.stderr: 90.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-06T21:50:20.875 INFO:teuthology.orchestra.run.smithi031.stderr: 94.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-06T21:50:22.793 INFO:teuthology.orchestra.run.smithi031.stderr: 94.7% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-06T21:50:24.407 INFO:teuthology.orchestra.run.smithi142.stderr: 94.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-06T21:50:25.231 INFO:teuthology.orchestra.run.smithi146.stderr: 94.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-06T21:50:25.816 INFO:teuthology.orchestra.run.smithi146.stderr: 95.1% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-06T21:50:45.624 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-06T21:50:45.965 INFO:teuthology.orchestra.run.smithi031.stderr: 90.6% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-06T21:50:51.207 INFO:teuthology.orchestra.run.smithi031.stderr: 90.6% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-06T21:50:52.442 INFO:teuthology.orchestra.run.smithi031.stderr: 90.7% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-06T21:50:56.292 INFO:teuthology.orchestra.run.smithi146.stderr: 90.5% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-06T21:51:01.257 INFO:teuthology.orchestra.run.smithi146.stderr: 90.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-06T21:51:03.557 INFO:teuthology.orchestra.run.smithi142.stderr: 91.2% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-06T21:51:04.037 INFO:teuthology.orchestra.run.smithi146.stderr: 90.4% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-06T21:51:05.845 INFO:teuthology.orchestra.run.smithi146.stderr: 90.5% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-06T21:51:05.879 INFO:teuthology.orchestra.run.smithi142.stderr: 90.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-06T21:51:11.614 INFO:teuthology.orchestra.run.smithi142.stderr: 90.7% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-06T21:51:21.468 INFO:teuthology.orchestra.run.smithi142.stderr: 90.8% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-06T21:53:12.145 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-06T21:53:17.139 INFO:teuthology.orchestra.run.smithi146.stderr: 94.0% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-06T21:53:17.142 INFO:teuthology.orchestra.run.smithi146.stderr: 2023-10-06T21:53:17.143 INFO:teuthology.orchestra.run.smithi146.stderr:real 3m3.090s 2023-10-06T21:53:17.143 INFO:teuthology.orchestra.run.smithi146.stderr:user 5m24.352s 2023-10-06T21:53:17.143 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m36.074s 2023-10-06T21:53:45.746 INFO:teuthology.orchestra.run.smithi142.stderr: 94.1% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-06T21:53:59.702 INFO:teuthology.orchestra.run.smithi142.stderr: 94.1% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-06T21:55:50.254 INFO:teuthology.orchestra.run.smithi142.stderr: 95.3% -- replaced with /var/log/ceph/9b5d7b0a-648c-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-06T21:55:50.317 INFO:teuthology.orchestra.run.smithi142.stderr: 2023-10-06T21:55:50.317 INFO:teuthology.orchestra.run.smithi142.stderr:real 5m36.267s 2023-10-06T21:55:50.317 INFO:teuthology.orchestra.run.smithi142.stderr:user 7m4.028s 2023-10-06T21:55:50.317 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m58.510s 2023-10-06T21:56:14.005 INFO:teuthology.orchestra.run.smithi031.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-10-06T21:56:14.005 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-10-06T21:56:14.005 INFO:teuthology.orchestra.run.smithi031.stderr:real 5m59.954s 2023-10-06T21:56:14.006 INFO:teuthology.orchestra.run.smithi031.stderr:user 6m8.269s 2023-10-06T21:56:14.006 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m49.570s 2023-10-06T21:56:14.006 INFO:tasks.cephadm:Archiving logs... 2023-10-06T21:56:14.007 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi031/log 2023-10-06T21:56:14.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:57:13.400 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi142/log 2023-10-06T21:57:13.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:58:13.397 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi146/log 2023-10-06T21:58:13.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:59:00.985 INFO:tasks.cephadm:Removing cluster... 2023-10-06T21:59:00.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:59:01.726 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:59:02.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:59:03.214 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:59:04.957 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:59:05.952 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 9b5d7b0a-648c-11ee-8db5-212e2dc638e7 2023-10-06T21:59:07.737 INFO:tasks.cephadm:Removing cephadm ... 2023-10-06T21:59:07.737 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:59:07.786 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:59:07.835 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:59:07.880 INFO:tasks.cephadm:Teardown complete 2023-10-06T21:59:07.880 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T21:59:07.891 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-06T21:59:07.892 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T21:59:07.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:59:07.897 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:59:07.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:59:08.138 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T21:59:08.149 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T21:59:08.149 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:59:08.154 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:59:08.159 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:59:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T21:59:08.198 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2023-10-06T21:59:08.200 INFO:teuthology.orchestra.run.smithi146.stderr:bash: ntpq: command not found 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 -419us[ -419us] +/- 90ms 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 +284us[ +313us] +/- 70ms 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +170us[ +170us] +/- 89ms 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:59:08.217 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:59:08.218 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-10-06T21:59:08.218 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 -433us[ -433us] +/- 90ms 2023-10-06T21:59:08.218 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 +249us[ +244us] +/- 70ms 2023-10-06T21:59:08.218 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +147us[ +147us] +/- 89ms 2023-10-06T21:59:08.218 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:59:08.220 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:59:08.220 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T21:59:08.220 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 -430us[ -420us] +/- 90ms 2023-10-06T21:59:08.221 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 +249us[ +260us] +/- 70ms 2023-10-06T21:59:08.221 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +143us[ +154us] +/- 89ms 2023-10-06T21:59:08.221 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:59:08.222 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T21:59:08.234 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T21:59:08.234 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T21:59:08.248 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:59:08.541 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:59:08.791 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:59:09.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:59:09.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:09.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:59:09.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:09.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:59:09.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:09.273 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T21:59:09.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T21:59:09.296 INFO:teuthology.task.internal:Duration was 3962.603919 seconds 2023-10-06T21:59:09.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T21:59:09.307 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T21:59:09.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:59:09.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:59:09.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:59:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:59:09.461 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:59:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:59:09.802 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T21:59:09.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T21:59:09.802 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:59:09.864 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2023-10-06T21:59:09.865 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:59:09.937 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2023-10-06T21:59:09.937 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:59:10.012 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T21:59:10.012 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:59:10.017 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:59:10.022 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:59:10.137 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T21:59:10.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:59:10.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:59:10.147 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:59:10.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T21:59:10.334 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T21:59:10.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:59:10.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:59:10.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:59:10.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T21:59:10.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.580 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.653 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-10-06T21:59:10.660 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2023-10-06T21:59:10.672 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2023-10-06T21:59:10.721 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:10.763 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:10.805 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:59:10.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:59:10.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T21:59:10.862 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T21:59:10.864 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi031 2023-10-06T21:59:10.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:59:10.954 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi142 2023-10-06T21:59:10.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:59:11.037 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652/remote/smithi146 2023-10-06T21:59:11.037 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:59:11.121 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T21:59:11.121 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:59:11.126 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:59:11.131 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:59:11.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T21:59:11.191 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T21:59:11.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T21:59:11.205 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T21:59:11.205 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:59:11.210 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:59:11.215 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:59:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:59 /home/ubuntu/cephtest 2023-10-06T21:59:11.253 INFO:teuthology.orchestra.run.smithi142.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:59 /home/ubuntu/cephtest 2023-10-06T21:59:11.259 INFO:teuthology.orchestra.run.smithi146.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:59 /home/ubuntu/cephtest 2023-10-06T21:59:11.263 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T21:59:11.276 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T21:59:11.294 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-06T21:59:11.294 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T21:59:11.324 DEBUG:teuthology.nuke:shortname: smithi031 2023-10-06T21:59:11.324 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:59:11.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.070021', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:59:11.353 DEBUG:teuthology.nuke:shortname: smithi142 2023-10-06T21:59:11.353 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:59:11.376 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.072660', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:59:11.381 DEBUG:teuthology.nuke:shortname: smithi146 2023-10-06T21:59:11.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:59:11.406 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:43:22.071680', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:59:11.416 INFO:teuthology.orchestra.console.smithi031:Power off 2023-10-06T21:59:11.416 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:59:11.438 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-10-06T21:59:11.438 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:11.457 INFO:teuthology.orchestra.console.smithi142:Power off 2023-10-06T21:59:11.458 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:59:11.469 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T21:59:11.480 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2023-10-06T21:59:11.480 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:11.496 INFO:teuthology.orchestra.console.smithi146:Power off 2023-10-06T21:59:11.497 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:59:11.506 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-10-06T21:59:11.518 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-10-06T21:59:11.519 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:11.545 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-10-06T21:59:15.469 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:15.507 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:15.546 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:15.580 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T21:59:15.621 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-10-06T21:59:15.656 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-10-06T21:59:19.584 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:19.622 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:19.656 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:19.694 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T21:59:19.732 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2023-10-06T21:59:19.767 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-10-06T21:59:19.833 INFO:teuthology.orchestra.console.smithi142:Power off completed 2023-10-06T21:59:19.868 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-10-06T21:59:23.698 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:59:23.808 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-06T21:59:23.908 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-10-06T21:59:24.013 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3962.603919029236 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c mon.b-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c mon.c-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c owner: scheduled_rishabh@teuthology status: fail success: false 2023-10-06T21:59:24.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T21:59:24.100 INFO:teuthology.run:FAIL